Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve candidate error message and relax constraint of rfc_4180 = false on quotes #15371

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Improve candidate error message and relax constraint of rfc_4180 = false on quotes #15371

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
dce0a8e
Making sure the csv reader is robust against this sneaky broken line
pdet Dec 16, 2024
a8610df
This test only makes sense with quotes set
pdet Dec 16, 2024
5ad0482
simplify this test
pdet Dec 16, 2024
81b0dd1
Also make quotes flexible at end of file
pdet Dec 17, 2024
8679ce2
New state to handle borked unescaped quotes
pdet Dec 17, 2024
2f63c27
More tests and fixes
pdet Dec 17, 2024
8e29a8a
wip
pdet Dec 18, 2024
c9683a3
Still have to do something about escape being defined
pdet Dec 19, 2024
3979aef
cleanup
pdet Dec 19, 2024
4531b8b
Merge remote-tracking branch 'origin/main' into hits_broken_line
pdet Dec 24, 2024
b065dc2
these shuold be statement maybe
pdet Dec 24, 2024
89f138b
Add windows test
pdet Dec 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
these shuold be statement maybe
  • Loading branch information
@pdet
pdet committed Dec 24, 2024
commit b065dc287a083686972895ba21bfae0446b893d8
75 changes: 37 additions & 38 deletions test/sql/copy/csv/test_afl.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,104 +5,103 @@
statement ok
PRAGMA enable_verification

statement error
statement maybe
FROM read_csv('data/csv/afl/1.csv', force_not_null=012%0);
----
Invalid value for 'force_not_null' paramenter

statement error
statement maybe
FROM read_csv('data/csv/afl/2.csv', max_line_size=-9151315542319464311);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0

statement error
statement maybe
FROM read_csv('data/csv/afl/3.csv', max_line_size=-1003718790012071149, ignore_errors=true);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0

statement error
statement maybe
FROM read_csv('data/csv/afl/4.csv', max_line_size=-432344490485710328, all_varchar=false);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0

statement error
statement maybe
FROM read_csv('data/csv/afl/5.csv', max_line_size=-68088296696312078);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0

statement error
statement maybe
FROM read_csv('data/csv/afl/6.csv', buffer_size=42);
----
It was not possible to automatically detect the CSV Parsing dialect/types

statement error
statement maybe
FROM read_csv('data/csv/afl/7.csv', buffer_size=42);
----
Value with unterminated quote found.

statement error
statement maybe
FROM read_csv('data/csv/afl/8.csv', buffer_size=42);
----
Invalid unicode (byte sequence mismatch) detected.

statement error
statement maybe
FROM read_csv('data/csv/afl/9.csv', buffer_size=42);
----
It was not possible to automatically detect the CSV Parsing dialect/types

statement error
statement maybe
FROM read_csv('data/csv/afl/10.csv', buffer_size=42);
----
It was not possible to automatically detect the CSV Parsing dialect/types

statement error
statement maybe
FROM read_csv('data/csv/afl/11.csv', max_line_size=-7987305294711008699, rejects_limit=42);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0

statement ok
statement maybe
FROM read_csv('data/csv/afl/12.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/13.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/14.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/15.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/16.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/17.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/18.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/19.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/20.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/21.csv', rejects_table='d');
----

statement ok
statement maybe
FROM read_csv('data/csv/afl/22.csv', rejects_table='d');
----

statement error
statement maybe
FROM read_csv('data/csv/afl/23.csv', buffer_size=42);
----
Error when sniffing file "data/csv/afl/23.csv".

statement ok
statement maybe
FROM read_csv('data/csv/afl/24.csv', buffer_size=26, delim=';', header=true, allow_quoted_nulls=true, allow_quoted_nulls=true);
----

statement error
statement maybe
FROM read_csv('data/csv/afl/25.csv', buffer_size=734771105608237082, max_line_size=-8825501086615982989, allow_quoted_nulls=true);
----
Invalid value for MAX_LINE_SIZE parameter: it cannot be smaller than 0