Kasper,

I think this is somewhat close to another thing I am describing here: https://github.com/svenvc/NeoCSV/issues/20

The problem with extending NeoCSV endlessly is that some of the things we need with "real-life" CSV files is the fact that they are not CSV files at all, so it is hard to tell if it is a good idea to litter NeoCSV with methods for edge cases that have literally to do with the fact that part of our files are not CSV at all...

I somehow have the feeling some of what we need is a subclass of Stream that knows about constraints like "a line-break is only the end of a record if it is not part of a quoted field". So I am somewhat torn between wanting that stuff in NeoCSV and not wanting to mix csv parsing with handling of stupid ideas people have when exporting data in some CSV-like file.

Maybe it is a good idea to collect a few concepts that have been mentioned in these threads:

Sometimes we want to skip lines (header, footer) without interpreting their contents and without any side effects (not CSV-compliant)
Sometimes we want to ignore "additional data" after the end of a defined number of columns (not CSV-compliant)
Sometimes we need to know which line/column couldn't be parsed correctly (related to CSV and non-CSV)
A plus would be if we could add the column name to the error message like in "The column 'amount' in line 34 cannot be interpreted as monetary amount" - but this is surely quite some work!
Sometimes we want to interpret columns by the column names in the header line (which may or may not be the first line of the file, only the former being CSV-compliant)

Of course this all doesn't mean I am not a fan of NeoCSV. It is well-written, works very well for "real" CSV and performs very well for my use cases. Most of the things we are talking about here are problems that arise when a CSV-file is not a CSV-file...

Joachim

Am 22.01.21 um 07:42 schrieb Kasper Osterbye:

As it happened, I ran into the exact same scenario as Joachim just the other day,
that is, the external provider of my csv had added some new columns. In my case

manifested itself in an error that an integer field was not an integer (because new

columns were added in the middle).

Reading through this whole thread leaves me with the feeling that no matter what Sven

adds, there is still a risk for error. Nevertheless, my suggestion would be to add a��

functionality to #skipHeaders, or make a sister method:��

#assertAndSkipHeaders: numberOfColumns onFailDo: aBlock given the actual number of headers

That would give me a way to handle the error up front.��

This will only be interesting if your data has headers of cause.

Thanks for NeoCSV which I use all the time!

Best,

Kasper��

-- 
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel          mailto:jtuchel@objektfabrik.de
Fliederweg 1                         http://www.objektfabrik.de
D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1