Re: [Pharo-users] Issue with NeoCSVReader
I guess Seb is right. The file cannot be read by Excel as well. So without having analysed this any further, it seems to me like the number of fields differs from row to row. NeoCSV does not handle this kind of "edge case". By defintion, each row in CSV has the same number of fields - even if you could possibly argue exceptions to this rule could be handled. OTOH, what does a missing field in a row usually mean? Is it the last one? One from the middle? is so, which one? So I think you should first check if the file is valid csv and if not, how you can get a valid one. If not, I guess you have to roll your own parser that foolows the "omission rules" of that file format. Joachim
On Wed, Jun 29, 2016 at 3:13 PM, Joachim Tuchel <jtuchel@objektfabrik.de> wrote:
I guess Seb is right. The file cannot be read by Excel as well. So without having analysed this any further, it seems to me like the number of fields differs from row to row. NeoCSV does not handle this kind of "edge case". By defintion, each row in CSV has the same number of fields - even if you could possibly argue exceptions to this rule could be handled. OTOH, what does a missing field in a row usually mean? Is it the last one? One from the middle? is so, which one?
Your questions are significant. The standard library should not be complicated by trying to handle undefined missing fields from a row. Such a file is simply invalid. cheers -ben
So I think you should first check if the file is valid csv and if not, how you can get a valid one. If not, I guess you have to roll your own parser that foolows the "omission rules" of that file format.
Joachim
Ben,
Ben Coman <btc@openinworld.com> hat am 29. Juni 2016 um 09:40 geschrieben: Your questions are significant. The standard library should not be complicated by trying to handle undefined missing fields from a row. Such a file is simply invalid. cheers -ben
I fully agree, I just wanted to give an example of questions that are way more difficult to answer than it might seem in the first place. I wanted to make an argument for not blaming NeoCSV for being unable to parse a file like that. I am surely not asking for NeoCSV to do some magic on invalid CSV files. It would be wrong in at least 50% of the cases anyways, no matter what kind of assumptions it would make. Joachim
Thanks for all your comments Alexandre
On Jun 29, 2016, at 3:45 AM, Joachim Tuchel <jtuchel@objektfabrik.de> wrote:
Ben,
Ben Coman <btc@openinworld.com> hat am 29. Juni 2016 um 09:40 geschrieben: Your questions are significant. The standard library should not be complicated by trying to handle undefined missing fields from a row. Such a file is simply invalid. cheers -ben
I fully agree, I just wanted to give an example of questions that are way more difficult to answer than it might seem in the first place. I wanted to make an argument for not blaming NeoCSV for being unable to parse a file like that. I am surely not asking for NeoCSV to do some magic on invalid CSV files. It would be wrong in at least 50% of the cases anyways, no matter what kind of assumptions it would make.
Joachim
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On 29 Jun 2016, at 09:45, Joachim Tuchel <jtuchel@objektfabrik.de> wrote:
Ben,
Ben Coman <btc@openinworld.com> hat am 29. Juni 2016 um 09:40 geschrieben: Your questions are significant. The standard library should not be complicated by trying to handle undefined missing fields from a row. Such a file is simply invalid. cheers -ben
I fully agree, I just wanted to give an example of questions that are way more difficult to answer than it might seem in the first place. I wanted to make an argument for not blaming NeoCSV for being unable to parse a file like that. I am surely not asking for NeoCSV to do some magic on invalid CSV files. It would be wrong in at least 50% of the cases anyways, no matter what kind of assumptions it would make.
Joachim
So yes, CSV is only defined for records that are all the same size. NeoCSVReader can however deal with shorter records. The first record (or header, or convertors) define the number of fields for the whole file. Obviously, only empty fields at the end are supported. Here is an example: (NeoCSVReader on: '1,2,3\5\3,2,1' withCRs readStream) emptyFieldValue: #empty; upToEnd. => #(#('1' '2' '3') #('5' #empty #empty) #('3' '2' '1')) The second line/record contains only one value, empty ones are added automatically. The #emptyFieldValue is optional and defaults to nil. Sven
participants (4)
-
Alexandre Bergel -
Ben Coman -
Joachim Tuchel -
Sven Van Caekenberghe