Hi Ben, Thanks for all the feedback, I am happy that NeoCSV is useful for you. Regarding your problem, I tried a related but different solution. When a user specifies that a field should be an integer or a float, IMO you want an error in case the input cannot be parsed. But indeed, empty/missing fields are an exception. I implemented the default option to skip further processing empty fields. No configuration needed. Please have a look and tell me if that solves your use case. I committed the following: === Name: Neo-CSV-Core-SvenVanCaekenberghe.10 Author: SvenVanCaekenberghe Time: 4 November 2012, 8:12:54.095 pm UUID: e73487f9-57d1-40f1-bc0e-578c97c08721 Ancestors: Neo-CSV-Core-SvenVanCaekenberghe.9 changed #readNextRecord and #readNextRecordAsObject to not do anything if empty/missing fields are read; this also means that empty strings are not passed to converters or field accessors; this solves the problem reported by Ben Coman where missing integer/float fields gave errors === Name: Neo-CSV-Tests-SvenVanCaekenberghe.9 Author: SvenVanCaekenberghe Time: 4 November 2012, 8:13:54.008 pm UUID: ffaaa773-d3f6-4b10-adaa-bd2588af8eb5 Ancestors: Neo-CSV-Tests-SvenVanCaekenberghe.8 added #testEmptyConversions and #testEmptyConversionsTestObject; changed #readNextRecord and #readNextRecordAsObject to not do anything if empty/missing fields are read; this also means that empty strings are not passed to converters or field accessors; this solves the problem reported by Ben Coman where missing integer/float fields gave errors === Regards, Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill On 04 Nov 2012, at 03:32, Ben Coman <btc@openInWorld.com> wrote:
Thanks Sven... and also thanks very much for your NeoCSV package [1] [2]. Coincidently my original question was related to practicing how to use NeoCSV to read some tab-separated data. A bit of feedback on NeoCSV...
1. Very easy to understand and I loved the #recordClass feature returning a model object.
2. Nicely documented [1]. Also perhaps mention the repository location and also that it can be found under Pharo Tool > Configuration Browser. 3. My data records had blanks in a lot of fields (eg data1,data2,,data4) so that #addFloatField: was failing. So I extended with the following method... ---- addFloatField: accessor ifFail: failBlock self addField: accessor converter: [ :string | Float readFrom: string ifFail: failBlock ] ---- Perhaps you could add similiar, as well as a default 'failBlock' (maybe one per Integer/Float)
4. Before I added (3.), the debugger would open about eight levels deeper than required to identify the culprit in the file. It would be useful if a custom Exception could be generated that held the line and accessor that was being processed (not that I'm know anything about Exceptions to know if that would be possible). That might be a good default 'failBlock'.
[1] https://github.com/svenvc/docs/blob/master/neo/neo-csv-paper.md [2] http://forum.world.st/ANN-NeoCSV-td4636208.html [3] http://mc.stfx.eu/Neo
Btw, based on your response to my original post, while I was playing in Workspace I ended up using it like this.... ----change test file source--- csvStream close. csvStream := FileStream readOnlyFileNamed: 'C:\#ENG8002\Test Data\Motors\motors.TXT'.
----iteratively-build-and-test-field-list---- csvStream reset. csvStream nextLine. "ignore headers" reader := (NeoCSVReader on: csvStream) separator: Character tab ; recordClass: LEMotorData . addIntegerField: #idMotor: ifFail: [ nil ] ; addFloatField: #hp: ifFail: [ nil ] ; ...etc reader upToEnd explore. ----etc----
Thanks again. cheers -ben
Sven Van Caekenberghe wrote:
Ben,
On 03 Nov 2012, at 03:23, Ben Coman <btc@openInWorld.com> wrote:
I am refereshing my familiarity with streams by looking at PharoByExample1 which says... "You have to remember that each time you open a stream on a ï¬le, you have to close it too."
Does that mean if in Workspace I iteratively develop some code to process a stream, so that I am continually running... myStream := FileStream readOnlyFileNamed: 'C:\test.txt. ...so that 'myStream' is overwritten each time such that I lose reference to the previous stream, and I am no longer able to send #close to it, am I creating a memory leak in my image ?
My prior assumption has been that once 'myStream' drops the reference drops to the previous stream, garbage collection would take care of 'everything'.
cheers -ben
Yes, external resources need to be closed, even though finialization (the garbage collector telling an object that it is about to become garbage) will try to clean up as well but it might take some time.
The proper idiom to use is
FileStream readOnlyFileNamed: 'C:\test.txt' do: [ :stream | ⦠].
or in Pharo 2.0
'C:\test.txt' asFileReference readStreamDo: [ :stream | ⦠]. 'C:\test.txt' asFileReference writeStreamDo: [ :stream | ⦠].
HTH,
Sven
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill