Continuing on... (The kids came in and I sent the last post off before I finished :) 5. Just dreaming... but since 80% of my time using NeoCSV was reviewing the data, transferring field headings to camelCase for the class definition, typing in each #addField: method. and since a lot of csv files have headings on the first line, something like a class NeoCSVGenerator that read the headings to create a class with matching instance variables would be really nice. In the first instance it could just output a string for the user to be manually copy/execute to the right location. Maybe later push it through the refactoring browser. For example.... ---- csvgen := NeoCSVGenerator createclass: MotorData on: csvStream csvgen classDefinition inspect. csvgen methodDefinition inspect. ---- #methodDefinition could read in the whole sample file and add comments for each field line showing the Integer/Float/blank statistics for to assist with decision making. ---- addField: #horsePower: ; "100 floats, 3 integers, 2 blanks" ---- In the new year (after I have handed in my thesis), you might bump me to add this myself (if you are interested but hadn't done it already). cheers -ben Ben Coman wrote:
Thanks Sven... and also thanks very much for your NeoCSV package [1] [2]. Coincidently my original question was related to practicing how to use NeoCSV to read some tab-separated data. A bit of feedback on NeoCSV...
1. Very easy to understand and I loved the #recordClass feature returning a model object.
2. Nicely documented [1]. Also perhaps mention the repository location and also that it can be found under Pharo Tool > Configuration Browser. 3. My data records had blanks in a lot of fields (eg data1,data2,,data4) so that #addFloatField: was failing. So I extended with the following method... ---- addFloatField: accessor ifFail: failBlock self addField: accessor converter: [ :string | Float readFrom: string ifFail: failBlock ] ---- Perhaps you could add similiar, as well as a default 'failBlock' (maybe one per Integer/Float)
4. Before I added (3.), the debugger would open about eight levels deeper than required to identify the culprit in the file. It would be useful if a custom Exception could be generated that held the line and accessor that was being processed (not that I'm know anything about Exceptions to know if that would be possible). That might be a good default 'failBlock'.
[1] https://github.com/svenvc/docs/blob/master/neo/neo-csv-paper.md [2] http://forum.world.st/ANN-NeoCSV-td4636208.html [3] http://mc.stfx.eu/Neo
Btw, based on your response to my original post, while I was playing in Workspace I ended up using it like this.... ----change test file source--- csvStream close. csvStream := FileStream readOnlyFileNamed: 'C:\#ENG8002\Test Data\Motors\motors.TXT'.
----iteratively-build-and-test-field-list---- csvStream reset. csvStream nextLine. "ignore headers" reader := (NeoCSVReader on: csvStream) separator: Character tab ; recordClass: LEMotorData . addIntegerField: #idMotor: ifFail: [ nil ] ; addFloatField: #hp: ifFail: [ nil ] ; ...etc reader upToEnd explore. ----etc----
Thanks again. cheers -ben
Sven Van Caekenberghe wrote:
Ben,
On 03 Nov 2012, at 03:23, Ben Coman <btc@openInWorld.com> wrote:
I am refereshing my familiarity with streams by looking at PharoByExample1 which says... "You have to remember that each time you open a stream on a ï¬le, you have to close it too."
Does that mean if in Workspace I iteratively develop some code to process a stream, so that I am continually running... myStream := FileStream readOnlyFileNamed: 'C:\test.txt. ...so that 'myStream' is overwritten each time such that I lose reference to the previous stream, and I am no longer able to send #close to it, am I creating a memory leak in my image ?
My prior assumption has been that once 'myStream' drops the reference drops to the previous stream, garbage collection would take care of 'everything'.
cheers -ben
Yes, external resources need to be closed, even though finialization (the garbage collector telling an object that it is about to become garbage) will try to clean up as well but it might take some time.
The proper idiom to use is
FileStream readOnlyFileNamed: 'C:\test.txt' do: [ :stream | ⦠].
or in Pharo 2.0
'C:\test.txt' asFileReference readStreamDo: [ :stream | ⦠]. 'C:\test.txt' asFileReference writeStreamDo: [ :stream | ⦠].
HTH,
Sven
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill