nextRow

| row |

row := OrderedCollection new.

stream skipSeparators.

[self atEndOfLine]

whileFalse: [row add: self nextValue].

"stream skip: -1."

stream next = $, ifTrue: [row add: ''].

^ row

This is how I test it:

stream := FileStream readOnlyFileNamed: 'Data.csv'.

[(CSVParser on: stream)�

useDelimiter: $;;�

rows ] ensure: [stream close].

I attach the bigger data file. The shorter one can be produced by deleting rows.

�

2011/11/16 Sven Van Caekenberghe <sven@beta9.be>

Dennis,

On 16 Nov 2011, at 19:30, Dennis Schetinin wrote:

> Hi!
>
> Parsing a UTF-8 CSV file with Russian symbols (using CSVParser), I found it really slow. It takes about 5 seconds to read a 53Kb file (136 lines) and several minutes to parse a 3.4Mb file (about 8K lines).
>
> Profiler shows that nearly all the time is eaten by WideString >> at:put: which is invoked from WriteStream>>nextPut:. If I'm not mistaken, this means that <primitive: 66> there does not work in this case� So, to be short: is there a simple way to improve the performance? TIA
>
> --
> Dennis Schetinin

I am interested in this problem.

Would it be possible to provide a test file (not that I can read Russian) and your test code ?

I would like to try this myself.

Sven

--
Dennis Schetinin