July 2018 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Pharo 7 file streams guideline

by Damien Pollet

On Wed, 25 Jul 2018 at 13:48, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 25 Jul 2018, at 13:39, Damien Pollet <damien.pollet+pharo(a)gmail.com> > wrote: > > Related issue: command line arguments come from VM system attributes as > ByteStringsâ¦ and thus interpreted as iso-8859-1, which is incorrect in most > cases nowadays, even though it seems to work as long as you only use ASCII. > Decoding them is easy enough, but it requires two copies (asByteString > utf8Decoded) > > Yes this is a really big issue. Anything coming in as command line arg or > environment variable (or clipboard) is in a basically unknown OS determined > encoding. I would assume/hope the UTF-8 is the sensible default today, but > apparently not. And it is hard to find a cross platform solution. > My point here was that it would make more sense for those to be passed into the image as ByteArrays, revealing the fact that their encoding is unknown. Currently the bytes are correct, but since they've been shoved into ByteStrings by the VM, the characters will be wrong unless your system happens to be using Latin 1. I suppose we can either have a setting for decoding (since it's pretty much arbitrary), or heuristics like checking LC_CTYPE or whatever. Pablo mentioned the Locale class, but it doesn't seem to detect anything correct from the environment.

July 25, 2018

Re: [Pharo-users] Pharo 7 file streams guideline

by Sven Van Caekenberghe

> On 25 Jul 2018, at 13:39, Damien Pollet <damien.pollet+pharo(a)gmail.com> wrote: > > On Tue, 24 Jul 2018 at 11:39, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > On 23 Jul 2018, at 12:07, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > So, > > Stdio stdout > > should return return a character write stream with UTF-8 encoding while > > Stdio binaryStdout > > should be the lower level binary one. > This would be more in line with the other streams. > A non-UTF-8 encoding can be used as per Pavel's example. > > +1 > > I didn't suggest this earlier because it isn't backward compatible. But I do think it is the better solution. > > +2 > > I had a look at Stdio recently for Clap. The current implementation with Stdio stdout returning the binary stream is a bit confusing, but at least you can wrap it. > The above proposition with an explicit binaryStdout for the lower level uncommon case would be much clearer indeed. > > Related issue: command line arguments come from VM system attributes as ByteStringsâ¦ and thus interpreted as iso-8859-1, which is incorrect in most cases nowadays, even though it seems to work as long as you only use ASCII. Decoding them is easy enough, but it requires two copies (asByteString utf8Decoded) Yes this is a really big issue. Anything coming in as command line arg or environment variable (or clipboard) is in a basically unknown OS determined encoding. I would assume/hope the UTF-8 is the sensible default today, but apparently not. And it is hard to find a cross platform solution. We've had serious issues already with this, like $HOME set to a non-ASCII path that then breaks almost everything.

July 25, 2018

Re: [Pharo-users] Pharo 7 file streams guideline

by Damien Pollet

On Tue, 24 Jul 2018 at 11:39, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > On 23 Jul 2018, at 12:07, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> So, >> >> Stdio stdout >> >> should return return a character write stream with UTF-8 encoding while >> >> Stdio binaryStdout >> >> should be the lower level binary one. >> This would be more in line with the other streams. >> A non-UTF-8 encoding can be used as per Pavel's example. >> > > +1 > > I didn't suggest this earlier because it isn't backward compatible. But I > do think it is the better solution. > +2 I had a look at Stdio recently for Clap. The current implementation with Stdio stdout returning the binary stream is a bit confusing, but at least you can wrap it. The above proposition with an explicit binaryStdout for the lower level uncommon case would be much clearer indeed. Related issue: command line arguments come from VM system attributes as ByteStringsâ¦ and thus interpreted as iso-8859-1, which is incorrect in most cases nowadays, even though it seems to work as long as you only use ASCII. Decoding them is easy enough, but it requires two copies (asByteString utf8Decoded)

July 25, 2018

Re: [Pharo-users] OmniBase for Pharo 6

by marco

One serious problem left: OmniBaseTest>>testGC fails. Have a look at the 2 methods /ODBContainer>>lockDataFile "Dec, 2010 sas: this code was relying in evaluating a block that was happening only in win32 but not in *nix. So now I'm making it wait for the lock and then execute the code normally because it seems the intention was the same." | currentSpace defaultSpace | activeDataFile waitForAddingLock. currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile] ODBFile>>waitForLockAt: pos length: length whileWaitingDo: aBlock "Wait for lock at given position. Evaluate aBlock in each iteration." "Dec, 2010. sas: evaluating code in each iteration while waiting sounds weird but whatever. I think this was designed on the wrong assumption that the OS always will provide a false in the first try but what actually happens is that in win32 you always will get at least one false but in *nixes you don't do see that happening (which is totally reasonable). BEWARE of stupid code relying on this stupid assumption (I've already patched #lockDataFile because of this)." | startTime currentTime | (stream lockAt: pos length: length) ifFalse: [ startTime := Time totalSeconds. [stream lockAt: pos length: length] whileFalse: [ aBlock value. currentTime := Time totalSeconds. currentTime - startTime < self timeOutLength ifFalse: [ ODBLockNotification signal ifTrue: [startTime := currentTime] ifFalse: [^ODBCannotLock signal]]]] / So "sas" did a refactoring in 2010 because he/she thought it is stupid code. This code change breaks the GarbageCollection of Omnibase. I reverted the first method to the original code of the Omnibase author, David Gorisek: /lockDataFile | currentSpace defaultSpace | activeDataFile waitForAddingLockWhileWaitingDo: [currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile]] / I removed the comment of "sas" from the second method. It is wrong. The garbage collection works again and the test is green. GC is an important feature of Omnibase. It creates a new repository/database file and copies the data into the new file, compressing the data. It keeps your database compact and clean. If you don't run it (frequently) your db files keep on growing... Matias, your fixes are absolutely valid and should be published - together with the one above - in Esteban's repository. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

July 25, 2018

Dynabook learning model

by Hilaire

A critic view on the underneath learning model of the Dynabook: http://blog.drgeo.eu/post/2018/The-Dynabook-and-its-learning-models Hilaire -- Dr. Geo http://drgeo.eu

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Peter UhnÃ¡k

Ah, you are right that it might not be human-friendly. I don't want to rely on the order, but for (file) versioning it is quite valuable. (or rather changing the order can easily make versioning quite worthless). I can manually sort it, but it creates a lot of mess: (NeoJSONWriter on: stream) prettyPrint: true; for: Something do: [ :mapping | mapping mapInstVars: (mapping identifier instVarNames \ #(unwanted)) sorted ]; for: Otherthing do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames sorted ]; for: Point do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames sorted ]; nextPut: anObject. I'd much rather have the following: (NeoJSONWriter on: stream) prettyPrint: true; sorted: true; for: Something do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames \ #(unwanted) ]; mapInstVarsFor: Otherthing; mapInstVarsFor: Point; nextPut: anObject. or more fine-grained, although the above is imo nicer: (NeoJSONWriter on: stream) prettyPrint: true; for: Something do: [ :mapping | mapping mapInstVars: (mapping identifier instVarNames \ #(unwanted)) sorted ]; mapInstVarsFor: Otherthing sorted: true; mapInstVarsFor: Point sorted: true; nextPut: anObject. (or mapSortedInstVarsFor: or something).. Peter On Wed, Jul 25, 2018 at 11:20 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > From json.org > > "An object is an unordered set of name/value pairs." > > IOW, it is not in the spec, it would be dangerous to rely up it in any way > (not that you are suggesting that). It might look better sometimes, but > even then alphabetic sorting might not be the most human friendly or most > meaningful. > > Note that 'properties' in NeoJSONMapping is an OrderedCollection, so it > keeps it order. So if you add them in the right order, that would do it. > > > On 25 Jul 2018, at 11:12, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > > > > Hi, > > > > would it be possible to provide option to automatically sort instance > variables when serializing objects? > > > > Either always sorted > > > > NeoJSONObjectMapping>>writeObject: anObject on: jsonWriter > > jsonWriter writeMapStreamingDo: [ :jsonMapWriter | > > (properties sorted: #propertyName ascending) do: [ :each | > > each writeObject: anObject on: jsonMapWriter ] ] > > > > or put it behind some sortProperties boolean flag. (atm I don't see any > downsides in having sort always enabled) > > Speed ? > A false suggestion that is part of the spec ? > > > Thanks, > > Peter > > >

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Sven Van Caekenberghe

>From json.org "An object is an unordered set of name/value pairs." IOW, it is not in the spec, it would be dangerous to rely up it in any way (not that you are suggesting that). It might look better sometimes, but even then alphabetic sorting might not be the most human friendly or most meaningful. Note that 'properties' in NeoJSONMapping is an OrderedCollection, so it keeps it order. So if you add them in the right order, that would do it. > On 25 Jul 2018, at 11:12, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > > Hi, > > would it be possible to provide option to automatically sort instance variables when serializing objects? > > Either always sorted > > NeoJSONObjectMapping>>writeObject: anObject on: jsonWriter > jsonWriter writeMapStreamingDo: [ :jsonMapWriter | > (properties sorted: #propertyName ascending) do: [ :each | > each writeObject: anObject on: jsonMapWriter ] ] > > or put it behind some sortProperties boolean flag. (atm I don't see any downsides in having sort always enabled) Speed ? A false suggestion that is part of the spec ? > Thanks, > Peter

July 25, 2018

NeoJSON sorting instance variables/properties

by Peter UhnÃ¡k

Hi, would it be possible to provide option to automatically sort instance variables when serializing objects? Either always sorted NeoJSONObjectMapping>>writeObject: anObject on: jsonWriter jsonWriter writeMapStreamingDo: [ :jsonMapWriter | (properties sorted: #propertyName ascending) do: [ :each | each writeObject: anObject on: jsonMapWriter ] ] or put it behind some sortProperties boolean flag. (atm I don't see any downsides in having sort always enabled) Thanks, Peter

July 25, 2018

Re: [Pharo-users] NeoJSON ignore some instance variables

by Sven Van Caekenberghe

> On 25 Jul 2018, at 10:17, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > > Hi, > > what is the best way to ignore instance variables when serializing with NeoJSON? > > I figured out this so far... > > (NeoJSONWriter on: stream) > for: Something > do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames \ #(unwanted) ]; > nextPut: aSomething > > Thanks, > Peter That is a better and more beautiful way to do it then I would have come up with myself ;-)

July 25, 2018

NeoJSON ignore some instance variables

by Peter UhnÃ¡k

Hi, what is the best way to ignore instance variables when serializing with NeoJSON? I figured out this so far... (NeoJSONWriter on: stream) for: Something do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames \ #(unwanted) ]; nextPut: aSomething Thanks, Peter

July 25, 2018