July 2018 - Pharo-users - lists.pharo.org

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Peter UhnÃ¡k

> Actually I do not think that the above is that bad ;-) It is too explicit for me and easy to forget to do. :) > I guess you could add some extension methods to NeoJSONWriter, or subclass it - yourself. In the end, this is probably the most reasonable. And I can change some of the defaults too (like pretty printing & LF line ending). Thanks for the discussion! Peter On Wed, Jul 25, 2018 at 7:21 PM, PBKResearch <peter(a)pbkresearch.co.uk> wrote: > Just as a comment to Henrik, NeoJSON will already preserve the order of an > OrderedDictionary. When working with Stef on the HTML scraping booklet, I > decided to find a way to preserve the order of the selected items, partly > for aesthetic reasons, but mainly to demonstrate that the output matched > the > problem specification. I switched the collection of the output items to an > OrderedDictionary, not knowing whether it would work, and it came out as > expected. Building an OrderedDictionary of just the required instvars would > solve both the OP's problems, though perhaps less elegantly. > > Peter Kenny > > -----Original Message----- > From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Sven > Van Caekenberghe > Sent: 25 July 2018 17:49 > To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] NeoJSON sorting instance variables/properties > > > > > On 25 Jul 2018, at 17:54, Henrik-Nergaard <draagren(a)outlook.com> wrote: > > > > Hi Peter, > > > > This functionallity is already available in STON where it preserves > > the order when you use an OrderedDictionary. > > And was implemented for the usecase of dealing with fileout and > versioning. > > Perhaps the same could be added to NeoJson so that instead of some > > magic flag you provide an OrderedDictionary sorted as you would like > > and it would then preserve the order. > > You are right, using OrderedDictionaries would fix the order. > > But I don't think that is what he wants: to fix the order of arbitrary > objects being mapped. > > Even in STON the order of properties of objects is undefined (as it is in > JSON). > > > If you aim to use it to preserve metada for filetree then there is a > > patch for that which can be found at > > https://github.com/DraagrenKirneh/PharoUtilities/tree/master/FileTreeF > > ix > > > > Best regards, > > Henrik > > > > > > > > -- > > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > > > > > >

July 25, 2018

Re: [Pharo-users] OmniBase for Pharo 6

by PBKResearch

Marco Thanks for this. Just for curiosity, did you test it just on Windows, or on other OSs as well. The logic of the Dec 2010 fix was that David GorisekÂs original, being developed on Dolphin, could not be guaranteed to work except on Windows. I only use Windows, so it is just curiosity. BTW, the author of the Dec 2010 fix, ÂsasÂ, is Sebastian Sastre, so definitely ÂheÂ. Peter Kenny From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Matias Maretto Sent: 25 July 2018 13:34 To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] OmniBase for Pharo 6 Marco: thank you for this. A few weeks ago when I was trying to find the "lock" problem I saw this method but I did not dare to change it; and just as you said I been using OminBase with no problem, but never runned the Garbage Collector, Surely I was going to have a bigger problem in the future for not running the garbageCollect. I have just update this method on my project and tried both the garbageCollect and the reorganize method , both worked fine (win 7 32bits, later I am going to test on Win 10 64 bits). Thanks again. MatÃas. _____ De: Pharo-users <pharo-users-bounces(a)lists.pharo.org <mailto:pharo-users-bounces@lists.pharo.org> > en nombre de marco <marco(a)omeleon.de <mailto:marco@omeleon.de> > Enviado: miÃ©rcoles, 25 de julio de 2018 10:23 a. m. Para: pharo-users(a)lists.pharo.org <mailto:pharo-users@lists.pharo.org> Asunto: Re: [Pharo-users] OmniBase for Pharo 6 One serious problem left: OmniBaseTest>>testGC fails. Have a look at the 2 methods /ODBContainer>>lockDataFile "Dec, 2010 sas: this code was relying in evaluating a block that was happening only in win32 but not in *nix. So now I'm making it wait for the lock and then execute the code normally because it seems the intention was the same." | currentSpace defaultSpace | activeDataFile waitForAddingLock. currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile] ODBFile>>waitForLockAt: pos length: length whileWaitingDo: aBlock "Wait for lock at given position. Evaluate aBlock in each iteration." "Dec, 2010. sas: evaluating code in each iteration while waiting sounds weird but whatever. I think this was designed on the wrong assumption that the OS always will provide a false in the first try but what actually happens is that in win32 you always will get at least one false but in *nixes you don't do see that happening (which is totally reasonable). BEWARE of stupid code relying on this stupid assumption (I've already patched #lockDataFile because of this)." | startTime currentTime | (stream lockAt: pos length: length) ifFalse: [ startTime := Time totalSeconds. [stream lockAt: pos length: length] whileFalse: [ aBlock value. currentTime := Time totalSeconds. currentTime - startTime < self timeOutLength ifFalse: [ ODBLockNotification signal ifTrue: [startTime := currentTime] ifFalse: [^ODBCannotLock signal]]]] / So "sas" did a refactoring in 2010 because he/she thought it is stupid code. This code change breaks the GarbageCollection of Omnibase. I reverted the first method to the original code of the Omnibase author, David Gorisek: /lockDataFile | currentSpace defaultSpace | activeDataFile waitForAddingLockWhileWaitingDo: [currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile]] / I removed the comment of "sas" from the second method. It is wrong. The garbage collection works again and the test is green. GC is an important feature of Omnibase. It creates a new repository/database file and copies the data into the new file, compressing the data. It keeps your database compact and clean. If you don't run it (frequently) your db files keep on growing... Matias, your fixes are absolutely valid and should be published - together with the one above - in Esteban's repository. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> Smalltalk - Pharo Smalltalk Users | Mailing List Archive forum.world.st Pharo Smalltalk Users forum and mailing list archive. Pharo User Forum

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by PBKResearch

Just as a comment to Henrik, NeoJSON will already preserve the order of an OrderedDictionary. When working with Stef on the HTML scraping booklet, I decided to find a way to preserve the order of the selected items, partly for aesthetic reasons, but mainly to demonstrate that the output matched the problem specification. I switched the collection of the output items to an OrderedDictionary, not knowing whether it would work, and it came out as expected. Building an OrderedDictionary of just the required instvars would solve both the OP's problems, though perhaps less elegantly. Peter Kenny -----Original Message----- From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 25 July 2018 17:49 To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] NeoJSON sorting instance variables/properties > On 25 Jul 2018, at 17:54, Henrik-Nergaard <draagren(a)outlook.com> wrote: > > Hi Peter, > > This functionallity is already available in STON where it preserves > the order when you use an OrderedDictionary. > And was implemented for the usecase of dealing with fileout and versioning. > Perhaps the same could be added to NeoJson so that instead of some > magic flag you provide an OrderedDictionary sorted as you would like > and it would then preserve the order. You are right, using OrderedDictionaries would fix the order. But I don't think that is what he wants: to fix the order of arbitrary objects being mapped. Even in STON the order of properties of objects is undefined (as it is in JSON). > If you aim to use it to preserve metada for filetree then there is a > patch for that which can be found at > https://github.com/DraagrenKirneh/PharoUtilities/tree/master/FileTreeF > ix > > Best regards, > Henrik > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Sven Van Caekenberghe

> On 25 Jul 2018, at 17:54, Henrik-Nergaard <draagren(a)outlook.com> wrote: > > Hi Peter, > > This functionallity is already available in STON where it preserves the > order when you use an OrderedDictionary. > And was implemented for the usecase of dealing with fileout and versioning. > Perhaps the same could be added to NeoJson so that instead of some magic > flag you provide an OrderedDictionary sorted as you would like and it would > then preserve the order. You are right, using OrderedDictionaries would fix the order. But I don't think that is what he wants: to fix the order of arbitrary objects being mapped. Even in STON the order of properties of objects is undefined (as it is in JSON). > If you aim to use it to preserve metada for filetree then there is a patch > for that which can be found at > https://github.com/DraagrenKirneh/PharoUtilities/tree/master/FileTreeFix > > Best regards, > Henrik > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >

July 25, 2018

Pharo7, consistent image freeze/deadlock on snapshotAndQuit

by Johan Brichau

Hi, I ran into a freeze issue when trying to save and/or quit an image after loading Seaside3 and Zinc (not an infrequent combination :) The cause seems to be that terminating processes in a `shutDown` handler (called on image shutdown / snapshot) leads to a deadlock (somewhere). Both Comet and Zinc will send #terminate to a process when an image is quit or saved. If I disable one of those, the deadlock does not occur. I managed to reconstruct the issue with a simple example, attached. The attached Freeze class creates two processes (that loop with a Delay>>wait inside) in the `startUp` method (on image start). These processes are terminated by the shutdown method (called on image quit/save). However, when you try to quit and/or save the image, you will experience an image freeze. Iâm not familiar with the process scheduler, or anything else in this contextâ¦ so help would be appreciated while I try to dive further into it :) To reproduce: just file in the attached file in a Pharo7 image and try to quit. For reference, System Reporter output for my configuration. Any ideas? Cheers Johan Image ----- /Users/jbrichau/Pharo/images/Pharo 7.0 - 64bit (development version)/Pharo 7.0 - 64bit (development version).image Pharo7.0alpha Build information: Pharo-7.0+alpha.build.1131.sha.a46bfaef44ba209888c92fa0e3bd3a5e04a38c5d (64 Bit) Unnamed Virtual Machine --------------- /Users/jbrichau/Pharo/vms/70-x64/Pharo.app/Contents/MacOS/Pharo CoInterpreter VMMaker.oscog-eem.2401 uuid: 29232e0e-c9e3-41d8-ae75-519db862e02c Jun 28 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2401 uuid: 29232e0e-c9e3-41d8-ae75-519db862e02c Jun 28 2018 VM: 201806281256 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Thu Jun 28 14:56:30 2018 CommitHash: a8a1dc1 Plugins: 201806281256 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Mac OS X built on Jun 28 2018 13:07:33 UTC Compiler: 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31) VMMaker versionString VM: 201806281256 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Thu Jun 28 14:56:30 2018 CommitHash: a8a1dc1 Plugins: 201806281256 https://github.com/OpenSmalltalk/opensmalltalk-vm.git CoInterpreter VMMaker.oscog-eem.2401 uuid: 29232e0e-c9e3-41d8-ae75-519db862e02c Jun 28 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2401 uuid: 29232e0e-c9e3-41d8-ae75-519db862e02c Jun 28 2018

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Henrik-Nergaard

Hi Peter, This functionallity is already available in STON where it preserves the order when you use an OrderedDictionary. And was implemented for the usecase of dealing with fileout and versioning. Perhaps the same could be added to NeoJson so that instead of some magic flag you provide an OrderedDictionary sorted as you would like and it would then preserve the order. If you aim to use it to preserve metada for filetree then there is a patch for that which can be found at https://github.com/DraagrenKirneh/PharoUtilities/tree/master/FileTreeFix Best regards, Henrik -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

July 25, 2018

Re: [Pharo-users] NeoJSON sorting instance variables/properties

by Sven Van Caekenberghe

> On 25 Jul 2018, at 11:34, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > > Ah, you are right that it might not be human-friendly. > > I don't want to rely on the order, but for (file) versioning it is quite valuable. (or rather changing the order can easily make versioning quite worthless). Ah, OK, I understand the use case. > I can manually sort it, but it creates a lot of mess: > > (NeoJSONWriter on: stream) > prettyPrint: true; > for: Something > do: [ :mapping | mapping mapInstVars: (mapping identifier instVarNames \ #(unwanted)) sorted ]; > for: Otherthing > do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames sorted ]; > for: Point > do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames sorted ]; > nextPut: anObject. Actually I do not think that the above is that bad ;-) > I'd much rather have the following: > > (NeoJSONWriter on: stream) > prettyPrint: true; > sorted: true; > for: Something > do: [ :mapping | mapping mapInstVars: mapping identifier instVarNames \ #(unwanted) ]; > mapInstVarsFor: Otherthing; > mapInstVarsFor: Point; > nextPut: anObject. > > or more fine-grained, although the above is imo nicer: > > (NeoJSONWriter on: stream) > prettyPrint: true; > for: Something > do: [ :mapping | mapping mapInstVars: (mapping identifier instVarNames \ #(unwanted)) sorted ]; > mapInstVarsFor: Otherthing sorted: true; > mapInstVarsFor: Point sorted: true; > nextPut: anObject. > > (or mapSortedInstVarsFor: or something).. I guess you could add some extension methods to NeoJSONWriter, or subclass it - yourself. After some consideration I still think that adding sorting options would be confusing. Unless there would be some convention in other JSON tools for this issue, or a larger demand (more use cases). > Peter > > On Wed, Jul 25, 2018 at 11:20 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > From json.org > > "An object is an unordered set of name/value pairs." > > IOW, it is not in the spec, it would be dangerous to rely up it in any way (not that you are suggesting that). It might look better sometimes, but even then alphabetic sorting might not be the most human friendly or most meaningful. > > Note that 'properties' in NeoJSONMapping is an OrderedCollection, so it keeps it order. So if you add them in the right order, that would do it. > > > On 25 Jul 2018, at 11:12, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > > > > Hi, > > > > would it be possible to provide option to automatically sort instance variables when serializing objects? > > > > Either always sorted > > > > NeoJSONObjectMapping>>writeObject: anObject on: jsonWriter > > jsonWriter writeMapStreamingDo: [ :jsonMapWriter | > > (properties sorted: #propertyName ascending) do: [ :each | > > each writeObject: anObject on: jsonMapWriter ] ] > > > > or put it behind some sortProperties boolean flag. (atm I don't see any downsides in having sort always enabled) > > Speed ? > A false suggestion that is part of the spec ? > > > Thanks, > > Peter

July 25, 2018

Re: [Pharo-users] Using Icerberg programmatically.

by Arturo Zambrano

Hi Guille. thanks for the advise. I will use it carefully, and try to move to Pharo 7 soon. Arturo On Wed, Jul 25, 2018 at 5:02 AM Guillermo Polito <guillermopolito(a)gmail.com> wrote: > Hi Arturo, > > First, I would not recommend too much to use the version on Pharo 6.1. > That version is a preview version that is >6 months older than the new one > in Pharo 7. This does not mean you cannot write a script, but we do not > guarantee it will work in more recent version :). > > If you then settle for the version on Pharo7, something you have to also > care about is that while most of the basic API is stable enough, some > others will evolve in the near future. There are many missing features, and > we are trying to keep it as simple as possible so it can easily evolve... > > Just a note about what you want to do: Be careful with programmatic > pulling. A pull is a fetch + a merge. If you did not commit in your branch, > the merge will be a simple fast-forward and you will have no further > problems. But if you did commit, you will have a merge, with potentially > conflicts. And conflicts have so far to be manually resolved. > > Guille > > On Wed, Jul 25, 2018 at 9:51 AM Christophe Demarey < > christophe.demarey(a)inria.fr> wrote: > >> Hi Arthur, >> >> The starting point is the IceRepository. >> You can build you own repository with IceRepositoryCreator or get an >> existing one from IceRepository registry. >> Then on the repository you can call #pull. >> >> Christophe >> >> > Le 24 juil. 2018 Ã 20:09, Arturo Zambrano <arturo.zambrano(a)gmail.com> >> a Ã©crit : >> > >> > Hi All, >> > the new Iceberg is really cool. I'm using it in Pharo 6.1. >> > >> > Is there some documentation or script examples on how to use >> Iceberg programmatically? I would like to write an script to pull changes >> to an existing repository, before doing some other stuff. I can do it with >> the UI, but I would like to automatize it. Any pointer is welcome. >> > >> > Thanks in advance. >> > Arturo >> >> >> > > -- > > > > Guille Polito > > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - *http://www.cnrs.fr > <http://www.cnrs.fr>* > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 >

July 25, 2018

Re: [Pharo-users] Using Icerberg programmatically.

by Arturo Zambrano

Thanks Christophe, that's what I was looking for. Arturo On Wed, Jul 25, 2018 at 4:51 AM Christophe Demarey < christophe.demarey(a)inria.fr> wrote: > Hi Arthur, > > The starting point is the IceRepository. > You can build you own repository with IceRepositoryCreator or get an > existing one from IceRepository registry. > Then on the repository you can call #pull. > > Christophe > > > Le 24 juil. 2018 Ã 20:09, Arturo Zambrano <arturo.zambrano(a)gmail.com> a > Ã©crit : > > > > Hi All, > > the new Iceberg is really cool. I'm using it in Pharo 6.1. > > > > Is there some documentation or script examples on how to use Iceberg > programmatically? I would like to write an script to pull changes to an > existing repository, before doing some other stuff. I can do it with the > UI, but I would like to automatize it. Any pointer is welcome. > > > > Thanks in advance. > > Arturo > > >

July 25, 2018

Re: [Pharo-users] OmniBase for Pharo 6

by Matias Maretto

Marco: thank you for this. A few weeks ago when I was trying to find the "lock" problem I saw this method but I did not dare to change it; and just as you said I been using OminBase with no problem, but never runned the Garbage Collector, Surely I was going to have a bigger problem in the future for not running the garbageCollect. I have just update this method on my project and tried both the garbageCollect and the reorganize method , both worked fine (win 7 32bits, later I am going to test on Win 10 64 bits). Thanks again. MatÃas. ________________________________ De: Pharo-users <pharo-users-bounces(a)lists.pharo.org> en nombre de marco <marco(a)omeleon.de> Enviado: miÃ©rcoles, 25 de julio de 2018 10:23 a. m. Para: pharo-users(a)lists.pharo.org Asunto: Re: [Pharo-users] OmniBase for Pharo 6 One serious problem left: OmniBaseTest>>testGC fails. Have a look at the 2 methods /ODBContainer>>lockDataFile "Dec, 2010 sas: this code was relying in evaluating a block that was happening only in win32 but not in *nix. So now I'm making it wait for the lock and then execute the code normally because it seems the intention was the same." | currentSpace defaultSpace | activeDataFile waitForAddingLock. currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile] ODBFile>>waitForLockAt: pos length: length whileWaitingDo: aBlock "Wait for lock at given position. Evaluate aBlock in each iteration." "Dec, 2010. sas: evaluating code in each iteration while waiting sounds weird but whatever. I think this was designed on the wrong assumption that the OS always will provide a false in the first try but what actually happens is that in win32 you always will get at least one false but in *nixes you don't do see that happening (which is totally reasonable). BEWARE of stupid code relying on this stupid assumption (I've already patched #lockDataFile because of this)." | startTime currentTime | (stream lockAt: pos length: length) ifFalse: [ startTime := Time totalSeconds. [stream lockAt: pos length: length] whileFalse: [ aBlock value. currentTime := Time totalSeconds. currentTime - startTime < self timeOutLength ifFalse: [ ODBLockNotification signal ifTrue: [startTime := currentTime] ifFalse: [^ODBCannotLock signal]]]] / So "sas" did a refactoring in 2010 because he/she thought it is stupid code. This code change breaks the GarbageCollection of Omnibase. I reverted the first method to the original code of the Omnibase author, David Gorisek: /lockDataFile | currentSpace defaultSpace | activeDataFile waitForAddingLockWhileWaitingDo: [currentSpace := activeDataFile == dataFileA ifTrue: [0] ifFalse: [1]. defaultSpace := objectManager defaultObjectSpace. currentSpace == defaultSpace ifFalse: [defaultSpace == 0 ifTrue: [dataFileA isNil ifTrue: [dataFileA := ODBObjectStorage openOn: self dataFileNameA]. activeDataFile := dataFileA] ifFalse: [dataFileB isNil ifTrue: [dataFileB := ODBObjectStorage openOn: self dataFileNameB]. activeDataFile := dataFileB]. ^self lockDataFile]] / I removed the comment of "sas" from the second method. It is wrong. The garbage collection works again and the test is green. GC is an important feature of Omnibase. It creates a new repository/database file and copies the data into the new file, compressing the data. It keeps your database compact and clean. If you don't run it (frequently) your db files keep on growing... Matias, your fixes are absolutely valid and should be published - together with the one above - in Esteban's repository. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html Smalltalk - Pharo Smalltalk Users | Mailing List Archive<http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> forum.world.st Pharo Smalltalk Users forum and mailing list archive. Pharo User Forum

July 25, 2018