December 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] 12259: FileSystem memory reads writesusing a binary stream by default

by Max Leske

There are several different approaches in different places: - FileStream reads strings by default. #binary and #ascii switch between formats. File streams use an internal buffer which is either a String (default) or a ByteArray. Itâs even possible to switch between binary and ascii midstream without losing information (if done right) because it only affects the buffer. - ReadStream and WriteStream cannot change their format. Their behavior is determined by the underlying collection. Forcing conversions (e.g. by #asString) can lead to loss of information - RWBinaryOrTextStream (and other subclasses of ReadWriteStream) also support the #binary #ascii method of switching format. Default is #ascii - SocketStream uses the same #binary / #ascii mechanism. Default is #ascii - ZnLimitedReadStream uses the same #binary / #ascii mechanism. Default is #binary (implicit); depends on the underlying stream I think the pattern to follow is clear: ReadStream and WriteStream should allow switching format with #ascii and #binary, default should be #ascii. However, I suspect thereâs a reason that these classes donât support switching, namely that switching makes the implementation more complicated and also slower because more checks need to be made. The easiest solution I see would be to implement something like this: ReadStream>>next ^ self isBinary ifTrue: [ self basicNext asCharacter ] ifFalse: [ self basicNext ] However, #next et al. are implemented in a plugin and the primitive method looks like this: ReadStream>>next <primitive: 65> position >= readLimit ifTrue: [^nil] ifFalse: [^collection at: (position := position + 1)] This means the collection instance variable has to hold either a binary or a string collection. Iâve found a solution which would work and Iâve whipped up a working way (thereâs space for improvementâ¦): ReadStream>>binary collection isString ifFalse: [ ^ self ]. collection := (ByteArray new: collection size) copyReplaceFrom: 1 to: collection size with: collection ReadStrem>>ascii collection isString ifTrue: [ ^ self ]. collection := (String new: collection size) copyReplaceFrom: 1 to: collection size with: collection @Damien opposed to what I wrote earlier, #asString does *not* destroy non-printable characters. Instead, every byte (from 0 to 255) is encoded as a character and thus the string can be converted back to a ByteArray *without* loss of information. Sorry about that. With this change in place the 12259 would become obsolete. Please let me know what you think. This is a pretty big change that might have a lot of consequences in the image. Cheers, Max On 04.12.2013, at 13:14, Max Leske <maxleske(a)gmail.com> wrote: > Let me see what I can come up with. > > > On 03.12.2013, at 19:36, Damien Cassou <damien.cassou(a)gmail.com> wrote: > >> Thanks Max for the report. Do you have an idea on how we could solve the problem ? The previous behaviour was not acceptable either because the streams that came out of a memory filesystem were the only ones with binary content >> >> On Dec 3, 2013 5:35 PM, "Max Leske" <maxleske(a)gmail.com> wrote: >> Damien, Marcus >> >> this change breaks a lot of things in FileSystem-Git. I donât disagree with the idea that reading characters should be default (one could argue about itâ¦) but your change makes it IMPOSSIBLE to read bytes because unprintable characters are discarded! So if my ByteArray is a NULL terminated string, for instance, I can not check for the NULL termination anymore. >> >> Cheers, >> Max >

Dec. 5, 2013

Issue life cycle?

by Bernardo Ezequiel Contreras

Hi all, sorry to bother you again, but is there a issue life cycle graph or a text description of it? because sometimes i am not sure how to change the issue status. Thanks. -- Bernardo E.C. Sent from a cheap desktop computer in South America.

Dec. 5, 2013

Re: [Pharo-dev] [Issue tracker] Help for reviewing needed

by Bernardo Ezequiel Contreras

i review this case https://pharo.fogbugz.com/f/cases/12252/Canceling-Save-As-triggers-a-MNU in my opinion it is ready for integration. On Thu, Dec 5, 2013 at 6:11 AM, Marcus Denker <marcus.denker(a)inria.fr>wrote: > Hello, > > Many people submit lots of nice things, which is greatâ¦ but not many > people look at the issues. > > https://pharo.fogbugz.com/f/filters/45/Review > > Help would be appreciated. > > Marcus > -- Bernardo E.C. Sent from a cheap desktop computer in South America.

Dec. 5, 2013

Re: [Pharo-dev] Roassal Bug related to Athens?

by StÃ©phane Ducasse

And what is important to see is that when Pharo will use Athens fully, the system will manage the surface since it acts as the current display. and AthensSceneView is a good to see a morph embedding its own surface and that should serve as a mini full world in which athens widgets can be drawn. Stef > So, in a meeting, Igor convinced me that surface cannot be handled at the level of framework. The reason is that the surfaces in athens created by the client applications have some application-level semantics such as extent, that cannot be known by the framework. So, application has to integrate session logic so that surface is initialized whenever image is restarted. > > Hence, checkSession default implementation in Athens throws an exception. > > For new comers like me, for creating surfaces and maintaining surfaces, this is useful to understand the session checking code : > AthensFlakeDemo new openInWorld. > > I hope this helps others. > > Usman > > > > On Tue, Dec 3, 2013 at 11:21 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 3 December 2013 23:05, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > > > > On Tue, Dec 3, 2013 at 10:35 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 3 December 2013 15:21, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > > On Tue, Dec 3, 2013 at 2:29 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 3 December 2013 14:18, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > Igor, > > I had a look at Athens demo to understand how to manage sessions and surfaces because Roassal uses AthensSurface and there is a defensive mechanism in place in Roassal to initialize the surface in case it is not done. In the meantime, I have discovered that the red rectangle bug is present in Athens demos as well. > > So, to reproduce. > AthensFlakeDemo new openInWorld > save image > open image > > I tested in Moose 5.0. > > 1. Should I open a bug entry in Pharo? > > only if you intend to fix it. because this demo was not intended to be 'fully featured, end-user compatible > and fool-proof demo'. it just a demo to show animation and discard it, but if you insist this is a bug, > feel free to fix it. > > Unfortunately, I wont have time for this. For me it would have been good, had Athens provided an example of session management as a good client so that example-oriented people like me could understand easily. Because, otherwise, I'll have to look at the code of NativeBoost and that adds to the level of complexity because I would do things with trial and error. > > No need. There is one example: look at AthensSceneView > > > 2. With my superficial knowledge of Athens, my guess is that the session management can be done in Athens so that Roassal (and other tools built on top of Athens) do not need to do it. May be it is the case already but the demos are not using it then. > > I disagree. > Because then, files can also open/close and delete themselves automatically, so you will be left only to do reading and writing... > Resource management is an application-level responsibility, not framework level. > I cannot predict in Athens, how often one wants to create/destroy or (re)use surfaces, and therefore i cannot > create and dispose them when i see fit within framework. > Correct me if i wrong. > > I think this doesn't compare to file handling because there are many types of operations possible on files and that are application-specific. Here resource management is actually exception-handling: I should not give to my client a surface that does not exist any more. Now, of course, if there is some application-level intelligence is involved, we cannot put that in the framework. > > Surface is like file, you create it, delete it.. write to it.. > the management of surfaces is absolutely out of scope of framework. > More than that, surfaces is not something which you operate with usually. Because usually its just a canvas. > > Still, I will maintain that a minimalistic session logic should be built by default in Athens. Because otherwise each application has to maintain a session instance variable and a logic to test that session hasn't changed in between to keep a surface alive, which is lowest common denominator for all the applications using Athens. > > Usman, i have impression that you didn't looked at code at all. > > AthensSurface subclass: #AthensCairoSurface > uses: TCairoLibrary > instanceVariableNames: 'handle context builder id ftFontRenderer session' > classVariableNames: '' > poolDictionaries: 'AthensCairoDefs' > category: 'Athens-Cairo' > > > what you think 'session' ivar and checkSession method does there? > You can make subclass if you want and redefine default behavior. > > But my main problem that you still don't see that the code in example(s) is NOT the lowest common denominator for all applications using Athens. > Lowest common denominator is Athens API. > Athens API defines and describes roles (surface, canvas, paints etc) and their protocols, but NOT how to create/initialize and manage such objects. > > > > regards, > > Usman > > > But I would understand the absence of such a mechanism in the framework but then a tutorial should explain to novices/first-timers how to do it. > > > this sort of things is tiniest (but of course necessary) parts of whole application, and usually will belong to some service layers built around/on top of athens. > in future, sure thing you won't need to care about it. > > Usman > > > > > Usman > > > > On Mon, Dec 2, 2013 at 5:14 PM, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > Hello Igor, > > Moose 5.0 is using Athens as default canvas for Roassal and we have bug with Roassal that seems to be related to Athens. > http://code.google.com/p/moose-technology/issues/detail?id=1019 > > I think it is related to the fact that we create a surface in the OS with Athens and once we quit the image, the surface is destroyed as well. So, when image is restarted with the visualization trying to use the surface, we get the error. > > Could you point to what possibly can be done to avoid this error? Merely checking the existence of an appropriate drawing surface in Athens every time visualization is drawn, would it suffice? > > regards, > > Usman > > > > > -- > Best regards, > Igor Stasenko. > > > > > -- > Best regards, > Igor Stasenko. > > > > > -- > Best regards, > Igor Stasenko. >

Dec. 5, 2013

Re: [Pharo-dev] Roassal Bug related to Athens?

by Usman Bhatti

So, in a meeting, Igor convinced me that surface cannot be handled at the level of framework. The reason is that the surfaces in athens created by the client applications have some application-level semantics such as extent, that cannot be known by the framework. So, application has to integrate session logic so that surface is initialized whenever image is restarted. Hence, checkSession default implementation in Athens throws an exception. For new comers like me, for creating surfaces and maintaining surfaces, this is useful to understand the session checking code : AthensFlakeDemo new openInWorld. I hope this helps others. Usman On Tue, Dec 3, 2013 at 11:21 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 3 December 2013 23:05, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > >> >> >> >> On Tue, Dec 3, 2013 at 10:35 PM, Igor Stasenko <siguctua(a)gmail.com>wrote: >> >>> >>> >>> >>> On 3 December 2013 15:21, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: >>> >>>> >>>> On Tue, Dec 3, 2013 at 2:29 PM, Igor Stasenko <siguctua(a)gmail.com>wrote: >>>> >>>>> >>>>> >>>>> >>>>> On 3 December 2013 14:18, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: >>>>> >>>>>> Igor, >>>>>> >>>>>> I had a look at Athens demo to understand how to manage sessions and >>>>>> surfaces because Roassal uses AthensSurface and there is a defensive >>>>>> mechanism in place in Roassal to initialize the surface in case it is not >>>>>> done. In the meantime, I have discovered that the red rectangle bug is >>>>>> present in Athens demos as well. >>>>>> >>>>>> So, to reproduce. >>>>>> AthensFlakeDemo new openInWorld >>>>>> save image >>>>>> open image >>>>>> >>>>> >>>>>> I tested in Moose 5.0. >>>>>> >>>>>> 1. Should I open a bug entry in Pharo? >>>>>> >>>>>> only if you intend to fix it. because this demo was not intended to >>>>> be 'fully featured, end-user compatible >>>>> and fool-proof demo'. it just a demo to show animation and discard it, >>>>> but if you insist this is a bug, >>>>> feel free to fix it. >>>>> >>>> >>>> Unfortunately, I wont have time for this. For me it would have been >>>> good, had Athens provided an example of session management as a good client >>>> so that example-oriented people like me could understand easily. Because, >>>> otherwise, I'll have to look at the code of NativeBoost and that adds to >>>> the level of complexity because I would do things with trial and error. >>>> >>>> >>> No need. There is one example: look at AthensSceneView >>> >>> >>>> >>>>> >>>>>> 2. With my superficial knowledge of Athens, my guess is that the >>>>>> session management can be done in Athens so that Roassal (and other tools >>>>>> built on top of Athens) do not need to do it. May be it is the case already >>>>>> but the demos are not using it then. >>>>>> >>>>>> I disagree. >>>>> >>>> Because then, files can also open/close and delete themselves >>>>> automatically, so you will be left only to do reading and writing... >>>>> Resource management is an application-level responsibility, not >>>>> framework level. >>>>> I cannot predict in Athens, how often one wants to create/destroy or >>>>> (re)use surfaces, and therefore i cannot >>>>> create and dispose them when i see fit within framework. >>>>> Correct me if i wrong. >>>>> >>>> >>>> I think this doesn't compare to file handling because there are many >>>> types of operations possible on files and that are application-specific. >>>> Here resource management is actually exception-handling: I should not give >>>> to my client a surface that does not exist any more. Now, of course, if >>>> there is some application-level intelligence is involved, we cannot put >>>> that in the framework. >>>> >>>> Surface is like file, you create it, delete it.. write to it.. >>> the management of surfaces is absolutely out of scope of framework. >>> More than that, surfaces is not something which you operate with >>> usually. Because usually its just a canvas. >>> >> >> Still, I will maintain that a minimalistic session logic should be built >> by default in Athens. Because otherwise each application has to maintain a >> session instance variable and a logic to test that session hasn't changed >> in between to keep a surface alive, which is lowest common denominator for >> all the applications using Athens. >> >> Usman, i have impression that you didn't looked at code at all. > > AthensSurface subclass: #AthensCairoSurface > uses: TCairoLibrary > instanceVariableNames: 'handle context builder id ftFontRenderer > session' > classVariableNames: '' > poolDictionaries: 'AthensCairoDefs' > category: 'Athens-Cairo' > > > what you think 'session' ivar and checkSession method does there? > You can make subclass if you want and redefine default behavior. > > But my main problem that you still don't see that the code in example(s) > is NOT the lowest common denominator for all applications using Athens. > Lowest common denominator is Athens API. > Athens API defines and describes roles (surface, canvas, paints etc) and > their protocols, but NOT how to create/initialize and manage such objects. > > > > >> regards, >> >> Usman >> >> >>> >>> But I would understand the absence of such a mechanism in the framework >>>> but then a tutorial should explain to novices/first-timers how to do it. >>>> >>>> >>> this sort of things is tiniest (but of course necessary) parts of whole >>> application, and usually will belong to some service layers built around/on >>> top of athens. >>> in future, sure thing you won't need to care about it. >>> >>> >>>> Usman >>>> >>>> >>>> >>>> >>>>> >>>>> >>>>>> Usman >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Dec 2, 2013 at 5:14 PM, Usman Bhatti <usman.bhatti(a)gmail.com>wrote: >>>>>> >>>>>>> Hello Igor, >>>>>>> >>>>>>> Moose 5.0 is using Athens as default canvas for Roassal and we have >>>>>>> bug with Roassal that seems to be related to Athens. >>>>>>> http://code.google.com/p/moose-technology/issues/detail?id=1019 >>>>>>> >>>>>>> I think it is related to the fact that we create a surface in the OS >>>>>>> with Athens and once we quit the image, the surface is destroyed as well. >>>>>>> So, when image is restarted with the visualization trying to use the >>>>>>> surface, we get the error. >>>>>>> >>>>>>> Could you point to what possibly can be done to avoid this error? >>>>>>> Merely checking the existence of an appropriate drawing surface in Athens >>>>>>> every time visualization is drawn, would it suffice? >>>>>>> >>>>>>> regards, >>>>>>> >>>>>>> Usman >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Igor Stasenko. >>>>> >>>> >>>> >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko. >>> >> >> > > > -- > Best regards, > Igor Stasenko. >

Dec. 5, 2013

Re: [Pharo-dev] Json case: Mapping objects to dictionaries

by Paul DeBruicker

FWIW - The ElasitcSearch stuff only maps to Dictionaries because of an oversight on my part. It should be fine if those objects materialized as JsonObjects. What are you doing to get Dictionaries? Its been a while since I looked at the code. To map domain objects I usually do something custom because I usually don't want all inst vars being sent as JSON. An example of what I normally do is: #jsonWriteOn: aStream | y | y := JsonObject new. self varNamesToSerializeAsJson. do: [ :each | (self perform: each asSymbol) isNil ifFalse: [ y at: each put: (self perform: each asSymbol) ] ]. y jsonWriteOn: aStream and then when materializing: fromJson: aJsonResult self varsToMaterializeFromJson do: [ :eachSymbol | | jsonObj class obj | jsonObj := aJsonResult at: eachSymbol asString ifAbsent: [ nil ]. jsonObj notNil ifTrue: [ class := MyDomain classForJsonVariableNamed: eachSymbol. obj := class isNil ifTrue: [ jsonObj trimBoth ] ifFalse: [ class fromJson: jsonObj ]. self perform: (eachSymbol , ':') asSymbol with: obj ] ] #classForJsonVariableNamed: just looks up in a dictionary that contains inst var names as keys and classes as values. That all being said I'd be interested to learn a better way to do it. Hope this helps Paul Norbert Hartl wrote > What would be the best software to map my objects to dictionaries? There > are a lot of packages in pharo that deal with JSON somehow. But it seems > none does mapping from objects to dictionaries and is standalone. Weâve > got > > - NeoJSON package . Maps objects to strings directly for performance > reason > - JSON package. Maps string to JsonObject (subclassed from dictionary) > - Voyage magritte based mapper: Closest to what I find useful but buried > inside of Voyage. And Iâm not sure it would be easy to factor out this > module as a standalone mapper. > - Seaside javascriptâ¦ : I forgot what it really does. I often shy away > looking at it because I donât want to suck in all the dependencies. > > Iâm asking because for me this is an important asset. If we talk about > JSON there are three things: Objects, JSON structure (being dictionaries > and other collections in pharo) and JSON strings. Writing to another > server or to a file producing JSON strings seems to be enough. But using > it for another system like MongoDB or ElasticSearch this does not work. > Here the common interface seems to be dictionaries. With ElasticSearch > mapping dictionaries to JSON and MongoDB mapping dictionaries to BSON. So > a good descriptive way to map objects to dictionaries would be a good > thing to have. > > Any thoughts? > > Norbert -- View this message in context: http://forum.world.st/Json-case-Mapping-objects-to-dictionaries-tp4727773p4… Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Dec. 5, 2013

Re: [Pharo-dev] Json case: Mapping objects to dictionaries

by Esteban A. Maringolo

Norbert, I'm currently building the dictionaries myself. I have a specific domain class of mapper for each "message" (JSON). The mapper builder manually builds my domain objects from the dictionary returned by "JSJsonParser parse: aString", when writing creates a Dictionary and sends #asJson to it. E.g. I have a EventBuilder which can build an Event from a JSON string or viceversa. I used NeoJSON in the beggining and it's blazing fast (because it is stream based), but I couldn't get my head around some particular cases (which I don't remember now, sorry). I WILL give another try, but with time to fully understand it. In the Java side of my app, this is one of the simplest things to deal with, manually to read and write, by using JSONObject and JSONArray, there's not more of it than this. Regards, Esteban A. Maringolo 2013/12/5 Norbert Hartl <norbert(a)hartl.name>: > What would be the best software to map my objects to dictionaries? There are a lot of packages in pharo that deal with JSON somehow. But it seems none does mapping from objects to dictionaries and is standalone. Weâve got > > - NeoJSON package . Maps objects to strings directly for performance reason > - JSON package. Maps string to JsonObject (subclassed from dictionary) > - Voyage magritte based mapper: Closest to what I find useful but buried inside of Voyage. And Iâm not sure it would be easy to factor out this module as a standalone mapper. > - Seaside javascriptâ¦ : I forgot what it really does. I often shy away looking at it because I donât want to suck in all the dependencies. > > Iâm asking because for me this is an important asset. If we talk about JSON there are three things: Objects, JSON structure (being dictionaries and other collections in pharo) and JSON strings. Writing to another server or to a file producing JSON strings seems to be enough. But using it for another system like MongoDB or ElasticSearch this does not work. Here the common interface seems to be dictionaries. With ElasticSearch mapping dictionaries to JSON and MongoDB mapping dictionaries to BSON. So a good descriptive way to map objects to dictionaries would be a good thing to have. > > Any thoughts? > > Norbert

Dec. 5, 2013

Re: [Pharo-dev] Json case: Mapping objects to dictionaries

by Chris Cunningham

I've used the SCouchDB interface - which include mapping object to dictionaries (well, JSONObjects) and mapping back from them. http://www.squeaksource.com/SCouchDB.html Of course, it is tied to CouchDB, but shouldn't be to hard to pry the mapping part out (once you find it). On Thu, Dec 5, 2013 at 7:44 AM, Norbert Hartl <norbert(a)hartl.name> wrote: > What would be the best software to map my objects to dictionaries? There > are a lot of packages in pharo that deal with JSON somehow. But it seems > none does mapping from objects to dictionaries and is standalone. Weâve got > > - NeoJSON package . Maps objects to strings directly for performance reason > - JSON package. Maps string to JsonObject (subclassed from dictionary) > - Voyage magritte based mapper: Closest to what I find useful but buried > inside of Voyage. And Iâm not sure it would be easy to factor out this > module as a standalone mapper. > - Seaside javascriptâ¦ : I forgot what it really does. I often shy away > looking at it because I donât want to suck in all the dependencies. > > Iâm asking because for me this is an important asset. If we talk about > JSON there are three things: Objects, JSON structure (being dictionaries > and other collections in pharo) and JSON strings. Writing to another server > or to a file producing JSON strings seems to be enough. But using it for > another system like MongoDB or ElasticSearch this does not work. Here the > common interface seems to be dictionaries. With ElasticSearch mapping > dictionaries to JSON and MongoDB mapping dictionaries to BSON. So a good > descriptive way to map objects to dictionaries would be a good thing to > have. > > Any thoughts? > > Norbert >

Dec. 5, 2013

Re: [Pharo-dev] [Issue tracker] Help for reviewing needed

by btc＠openinworld.com

Dec. 5, 2013

Json case: Mapping objects to dictionaries

by Norbert Hartl

What would be the best software to map my objects to dictionaries? There are a lot of packages in pharo that deal with JSON somehow. But it seems none does mapping from objects to dictionaries and is standalone. Weâve got - NeoJSON package . Maps objects to strings directly for performance reason - JSON package. Maps string to JsonObject (subclassed from dictionary) - Voyage magritte based mapper: Closest to what I find useful but buried inside of Voyage. And Iâm not sure it would be easy to factor out this module as a standalone mapper. - Seaside javascriptâ¦ : I forgot what it really does. I often shy away looking at it because I donât want to suck in all the dependencies. Iâm asking because for me this is an important asset. If we talk about JSON there are three things: Objects, JSON structure (being dictionaries and other collections in pharo) and JSON strings. Writing to another server or to a file producing JSON strings seems to be enough. But using it for another system like MongoDB or ElasticSearch this does not work. Here the common interface seems to be dictionaries. With ElasticSearch mapping dictionaries to JSON and MongoDB mapping dictionaries to BSON. So a good descriptive way to map objects to dictionaries would be a good thing to have. Any thoughts? Norbert

Dec. 5, 2013