- Pharo-users - lists.pharo.org

Re: [Pharo-users] voyage mongo and transactionality

by PBKResearch

It may be irrelevant, but I have been playing recently with OmniBase, which is a fully object-oriented database system, now over 20 years old, but it still works very well for my uses. David GoriÅ¡ek, the author, claims that it has ACID properties. From my reading, updates operate on a proxy object, which is not written to the database until an explicit commit is given. A second transaction accessing the same object will still see the original until the change is committed. What happens to a proxy which is never committed is not clear, but if GoriÅ¡ek is right, the stored data can never be contaminated. I think a proxy in this sense is equivalent to a memento. Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The code is ancient, there is no documentation and obviously no support, but it might be worth while for someone to try some software archaeology and put it to use. I have found it possible to create and maintain a small database of natural language information, and access is fairly quick and easy â and itâs all Smalltalk. It claims to store all kinds of Smalltalk objects, except block closures, and skimming through the code it seems to incorporate a serializer similar to Fuel. The only documentation I have found is a slideshow at https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found out a few things about it, if anyone is interested. Peter Kenny From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Norbert Hartl Sent: 09 October 2019 18:08 To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] voyage mongo and transactionality Am 09.10.2019 um 16:48 schrieb "jtuchel(a)objektfabrik.de <mailto:jtuchel@objektfabrik.de> " <jtuchel(a)objektfabrik.de <mailto:jtuchel@objektfabrik.de> >: This is a tricky mine field. Sometimes you need a lot of business functionality in objects referenced in your objects that are currently in the editor. So I'm still to see a project in which the memento pattern really worked for more complex scenarios. How deep do you dive to have enough memento objects to provide the functionality needed. I guess you can do that with some sort of object-level transaction framework that automatically creates mementos of whatever object is being navigated to during some kind of processing-context. I guess slots could be of use here. But this is not trivial for general cases. Yes it is tricky. You can have copies of business objects but you have always references to the business objects not pointing to the copy. And you need to know which objects should be tracked. In Gemstone IIRC it is easy as it is the time the object is copied from the stone to the gem it is registered in the current transaction. So you can check it and committing if it changed because you have to write it back. The important point here might be get noticed when a reference is acquired. In pharo it is not that easy but could be done if object would be reified and interceptable. In my experience, this problem area makes for the other 70% of the time spent on developing GUI or Web applications, besides the 60% for GUI design and implementation and 25% business logic... 70% + 60% + 25% + 30% = 185% sounds indeed very realistic if it comes to project planning. ðThere is even a rule saying that for the first 90% of the project you need the first 90% of time and for the last 10% of the project you need the second 90% of time. I'd be interested to learn about patterns to handle such more complex things. We constantly travel back and forth between implementing stuff in the GUI handlers (copying values to the GUI classes that access themselves during GUI operations and push values to the business objects when the users clicks on OK), using mementos (which most of the times are nets of mementos that are created manually - "we know what we'll touch in this Editor") and operating on business objects directly and relying on the persistence mechanism (Glorp in our case) and its rollback behaviour. All three have lots of weaknesses and seem to have their place nevertheless. So this is a very interesting discussion and I think this is an area that has not been solved yet. I think it isnât solved and I find every piece of information about it very interesting. Norbert Joachim Am 09.10.19 um 16:25 schrieb James Foster: Thanks for the explanation. And, yes, this is an artifact of your design; if you put intermediate values into domain objects then they will remain in your domain objects to be seen later. From what youâve described, I donât see how it would be any different in a non-image environment (Java, C#, etc.), unless you re-read the entire object graph from the database. As someone else mentioned, this would be a good place for the Memento Pattern. James On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren <jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com> > wrote: Hi James, I see how my explanation might be unclear. We have a main form for the agenda and a subform for an item, which is shown using Seaside call/answer. The save button of the subform is clicked, which adds the item to the underlying agenda model object, but the save button of the main form is not clicked by the user. The callback for the main save button sends the save message to the agenda object, causing the database to be updated. So yes, the browser does submit the data on the subform, it's the main form component that doesn't receive the save button callback. I realize that this is in large part an issue with our design. However, the way object persistence seems to work in the image environment plays a large role. Kind regards, Jonathan van Alteren Founding Member | Object Guild jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com> On 8 Oct 2019, 15:41 +0200, James Foster <Smalltalk(a)jgfoster.net <mailto:Smalltalk@jgfoster.net> >, wrote: On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren <jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com> > wrote: We've encountered an issue where a user makes changes to an agenda, but does not click the Save button. Instead, the user closes the browser or uses the navigation to go to a different part of the application. When navigating back to the original agenda, the changes made previously (e.g. items added) are still being displayed, even though they were never explicitly saved. Here is what I donât understand: how did the change get from the userâs client agent (browser) to the server? If you make a change to a field in a form and then close the browser, who sent the change to the server? If you show the save domain value in a different location, with a dynamically-generated id and name (so it isnât cached in the browser), or written to the Pharo Transcript, does the value still change? That is, are you sure that the change is in the reflected in the Smalltalk image and not just somehow cached in the browser? James -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

Oct. 9, 2019

Re: [Pharo-users] voyage mongo and transactionality

by Norbert Hartl

> Am 09.10.2019 um 16:48 schrieb "jtuchel(a)objektfabrik.de" <jtuchel(a)objektfabrik.de>: > > > This is a tricky mine field. Sometimes you need a lot of business functionality in objects referenced in your objects that are currently in the editor. So I'm still to see a project in which the memento pattern really worked for more complex scenarios. How deep do you dive to have enough memento objects to provide the functionality needed. I guess you can do that with some sort of object-level transaction framework that automatically creates mementos of whatever object is being navigated to during some kind of processing-context. I guess slots could be of use here. But this is not trivial for general cases. Yes it is tricky. You can have copies of business objects but you have always references to the business objects not pointing to the copy. And you need to know which objects should be tracked. In Gemstone IIRC it is easy as it is the time the object is copied from the stone to the gem it is registered in the current transaction. So you can check it and committing if it changed because you have to write it back. The important point here might be get noticed when a reference is acquired. In pharo it is not that easy but could be done if object would be reified and interceptable. > In my experience, this problem area makes for the other 70% of the time spent on developing GUI or Web applications, besides the 60% for GUI design and implementation and 25% business logic... > 70% + 60% + 25% + 30% = 185% sounds indeed very realistic if it comes to project planning. ðThere is even a rule saying that for the first 90% of the project you need the first 90% of time and for the last 10% of the project you need the second 90% of time. > I'd be interested to learn about patterns to handle such more complex things. We constantly travel back and forth between implementing stuff in the GUI handlers (copying values to the GUI classes that access themselves during GUI operations and push values to the business objects when the users clicks on OK), using mementos (which most of the times are nets of mementos that are created manually - "we know what we'll touch in this Editor") and operating on business objects directly and relying on the persistence mechanism (Glorp in our case) and its rollback behaviour. All three have lots of weaknesses and seem to have their place nevertheless. > > So this is a very interesting discussion and I think this is an area that has not been solved yet. > I think it isnât solved and I find every piece of information about it very interesting. Norbert > Joachim > > > > > > > > > > > > >> Am 09.10.19 um 16:25 schrieb James Foster: >> Thanks for the explanation. And, yes, this is an artifact of your design; if you put intermediate values into domain objects then they will remain in your domain objects to be seen later. From what youâve described, I donât see how it would be any different in a non-image environment (Java, C#, etc.), unless you re-read the entire object graph from the database. As someone else mentioned, this would be a good place for the Memento Pattern. >> >> James >> >>> On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren <jvalteren(a)objectguild.com> wrote: >>> >>> Hi James, >>> >>> I see how my explanation might be unclear. >>> >>> We have a main form for the agenda and a subform for an item, which is shown using Seaside call/answer. The save button of the subform is clicked, which adds the item to the underlying agenda model object, but the save button of the main form is not clicked by the user. The callback for the main save button sends the save message to the agenda object, causing the database to be updated. >>> >>> So yes, the browser does submit the data on the subform, it's the main form component that doesn't receive the save button callback. I realize that this is in large part an issue with our design. However, the way object persistence seems to work in the image environment plays a large role. >>> >>> >>> Kind regards, >>> >>> Jonathan van Alteren >>> >>> Founding Member | Object Guild >>> jvalteren(a)objectguild.com >>>> On 8 Oct 2019, 15:41 +0200, James Foster <Smalltalk(a)jgfoster.net>, wrote: >>>> >>>>> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren <jvalteren(a)objectguild.com> wrote: >>>>> >>>>> We've encountered an issue where a user makes changes to an agenda, but does not click the Save button. Instead, the user closes the browser or uses the navigation to go to a different part of the application. When navigating back to the original agenda, the changes made previously (e.g. items added) are still being displayed, even though they were never explicitly saved. >>>> >>>> Here is what I donât understand: how did the change get from the userâs client agent (browser) to the server? If you make a change to a field in a form and then close the browser, who sent the change to the server? If you show the save domain value in a different location, with a dynamically-generated id and name (so it isnât cached in the browser), or written to the Pharo Transcript, does the value still change? That is, are you sure that the change is in the reflected in the Smalltalk image and not just somehow cached in the browser? >>>> >>>> James >>>> >>>> >> > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > >

Oct. 9, 2019

Re: [Pharo-users] voyage mongo and transactionality

by jtuchel＠objektfabrik.de

Am 09.10.19 um 16:48 schrieb jtuchel(a)objektfabrik.de: > > In my experience, this problem area makes for the other 70% of the > time spent on developing GUI or Web applications, besides the 60% for > GUI design and implementation and 25% business logic... > I forgot the 30% for O/R Mapping when you use an ORM ;-) Joachim -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

Oct. 9, 2019

Re: [Pharo-users] voyage mongo and transactionality

by jtuchel＠objektfabrik.de

This is a tricky mine field. Sometimes you need a lot of business functionality in objects referenced in your objects that are currently in the editor. So I'm still to see a project in which the memento pattern really worked for more complex scenarios. How deep do you dive to have enough memento objects to provide the functionality needed. I guess you can do that with some sort of object-level transaction framework that automatically creates mementos of whatever object is being navigated to during some kind of processing-context. I guess slots could be of use here. But this is not trivial for general cases. In my experience, this problem area makes for the other 70% of the time spent on developing GUI or Web applications, besides the 60% for GUI design and implementation and 25% business logic... I'd be interested to learn about patterns to handle such more complex things. We constantly travel back and forth between implementing stuff in the GUI handlers (copying values to the GUI classes that access themselves during GUI operations and push values to the business objects when the users clicks on OK), using mementos (which most of the times are nets of mementos that are created manually - "we know what we'll touch in this Editor") and operating on business objects directly and relying on the persistence mechanism (Glorp in our case) and its rollback behaviour. All three have lots of weaknesses and seem to have their place nevertheless. So this is a very interesting discussion and I think this is an area that has not been solved yet. Joachim Am 09.10.19 um 16:25 schrieb James Foster: > Thanks for the explanation. And, yes, this is an artifact of your > design; if you put intermediate values into domain objects then they > will remain in your domain objects to be seen later. From what youâve > described, I donât see how it would be any different in a non-image > environment (Java, C#, etc.), unless you re-read the entire object > graph from the database. As someone else mentioned, this would be a > good place for the Memento Pattern. > > James > >> On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren >> <jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com>> wrote: >> >> Hi James, >> >> I see how my explanation might be unclear. >> >> We have a main form for the agenda and a subform for an item, which >> is shown using Seaside call/answer. The save button of the subform is >> clicked, which adds the item to the underlying agenda model object, >> but the save button of the main form _is not_Â clicked by the user. >> The callback for the main save button sends the save message to the >> agenda object, causing the database to be updated. >> >> So yes, the browser does submit the data on the subform, it's the >> main form component that doesn't receive the save button callback. I >> realize that this is in large part an issue with our design. However, >> the way object persistence seems to work in the image environment >> plays a large role. >> >> >> Kind regards, >> >> Jonathan van Alteren >> >> Founding Member | Object Guild >> jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com> >> On 8 Oct 2019, 15:41 +0200, James Foster <Smalltalk(a)jgfoster.net >> <mailto:Smalltalk@jgfoster.net>>, wrote: >>> >>>> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren >>>> <jvalteren(a)objectguild.com <mailto:jvalteren@objectguild.com>> wrote: >>>> >>>> We've encountered an issue where a user makes changes to an agenda, >>>> but does not click the Save button. Instead, the user closes the >>>> browser or uses the navigation to go to a different part of the >>>> application. When navigating back to the original agenda, the >>>> changes made previously (e.g. items added) are still being >>>> displayed, even though they were never explicitly saved. >>> >>> Here is what I donât understand: how did the change get from the >>> userâs client agent (browser) to the server? If you make a change to >>> a field in a form and then close the browser, who sent the change to >>> the server? If you show the save domain value in a different >>> location, with a dynamically-generated id and name (so it isnât >>> cached in the browser), or written to the Pharo Transcript, does the >>> value still change? That is, are you sure that the change is in the >>> reflected in the Smalltalk image and not just somehow cached in the >>> browser? >>> >>> James >>> >>> > -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

Oct. 9, 2019

Re: [Pharo-users] voyage mongo and transactionality

by James Foster

Thanks for the explanation. And, yes, this is an artifact of your design; if you put intermediate values into domain objects then they will remain in your domain objects to be seen later. From what youâve described, I donât see how it would be any different in a non-image environment (Java, C#, etc.), unless you re-read the entire object graph from the database. As someone else mentioned, this would be a good place for the Memento Pattern. James > On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren <jvalteren(a)objectguild.com> wrote: > > Hi James, > > I see how my explanation might be unclear. > > We have a main form for the agenda and a subform for an item, which is shown using Seaside call/answer. The save button of the subform is clicked, which adds the item to the underlying agenda model object, but the save button of the main form is not clicked by the user. The callback for the main save button sends the save message to the agenda object, causing the database to be updated. > > So yes, the browser does submit the data on the subform, it's the main form component that doesn't receive the save button callback. I realize that this is in large part an issue with our design. However, the way object persistence seems to work in the image environment plays a large role. > > > Kind regards, > > Jonathan van Alteren > > Founding Member | Object Guild > jvalteren(a)objectguild.com > On 8 Oct 2019, 15:41 +0200, James Foster <Smalltalk(a)jgfoster.net>, wrote: >> >>> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren <jvalteren(a)objectguild.com> wrote: >>> >>> We've encountered an issue where a user makes changes to an agenda, but does not click the Save button. Instead, the user closes the browser or uses the navigation to go to a different part of the application. When navigating back to the original agenda, the changes made previously (e.g. items added) are still being displayed, even though they were never explicitly saved. >> >> Here is what I donât understand: how did the change get from the userâs client agent (browser) to the server? If you make a change to a field in a form and then close the browser, who sent the change to the server? If you show the save domain value in a different location, with a dynamically-generated id and name (so it isnât cached in the browser), or written to the Pharo Transcript, does the value still change? That is, are you sure that the change is in the reflected in the Smalltalk image and not just somehow cached in the browser? >> >> James >> >>

Oct. 9, 2019

Re: [Pharo-users] Spec2 and Gtk3

by Steve Quezadas

Can't wait. You guys are doing an amazing job with this Glamorous Toolkit stuff! On Wed, Oct 9, 2019 at 5:04 AM Cyril Ferlicot <cyril.ferlicot(a)gmail.com> wrote: > On Wed, Oct 9, 2019 at 11:47 AM Shaping <shaping(a)uurda.org> wrote: > > > > Is there an ETA for Pharo's Spec2 with Gtk3 bindings? > > > > Hi, > > The project is currently in development and should have a stable > version for the Pharo 8 release. > > > > When is the earliest I can test it? > > > > > > Shaping > > > > > > > > > -- > Cyril Ferlicot > https://ferlicot.fr > >

Oct. 9, 2019

Re: [Pharo-users] How to zip a WideString

by Sven Van Caekenberghe

Actually, thinking about the original use case, I now feel that it would be best to remove #zipped/unzipped from String. The original problem was that 'Les Ã©lÃ¨ves FranÃ§aises ont 100 â¬' zipped unzipped. does not work (it fails on WideStrings), while we now have 'Les Ã©lÃ¨ves FranÃ§aises ont 100 â¬' utf8Encoded zipped unzipped utf8Decoded. which I would consider better form/style. The original is also very confusing, since the result of zipping is not a string but binary. > On 3 Oct 2019, at 13:21, Tomohiro Oda <tomohiro.tomo.oda(a)gmail.com> wrote: > > Sven, > > Yes, ByteArray>>zipped/unzipped are simple, neat and intuitive way of > zipping/unzipping binary data. > I also love the new idioms. They look clean and concise. > > Best Regards, > --- > tomo > > 2019å¹´10æ3æ¥(æ¨) 20:14 Sven Van Caekenberghe <sven(a)stfx.eu>: >> >> Actually, thinking about this a bit more, why not add #zipped #unzipped to ByteArray ? >> >> >> ByteArray>>#zipped >> "Return a GZIP compressed version of the receiver as a ByteArray" >> >> ^ ByteArray streamContents: [ :out | >> (GZipWriteStream on: out) nextPutAll: self; close ] >> >> ByteArray>>#unzipped >> "Assuming the receiver contains GZIP encoded data, >> return the decompressed data as a ByteArray" >> >> ^ (GZipReadStream on: self) upToEnd >> >> >> The original oneliner then becomes >> >> 'string' utf8Encoded zipped. >> >> and >> >> data unzipped utf8Decoded >> >> which is pretty clear, simple and intention-revealing, IMHO. >> >>> On 3 Oct 2019, at 13:04, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hi Tomo, >>> >>> Indeed, I stand corrected, it does indeed seem possible to use the existing gzip classes to work from bytes to bytes, this works fine: >>> >>> data := ByteArray streamContents: [ :out | (GZipWriteStream on: out) nextPutAll: 'foo 10 â¬' utf8Encoded; close ]. >>> >>> (GZipReadStream on: data) upToEnd utf8Decoded. >>> >>> Now regarding the encoding option, I am not so sure that is really necessary (though nice to have). Why would anyone use anything except UTF8 (today). >>> >>> Thanks again for the correction ! >>> >>> Sven >>> >>>> On 3 Oct 2019, at 12:41, Tomohiro Oda <tomohiro.tomo.oda(a)gmail.com> wrote: >>>> >>>> Peter and Sven, >>>> >>>> zip API from string to string works fine except that aWideString >>>> zipped generates malformed zip string. >>>> I think it might be a good guidance to define >>>> String>>zippedWithEncoding: and ByteArray>>unzippedWithEncoding: . >>>> Such as >>>> String>>zippedWithEncoding: encoder >>>> zippedWithEncoding: encoder >>>> ^ ByteArray >>>> streamContents: [ :stream | >>>> | gzstream | >>>> gzstream := GZipWriteStream on: stream. >>>> encoder >>>> next: self size >>>> putAll: self >>>> startingAt: 1 >>>> toStream: gzstream. >>>> gzstream close ] >>>> >>>> and ByteArray>>unzippedWithEncoding: encoder >>>> unzippedWithEncoding: encoder >>>> | byteStream | >>>> byteStream := GZipReadStream on: self. >>>> ^ String >>>> streamContents: [ :stream | >>>> [ byteStream atEnd ] >>>> whileFalse: [ stream nextPut: (encoder nextFromStream: >>>> byteStream) ] ] >>>> >>>> Then, you can write something like >>>> zipped := yourLongWideString zippedWithEncoding: ZnCharacterEncoder utf8. >>>> unzipped := zipped unzippedWithEncoding: ZnCharacterEncoder utf8. >>>> >>>> This will not affect the existing zipped/unzipped API and you can >>>> handle other encodings. >>>> This zippedWithEncoding: generates a ByteArray, which is kind of >>>> conformant to the encoding API. >>>> And you don't have to create many intermediate byte arrays and byte strings. >>>> >>>> I hope this helps. >>>> --- >>>> tomo >>>> >>>> 2019/10/3(Thu) 18:56 Sven Van Caekenberghe <sven(a)stfx.eu>: >>>>> >>>>> Hi Peter, >>>>> >>>>> About #zipped / #unzipped and the inflate / deflate classes: your observation is correct, these work from string to string, while clearly the compressed representation should be binary. >>>>> >>>>> The contents (input, what is inside the compressed data) can be anything, it is not necessarily a string (it could be an image, so also something binary). Only the creator of the compressed data knows, you cannot assume to know in general. >>>>> >>>>> It would be possible (and it would be very nice) to change this, however that will have serious impact on users (as the contract changes). >>>>> >>>>> About your use case: why would your DB not be capable of storing large strings ? A good DB should be capable of storing any kind of string (full unicode) efficiently. >>>>> >>>>> What DB and what sizes are we talking about ? >>>>> >>>>> Sven >>>>> >>>>>> On 3 Oct 2019, at 11:06, PBKResearch <peter(a)pbkresearch.co.uk> wrote: >>>>>> >>>>>> Hello >>>>>> >>>>>> I have a problem with text storage, to which I seem to have found a solution, but itâs a bit clumsy-looking. I would be grateful for confirmation that (a) there is no neater solution, (b) I can rely on this to work â I only know that it works in a few test cases. >>>>>> >>>>>> I need to store a large number of text strings in a database. To avoid the database files becoming too large, I am thinking of zipping the strings, or at least the less frequently accessed ones. Depending on the source, some of the strings will be instances of ByteString, some of WideString (because they contain characters not representable in one byte). Storing a WideString uncompressed seems to occupy 4 bytes per character, so I decided, before thinking of compression, to store the strings utf8Encoded, which yields a ByteArray. But zipped can only be applied to a String, not a ByteArray. >>>>>> >>>>>> So my proposed solution is: >>>>>> >>>>>> For compression: myZipString := myWideString utf8Encoded asString zipped. >>>>>> For decompression: myOutputString := myZipString unzipped asByteArray utf8Decoded. >>>>>> >>>>>> As I said, it works in all the cases I tried, whether WideString or not, but the chains of transformations look clunky somehow. Can anyone see a neater way of doing it? And can I rely on it working, especially when I am handling foreign texts with many multi-byte characters? >>>>>> >>>>>> Thanks in advance for any help. >>>>>> >>>>>> Peter Kenny >>>>> >>>>> >>>> >>> >> >> >

Oct. 9, 2019

Re: [Pharo-users] How to zip a WideString

by Sven Van Caekenberghe

Richard, Your implementation mixes zipping/unzipping and encoding/decoding, dictating a single way to do so, if I understand it correctly. The composition with several messages allows for end users to choose their own encoding format, depending on their own needs, which I think is more flexible. Sven > On 4 Oct 2019, at 06:36, Richard O'Keefe <raoknz(a)gmail.com> wrote: > > Here's how it would look in my library: > compressed := original zipped. > "There is currently one definition, in > AbstractStringOrByteArray, covering [ReadOnly]ByteArray, > [ReadOnly]String and its many subclasses, > ByteBuffer,StringBuffer, Substring, [ReadOnly]ShortArray, > [ReadOnly]MappedByteArray, and some others. This relies on > _ asByteArraySize and _ asByteArrayDo: _. There is no need for > a separate #utf8Encoded, that's what asByteArrayDo: *does*." > > copy := original class unzip: compressed. > "This is a little trickier, but not hugely so. There is no need > for special case code. [ReadOnly][Mapped]ByteArray and ByteBuffer > are > sequences of bytes, Stringy things are Unicode, and > [ReadOnly]ShortArrays are treated as UTF16." > > As far as I can tell, this just works for the original use case. > > On Fri, 4 Oct 2019 at 11:42, PBKResearch <peter(a)pbkresearch.co.uk> wrote: >> >> Richard >> >> I don't think so. The case being considered for my problem is the compression of a ByteArray produced by applying #utf8Encoded to a WideString, but it extends to any other form of ByteArray. If you substitute ByteArray for SomeClass in your examples, I think you will see why the chosen interface was used. >> >> Peter Kenny >> >> >> -----Original Message----- >> From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Richard O'Keefe >> Sent: 03 October 2019 23:08 >> To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] How to zip a WideString >> >> The interface should surely be >> SomeClass >> methods for: 'compression' >> zipped "return a byte array" >> >> class methods for: 'decompression' >> unzip: aByteArray "return an instance of SomeClass" >> >> >

Oct. 9, 2019

Re: [Pharo-users] [ ANN ] Neo Universal Binary JSON

by Sven Van Caekenberghe

> On 9 Oct 2019, at 14:29, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing. > > UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively. Here are some details/benchmarks. 1. A typical JSON payload data := ZnConstants httpStatusCodes associations collect: [ :each | { #code->each key. #reason->each value } asDictionary ]. [ NeoUBJSONWriter toByteArray: data ] bench. "'11108.114 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "2290" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'4412.670 per second'" [ NeoJSONWriter toString: data ] bench. "'11542.783 per second'" json := NeoJSONWriter toString: data. json size. "2358" [ NeoJSONReader fromString: json ] bench. "'4814.711 per second'" 2. A 1K integer array data := (1 to: 1024) asArray. [ NeoUBJSONWriter toByteArray: data ] bench. "'6945.444 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "2822" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'3280.632 per second'" [ NeoJSONWriter toString: data ] bench. "'4523.095 per second'" json := NeoJSONWriter toString: data. json size. "4014" [ NeoJSONReader fromString: json ] bench. "'1253.749 per second'" 3. A 1K byte array data := ByteArray new: 1024 streamContents: [ :out | 1024 timesRepeat: [ out nextPut: 256 atRandom - 1 ] ]. [ NeoUBJSONWriter toByteArray: data ] bench. "'538493.501 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "1031" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'216269.200 per second'" [ NeoJSONWriter toString: data ] bench. "'4579.084 per second'" json := NeoJSONWriter toString: data. json size. "3686" [ NeoJSONReader fromString: json ] bench. "'1297.362 per second'" Right now, only ByteArray got a highly optimised implementation. In the future, FloatArray and IntegerArray could receive the same treatment. Sven PS: Note that UBJSON is not the same as BSON, nor CBOR, nor MessagePack, which all exist in the same space.

Oct. 9, 2019

Re: [Pharo-users] [ ANN ] Neo Universal Binary JSON

by Sven Van Caekenberghe

> On 9 Oct 2019, at 14:49, PBKResearch <peter(a)pbkresearch.co.uk> wrote: > > Sven > > Excellent. Can the same idea be extended to STON - or is it there already? I did not (yet) consider that, I will think about it. However, Pharo contains two serialisation formats out of the box, STON, which is a textual format focusing on domain model objects, and FUEL, which is a fast binary format that can serialise special system objects as well. > Peter Kenny > > -----Original Message----- > From: Pharo-users <pharo-users-bounces(a)lists.pharo.org> On Behalf Of Sven > Van Caekenberghe > Sent: 09 October 2019 13:29 > To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Subject: [Pharo-users] [ ANN ] Neo Universal Binary JSON > > Hi, > > I just published an implementation of Universal Binary JSON (UBJSON) for > Pharo. > > https://github.com/svenvc/NeoUniversalBinaryJSON > > Universal Binary JSON (UBJSON) is a computer data interchange format. It is > a binary form directly imitating JSON, but requiring fewer bytes of data. It > aims to achieve the generality of JSON, combined with being easier and more > efficient to process than JSON. > > http://ubjson.org > https://en.wikipedia.org/wiki/UBJSON > > The size/speed/efficiency differences are minor for typical JSON payloads, > especially compared with compacted JSON. The implementation is simpler, > though, as there is no string escaping and no number parsing. > > UBJSON is making a larger difference when dealing with arrays containing > numbers. Especially with ByteArrays, UBJSON makes a huge difference, since > these are essentially stored natively. > > Sven > > >

Oct. 9, 2019