Keeping data with an application
Hey, all.. One of the things that I find SUPER necessary when developing an app is to be able to grab data from somewhere else and quickly and easily work on it from my machine. For instance, when I get a bug report that smells funny, the first thing i sometimes do is run a script that grabs the production data, and populates my local database with that data. i am fleshing out a project that i would really like to use Teapot (REST) for. i could then write several front ends in whatever framework/language. I was thinking about using Voyager, but the problem is, the data is VERY relational (see below, if interested).. objects are linked to each other fairly tightly. I donât know if this would be the way to handle the data, as there would be lots of duplication using mongodb as a datastore. Of course, my first thought is Gemstones, but I donât know how to grab the production data and pull it into my local development system. Does anyone have any thoughts on this? For those curious about the data design, i am setting up a REST server for an collaborative internet radio application⦠- there would be a global pool of stations (just streaming urls with metadata, really) - each user can select from or contribute to the global pool - users can add tags to stations (or search through the global stations based on tags) - users can attach comments to stations not complex or intimidating at all, and super easy to model in smalltalk, but not sure about how to store data so that I can quickly grab production data and troubleshoot if if necessary. Thanks! ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Sergio, You might take a look at the talk I gave on Voyage/Tugrik at ESUG(slides[1], video[2]). The project is in a functional, but roughed out stage --- but the main idea behind it is that with GemStone as the db serve, you are dealing with Smalltalk objects on both the client and server ... Voyage Server Blocks allow you to write code on the client that is executed against objects on the server ... STON is used for serialization/materialization ... Voyage Server Blocks let you blur the lines as to where do the objects live and where does the code execute ... Currently Tugrik uses MongoTalk as the model for Voyage-style interactions between client and server, but Voyage Server Blocks (which are not a feature of MongoTalk) may be useful on their own ... In the talk, I mention that I am looking for partners in pushing Voyage/Tugrik forward, because I think that real users with real problems are needed to drive the project forward --- I am more interested in meeting real developer needs, than building something that makes me happy:) MongoTalk was originally chosen to make it practical for folks already using Voyage/Mongo to easily kick the tires on Voyage/Tugrik, but I don't think that it is necessary to stay coupled to MongoTalk. Michal Balda is looking at something "simpler than Tugrik+MongoTalk+Voyage" and I wish that I could be in Zurich to see his presentation next week[3]:) as I want to support his efforts ... Dale [1] http://www.slideshare.net/esug/tugrik-a-new-persistence-option-for-pharo [2] https://www.youtube.com/watch?v=YwlUdRaqTwE&feature=youtu.be [3] http://forum.world.st/ANN-Zurich-Smalltalk-Meetup-Nov-8th-2016-tt4919147.htm... On 11/01/2016 08:54 AM, sergio ruiz wrote:
Hey, all..
One of the things that I find SUPER necessary when developing an app is to be able to grab data from somewhere else and quickly and easily work on it from my machine.
For instance, when I get a bug report that smells funny, the first thing i sometimes do is run a script that grabs the production data, and populates my local database with that data.
i am fleshing out a project that i would really like to use Teapot (REST) for. i could then write several front ends in whatever framework/language.
I was thinking about using Voyager, but the problem is, the data is VERY relational (see below, if interested).. objects are linked to each other fairly tightly. I donât know if this would be the way to handle the data, as there would be lots of duplication using mongodb as a datastore.
Of course, my first thought is Gemstones, but I donât know how to grab the production data and pull it into my local development system.
Does anyone have any thoughts on this?
For those curious about the data design, i am setting up a REST server for an collaborative internet radio applicationâ¦
- there would be a global pool of stations (just streaming urls with metadata, really) - each user can select from or contribute to the global pool - users can add tags to stations (or search through the global stations based on tags) - users can attach comments to stations
not complex or intimidating at all, and super easy to model in smalltalk, but not sure about how to store data so that I can quickly grab production data and troubleshoot if if necessary.
Thanks!
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Okay.. is Voyage ready to rock and/or roll? is today your birthday? for some reason, i got a notification of several things on my machine today.. one of those being your birthday.. if so, happy birthday! if not.. happy un-birthday! On November 1, 2016 at 2:16:57 PM, Dale Henrichs (dale.henrichs@gemtalksystems.com) wrote: You might take a look at the talk I gave on Voyage/Tugrik at ESUG(slides[1], video[2]). The project is in a functional, but roughed out stage ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
On 11/1/16 12:28 PM, sergio ruiz wrote:
Okay.. is Voyage ready to rock and/or roll? Voyage is definitely ready to rock and roll ... Voyage/Tugrik is ready to be rocked ... it is functional and passes the tests (on Pharo5.0).
For this project, I want to make progress based on feedback from users ... there are lots of different directions that could be taken from this starting point and I want to be driven by user feedback ... so if you are willing to provide feedback I am willing to work with you ... The only change that I think I need to make is to improve the session pool logic ... it works fine, but the session boundary is not quite at right level to make the best use of GemStone sessions ... With "direct access" to the server objects via Voyage Server Blocks I think that query api provided by MongoTalk may not be that interesting, but this is the kind of information I am looking for from users ... If MongoTalk _is_ interesting then you should know that I did not try to fill out the full MongoDB query language, yet ... I implemented the features excercised by tests, so if you hit something that you you find lacking then I will plug the gap ... it is relatively straight forward to implement the MongoDB query language (except for the bits that rely directly on the javascript language:) so it shouldn't take too long to fill in holes:)
is today your birthday? for some reason, i got a notification of several things on my machine today.. one of those being your birthday..
if so, happy birthday!
Tomorrow is my birthday, thanks:)
if not.. happy un-birthday!
Dale
For this project, I want to make progress based on feedback from users ... there are lots of different directions that could be taken from this starting point and I want to be driven by user feedback ... so if you are willing to provide feedback I am willing to work with you ... i think that for my current project, I want to move REALLY quickly. so it might not be a good fit, but i would totally like to play around with it under different use cases. ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Keeping data in image with a STON export would be good enough: On Wed, Nov 2, 2016 at 3:21 PM, sergio ruiz <sergio.rrd@gmail.com> wrote:
For this project, I want to make progress based on feedback from users ... there are lots of different directions that could be taken from this starting point and I want to be driven by user feedback ... so if you are willing to provide feedback I am willing to work with you ...
i think that for my current project, I want to move REALLY quickly. so it might not be a good fit, but i would totally like to play around with it under different use cases.
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Hmm⦠i need to look at this .. i can just burp out all my objects with ston and suck them back in on the local machine.. On November 2, 2016 at 10:25:51 AM, phil@highoctane.be (phil@highoctane.be) wrote: Keeping data in image with a STON export would be good enough: ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Fuel is another option , though ston will be more readable because ston is text fluel is binary . SQLite is another one . You actually have a ton of options here. You can even make your own binary or text format if want , it's easy. On Wed, 2 Nov 2016 at 16:30, sergio ruiz <sergio.rrd@gmail.com> wrote:
Hmm⦠i need to look at this ..
i can just burp out all my objects with ston and suck them back in on the local machine..
On November 2, 2016 at 10:25:51 AM, phil@highoctane.be (phil@highoctane.be) wrote:
Keeping data in image with a STON export would be good enough:
----
peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
On 2 Nov 2016, at 15:29, sergio ruiz <sergio.rrd@gmail.com> wrote:
Hmm⦠i need to look at this ..
i can just burp out all my objects with ston and suck them back in on the local machine..
Just try it ;-) STON put: myObjects onStream[Pretty]: fileStream. STON fromStream: fileStream. https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu... STON has already seen quite some usage, so you should be good.
On November 2, 2016 at 10:25:51 AM, phil@highoctane.be (phil@highoctane.be) wrote:
Keeping data in image with a STON export would be good enough:
peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Use these methods in your class. loadStonFromFile: aDataFilenameString | data | data := STON fromStream: ((self dataFileNamed: aFilenameString) readStream). ^data persistAsSton: anObject toFile: aDataFilenameString (self newDataFileNamed: aFilenameString) writeStreamDo: [ :out | out << (STON toString: anObject) ] dataFileNamed: filename "For an existing file" ^ (FileLocator imageDirectory / 'data' / filename) fullPath asFileReference newDataFileNamed: filename | file | file := (FileLocator imageDirectory / 'data' / filename ) fullPath asFileReference. file writeStream close. ^ file Usage: "Saving" self persistAsSton: someDataStructure toFile: self newDataFileNamed: 'persist.dat' "Loading" someDataStructure := self loadStonFromFile: self dataFileNamed: 'persist:dat' HTH Phil On Wed, Nov 2, 2016 at 3:38 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 2 Nov 2016, at 15:29, sergio ruiz <sergio.rrd@gmail.com> wrote:
Hmm⦠i need to look at this ..
i can just burp out all my objects with ston and suck them back in on the local machine..
Just try it ;-)
STON put: myObjects onStream[Pretty]: fileStream.
STON fromStream: fileStream.
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ lastSuccessfulBuild/artifact/book-result/STON/STON.html
STON has already seen quite some usage, so you should be good.
On November 2, 2016 at 10:25:51 AM, phil@highoctane.be ( phil@highoctane.be) wrote:
Keeping data in image with a STON export would be good enough:
peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Hum, use this instead: Usage: "Saving" self persistAsSton: someDataStructure toFile: 'persist.dat' "Loading" someDataStructure := self loadStonFromFile: 'persist:dat' Well, depends if you want to write to same file, different file etc, but this will get you on the right track On Wed, Nov 2, 2016 at 5:20 PM, phil@highoctane.be <phil@highoctane.be> wrote:
Use these methods in your class.
loadStonFromFile: aDataFilenameString
| data | data := STON fromStream: ((self dataFileNamed: aFilenameString) readStream). ^data persistAsSton: anObject toFile: aDataFilenameString (self newDataFileNamed: aFilenameString) writeStreamDo: [ :out | out << (STON toString: anObject) ]
dataFileNamed: filename
"For an existing file" ^ (FileLocator imageDirectory / 'data' / filename) fullPath asFileReference
newDataFileNamed: filename | file | file := (FileLocator imageDirectory / 'data' / filename ) fullPath asFileReference. file writeStream close. ^ file
Usage:
"Saving" self persistAsSton: someDataStructure toFile: self newDataFileNamed: 'persist.dat'
"Loading" someDataStructure := self loadStonFromFile: self dataFileNamed: 'persist:dat'
HTH
Phil
On Wed, Nov 2, 2016 at 3:38 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 2 Nov 2016, at 15:29, sergio ruiz <sergio.rrd@gmail.com> wrote:
Hmm⦠i need to look at this ..
i can just burp out all my objects with ston and suck them back in on the local machine..
Just try it ;-)
STON put: myObjects onStream[Pretty]: fileStream.
STON fromStream: fileStream.
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBo ok/lastSuccessfulBuild/artifact/book-result/STON/STON.html
STON has already seen quite some usage, so you should be good.
On November 2, 2016 at 10:25:51 AM, phil@highoctane.be ( phil@highoctane.be) wrote:
Keeping data in image with a STON export would be good enough:
peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Looks like I might stick with STON.. I am not really worried too much about execution time or anything.. Need to look at one more thing.. If a user has say.. many stations.. meaning that they have an OrderedCollection of Stations.. I need to check how that gets serialized, and what happens when it gets pulled back in.. I will play with it this afternoon.. Thanks! On November 2, 2016 at 12:24:48 PM, phil@highoctane.be (phil@highoctane.be) wrote: Fuel is another option , though ston will be more readable because ston is text fluel is binary . SQLite is another one . You actually have a ton of options here. You can even make your own binary or text format if want , it's easy. ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
There is a chapter in the Enterprise Pharo book on STON that may be of some help to you https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu... The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/ HTH, -- Does this mail seem too brief? Sorry for that, I donât mean to be rude! Please see http://emailcharter.org <http://emailcharter.org/> . Johan Fabry - http://pleiad.cl/~jfabry PLEIAD and RyCh labs - Computer Science Department (DCC) - University of Chile
On 02 Nov 2016, at 14:27, sergio ruiz <sergio.rrd@gmail.com> wrote:
Looks like I might stick with STON.. I am not really worried too much about execution time or anything..
Need to look at one more thing..
If a user has say.. many stations.. meaning that they have an OrderedCollection of Stations.. I need to check how that gets serialized, and what happens when it gets pulled back in..
I will play with it this afternoon..
Thanks!
On November 2, 2016 at 12:24:48 PM, phil@highoctane.be <mailto:phil@highoctane.be> (phil@highoctane.be <mailto:phil@highoctane.be>) wrote:
Fuel is another option , though ston will be more readable because ston is text fluel is binary . SQLite is another one . You actually have a ton of options here. You can even make your own binary or text format if want , it's easy.
peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 <http://bit.ly/29z9fG0> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com <http://www.village-buzz.com/> http://www.ThoseOptimizeGuys.com <http://www.thoseoptimizeguys.com/> http://www.coffee-black.com <http://www.coffee-black.com/> http://www.painlessfrugality.com <http://www.painlessfrugality.com/> http://www.twitter.com/sergio_101 <http://www.twitter.com/sergio_101> http://www.facebook.com/sergio101 <http://www.facebook.com/sergio101>
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all.. I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â thanks, pharo/smalltalk community for being simple/flexible/clean ! On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote: The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/ HTH, ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Hi, I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended. [1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl <mailto:jfabry@dcc.uchile.cl>) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas < offray.luna@mutabit.com> wrote:
Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended. [1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio
Cheers,
Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
As far as I know it doesn't "normalize" references, as Fuel does. The alternative is to have a Fuel serializer that serializes Fuel objects in STON format instead of binary :D Esteban A. Maringolo 2016-11-02 15:14 GMT-03:00 Dimitris Chloupis <kilon.alios@gmail.com>:
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas < offray.luna@mutabit.com> wrote:
Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended. [1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio
Cheers,
Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
On 2 Nov 2016, at 19:19, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
As far as I know it doesn't "normalize" references, as Fuel does.
What does that mean ? See my previous mail for an explanation for what STON does - it handles shared and cyclic references correctly.
The alternative is to have a Fuel serializer that serializes Fuel objects in STON format instead of binary :D
Esteban A. Maringolo
2016-11-02 15:14 GMT-03:00 Dimitris Chloupis <kilon.alios@gmail.com>: A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote: Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended.
[1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
2016-11-02 15:22 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 2 Nov 2016, at 19:19, Esteban A. Maringolo <emaringolo@gmail.com> wrote: As far as I know it doesn't "normalize" references, as Fuel does.
What does that mean ? See my previous mail for an explanation for what STON does - it handles shared and cyclic references correctly.
Excellent, I wasn't aware of that. Best regards, Esteban A. Maringolo
Basically, STON walks the graph noting each object it sees. When it encounters the same object again, it outputs a reference. Object equality is of course #== References are integers, indexes into the list of objects seen, ordered by walking depth first. Sven
On 2 Nov 2016, at 19:14, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote: Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended.
[1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
ok thanks will give it a try to get an idea On Wed, Nov 2, 2016 at 8:22 PM Sven Van Caekenberghe <sven@stfx.eu> wrote:
Basically, STON walks the graph noting each object it sees. When it encounters the same object again, it outputs a reference. Object equality is of course #==
References are integers, indexes into the list of objects seen, ordered by walking depth first.
Sven
On 2 Nov 2016, at 19:14, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas < offray.luna@mutabit.com> wrote: Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended.
[1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Ston rocks. Pun intended. Phil On Wed, Nov 2, 2016 at 7:35 PM, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
ok thanks will give it a try to get an idea
On Wed, Nov 2, 2016 at 8:22 PM Sven Van Caekenberghe <sven@stfx.eu> wrote:
Basically, STON walks the graph noting each object it sees. When it encounters the same object again, it outputs a reference. Object equality is of course #==
References are integers, indexes into the list of objects seen, ordered by walking depth first.
Sven
On 2 Nov 2016, at 19:14, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas < offray.luna@mutabit.com> wrote: Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended.
[1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Hi Sven, In our ongoing Data Week, we found that would be useful to use a unique identifier for objects that is not integer (some kind of SHA or similar). The use case was this: We were editing the same Grafoscopio tree in a DVCS (fossil, but could be git or something similar) and the nodes changed in a conflicting edition. Contents were different, but the order was the same. Do you think is useful this kind of unique ID index to make STON even DVCS friendlier? Cheers, Offray On 02/11/16 13:21, Sven Van Caekenberghe wrote:
Basically, STON walks the graph noting each object it sees. When it encounters the same object again, it outputs a reference. Object equality is of course #==
References are integers, indexes into the list of objects seen, ordered by walking depth first.
Sven
On 2 Nov 2016, at 19:14, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
A question I was meaning to ask, how efficient is STON at storing references , lets say I have an object that has instance variables that references another object and that object has a instance variable that references another object etc. Where exactly STON stops , or are there any limitations
On Wed, Nov 2, 2016 at 7:58 PM Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote: Hi,
I'm using STON for my Grafoscopio[1][2] project with pretty good results. It's flexible, readable and well supported and document. Better that JSON! Pretty recommended.
[1] http://mutabit.com/grafoscopio/index.en.html [2] http://smalltalkhub.com/#!/~Offray/Grafoscopio Cheers, Offray
On 02/11/16 12:50, sergio ruiz wrote:
The funny thing about STON is that, as a professional developer, I am so used to the whole world being a âhey, look! we have this too!!â world, that I never looked into it at all..
I should have knows that that Pharo/Smalltalk community would, instead of âme tooâ would say.. âlook, hereâs a cleaner, simpler, and more flexible way to do the thing..â
thanks, pharo/smalltalk community for being simple/flexible/clean !
On November 2, 2016 at 1:45:07 PM, Johan Fabry (jfabry@dcc.uchile.cl) wrote:
The book also talks about other storage mechanisms, see http://files.pharo.org/books/enterprise-pharo/
HTH,
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
2016-11-02 15:48 GMT-03:00 Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com>:
Hi Sven,
In our ongoing Data Week, we found that would be useful to use a unique identifier for objects that is not integer (some kind of SHA or similar). The use case was this:
We were editing the same Grafoscopio tree in a DVCS (fossil, but could be git or something similar) and the nodes changed in a conflicting edition. Contents were different, but the order was the same. Do you think is useful this kind of unique ID index to make STON even DVCS friendlier?
The ID for references should be some form of UUID/GUID. SHA or any other hash wouldn't work since it would be dependant on the contents of the object. However this would increase the file size considerably, 38 bytes per id (32 hexadecimal leters, 4 hyphens and the quotes for the whole string). However according to what Sven said, the integer used for the ref is the "order" of the element, so it would be a whole different referencing strategy. Esteban A. Maringolo
I tried to use my ChronosManager object with it and STON does not seem to like this m := ChronosManager new. s:= STON toString: m s inspect. STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
Of course, you need a specfic stonOn: aStream for objects that aren't standard objects: Check out the Ston chapter. Or a couple of stonOn: implementers. Phil On Wed, Nov 2, 2016 at 8:03 PM, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
I suspected so , ok thanks will keep in mind. On Wed, Nov 2, 2016 at 9:13 PM phil@highoctane.be <phil@highoctane.be> wrote:
Of course, you need a specfic stonOn: aStream for objects that aren't standard objects:
Check out the Ston chapter. Or a couple of stonOn: implementers.
Phil
On Wed, Nov 2, 2016 at 8:03 PM, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
You can't serialise blocks with STON. The design goal of STON was to be able to serialise regular domain models. Classes are written as a global reference to their name, as a kind of shortcut (as opposed to writing out the whole class structure). For blocks there is no easy solution, AFAIK.
On 2 Nov 2016, at 20:03, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
You could always store the closure as a string and then evaluate it on materialization, at least if it is clean (no thisContext and no return) For blocks without no self usage: ------------------ | blk | blk := [ :a :b | a <= b ]. #_ assert: blk isClean. blk := Compiler evaluate: blk asString. #_ assert: (blk value: 1 value: 2) ---------------- If self is used then it must also be stored to materialize it correctly again: ---------------- | blk ref | blk := [ self crLog ]. ref := blk receiver. blk := Smalltalk compiler evaluate: blk asString in: nil to: ref. blk value ---------------- On the topic of STON, is there a way to specify that a variable should only be serialized if it is referenced elsewhere depper in the tree? This could be useful, for example when storing morphs that are active in the world to only include #owner that are not the root serialized object and other structures were you might not want to serialize its parent but only deeper into the tree structure. Best regards, Henrik -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Sven Van Caekenberghe Sent: Wednesday, November 2, 2016 11:11 PM To: Any question about pharo is welcome <pharo-users@lists.pharo.org Subject: Re: [Pharo-users] About STONS efficiency at storing nested objects { was : (Re: Keeping data with an application)} You can't serialise blocks with STON. The design goal of STON was to be able to serialise regular domain models. Classes are written as a global reference to their name, as a kind of shortcut (as opposed to writing out the whole class structure). For blocks there is no easy solution, AFAIK.
On 2 Nov 2016, at 20:03, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
On Wed, Nov 2, 2016 at 8:51 PM, Henrik Nergaard <henrik.nergaard@uia.no> wrote:
You could always store the closure as a string and then evaluate it on materialization, at least if it is clean (no thisContext and no return)
For blocks without no self usage: ------------------ | blk |
blk := [ :a :b | a <= b ].
#_ assert: blk isClean.
blk := Compiler evaluate: blk asString.
#_ assert: (blk value: 1 value: 2) ----------------
If self is used then it must also be stored to materialize it correctly again:
---------------- | blk ref |
blk := [ self crLog ]. ref := blk receiver. blk := Smalltalk compiler evaluate: blk asString in: nil to: ref. blk value ----------------
On the topic of STON, is there a way to specify that a variable should only be serialized if it is referenced elsewhere depper in the tree? This could be useful, for example when storing morphs that are active in the world to only include #owner that are not the root serialized object and other structures were you might not want to serialize its parent but only deeper into the tree structure.
I proposed exactly that to SIXX developer which would allow you to serialize at least SortedCollection with clean closures.
Best regards, Henrik
-----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Sven Van Caekenberghe Sent: Wednesday, November 2, 2016 11:11 PM To: Any question about pharo is welcome <pharo-users@lists.pharo.org Subject: Re: [Pharo-users] About STONS efficiency at storing nested objects { was : (Re: Keeping data with an application)}
You can't serialise blocks with STON.
The design goal of STON was to be able to serialise regular domain models.
Classes are written as a global reference to their name, as a kind of shortcut (as opposed to writing out the whole class structure).
For blocks there is no easy solution, AFAIK.
On 2 Nov 2016, at 20:03, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
-- Mariano http://marianopeck.wordpress.com
OK, this is an idea (saving clean blocks as Smalltalk), but I am not sure I like it. This is a huge security problem (open code execution, [ Smalltalk quitPrimitive ]). Furthermore, it would create possible uncheckable portability problems. And even if it were added, it would not cover all blocks, there would still be exceptions.
On 3 Nov 2016, at 03:04, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Wed, Nov 2, 2016 at 8:51 PM, Henrik Nergaard <henrik.nergaard@uia.no> wrote: You could always store the closure as a string and then evaluate it on materialization, at least if it is clean (no thisContext and no return)
For blocks without no self usage: ------------------ | blk |
blk := [ :a :b | a <= b ].
#_ assert: blk isClean.
blk := Compiler evaluate: blk asString.
#_ assert: (blk value: 1 value: 2) ----------------
If self is used then it must also be stored to materialize it correctly again:
---------------- | blk ref |
blk := [ self crLog ]. ref := blk receiver. blk := Smalltalk compiler evaluate: blk asString in: nil to: ref. blk value ----------------
On the topic of STON, is there a way to specify that a variable should only be serialized if it is referenced elsewhere depper in the tree? This could be useful, for example when storing morphs that are active in the world to only include #owner that are not the root serialized object and other structures were you might not want to serialize its parent but only deeper into the tree structure.
I proposed exactly that to SIXX developer which would allow you to serialize at least SortedCollection with clean closures.
Best regards, Henrik
-----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Sven Van Caekenberghe Sent: Wednesday, November 2, 2016 11:11 PM To: Any question about pharo is welcome <pharo-users@lists.pharo.org Subject: Re: [Pharo-users] About STONS efficiency at storing nested objects { was : (Re: Keeping data with an application)}
You can't serialise blocks with STON.
The design goal of STON was to be able to serialise regular domain models.
Classes are written as a global reference to their name, as a kind of shortcut (as opposed to writing out the whole class structure).
For blocks there is no easy solution, AFAIK.
On 2 Nov 2016, at 20:03, Dimitris Chloupis <kilon.alios@gmail.com> wrote:
I tried to use my ChronosManager object with it and STON does not seem to like this
m := ChronosManager new. s:= STON toString: m s inspect.
STONWriter>>error: BlockClosure(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: STONMapWriter>>at:put: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] in [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :each | ... Array(SequenceableCollection)>>do: [ :dictionary | instanceVariableNames do: [ :each | (anObject instVarNamed: each) ifNotNil: [ :value | dictionary at: each asSymbol put: value ] ifNil: [ anObject stonShouldWriteNilInstVars ifTrue: [ dictionary at: each asSymbol put: nil ] ] ] ] in STONWriter>>writeObject: in Block: [ :dictionary | ... [ self newlineIndent. block value: mapWriter ] in [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: [ | mapWriter | mapWriter := STONMapWriter on: self. writeStream nextPut: ${. self indentedDo: [ self newlineIndent. block value: mapWriter ]. self newlineIndent. writeStream nextPut: $} ] in STONWriter>>writeObject:streamMap: in Block: [ | mapWriter |... [ writeStream nextPutAll: anObject class stonName. self prettyPrintSpace. block value ] in STONWriter>>writeObject:do: in Block: [ writeStream nextPutAll: anObject class stonName....etc... STONWriter>>with:do: STONWriter>>writeObject:do: STONWriter>>writeObject:streamMap: STONWriter>>writeObject: KMKeymap(Object)>>stonOn: STONWriter>>nextPut: STONWriter>>encodeKey:value: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] in [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ :key :value | ... [ :assoc | aBlock value: assoc key value: assoc value ] in Dictionary>>keysAndValuesDo: in Block: [ :assoc | aBlock value: assoc key value: assoc va...etc... [ :each | each ifNotNil: [ aBlock value: each ] ] in Dictionary>>associationsDo: in Block: [ :each | each ifNotNil: [ aBlock value: each ] ] Array(SequenceableCollection)>>do: Dictionary>>associationsDo: Dictionary>>keysAndValuesDo: [ self newlineIndent. pairs keysAndValuesDo: [ :key :value | first ifTrue: [ first := false ] ifFalse: [ self mapElementSeparator ]. self encodeKey: key value: value ] ] in STONWriter>>encodeMap: in Block: [ self newlineIndent.... STONWriter>>indentedDo: STONWriter>>encodeMap: [ self encodeMap: hashedCollection ] in STONWriter>>writeMap: in Block: [ self encodeMap: hashedCollection ] STONWriter>>with:do:
-- Mariano http://marianopeck.wordpress.com
On 02/11/16 13:56, Esteban A. Maringolo wrote:
2016-11-02 15:48 GMT-03:00 Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com>:
Hi Sven,
In our ongoing Data Week, we found that would be useful to use a unique identifier for objects that is not integer (some kind of SHA or similar). The use case was this:
We were editing the same Grafoscopio tree in a DVCS (fossil, but could be git or something similar) and the nodes changed in a conflicting edition. Contents were different, but the order was the same. Do you think is useful this kind of unique ID index to make STON even DVCS friendlier? The ID for references should be some form of UUID/GUID. SHA or any other hash wouldn't work since it would be dependant on the contents of the object.
However this would increase the file size considerably, 38 bytes per id (32 hexadecimal leters, 4 hyphens and the quotes for the whole string).
However according to what Sven said, the integer used for the ref is the "order" of the element, so it would be a whole different referencing strategy.
Esteban A. Maringolo
OK. We will think about this to device a better conflict resolution and collaborative editing strategy. Cheers, Offray
On 2 Nov 2016, at 19:56, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
2016-11-02 15:48 GMT-03:00 Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com>:
Hi Sven,
In our ongoing Data Week, we found that would be useful to use a unique identifier for objects that is not integer (some kind of SHA or similar). The use case was this:
We were editing the same Grafoscopio tree in a DVCS (fossil, but could be git or something similar) and the nodes changed in a conflicting edition. Contents were different, but the order was the same. Do you think is useful this kind of unique ID index to make STON even DVCS friendlier?
The ID for references should be some form of UUID/GUID. SHA or any other hash wouldn't work since it would be dependant on the contents of the object.
However this would increase the file size considerably, 38 bytes per id (32 hexadecimal leters, 4 hyphens and the quotes for the whole string).
However according to what Sven said, the integer used for the ref is the "order" of the element, so it would be a whole different referencing strategy.
Right (the last conclusion). The reference identifiers in STON are only used when there are actual references needed (in the case of shared instances or circular references). Furthermore they only make sense in one particular graph. If you want some form of external UUID you should add them to your objects.
Esteban A. Maringolo
Yes, I will do this.. i bet i can even serve it up via http, and pull it directly from my working image.. Thanks! On November 2, 2016 at 12:24:48 PM, phil@highoctane.be (phil@highoctane.be) wrote: Hum, use this instead: Usage: "Saving" self persistAsSton: someDataStructure toFile: 'persist.dat' "Loading" someDataStructure := self loadStonFromFile: 'persist:dat' ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
Sergio, I like the play around part:) Ping me when you find the time! Dale On 11/02/2016 07:21 AM, sergio ruiz wrote:
For this project, I want to make progress based on feedback from users ... there are lots of different directions that could be taken from this starting point and I want to be driven by user feedback ... so if you are willing to provide feedback I am willing to work with you ...
i think that for my current project, I want to move REALLY quickly. so it might not be a good fit, but i would totally like to play around with it under different use cases.
---- peace, sergio photographer, journalist, visionary
Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
I have a few projects that are more loosey goosey on timelines.. this one, people are wanting to see this info pretty soonish.. On November 3, 2016 at 2:23:39 PM, Dale Henrichs (dale.henrichs@gemtalksystems.com) wrote: I like the play around part:) Ping me when you find the time! ---- peace, sergio photographer, journalist, visionary Public Key: http://bit.ly/29z9fG0 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV http://www.Village-Buzz.com http://www.ThoseOptimizeGuys.com http://www.coffee-black.com http://www.painlessfrugality.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101
participants (10)
-
Dale Henrichs -
Dimitris Chloupis -
Esteban A. Maringolo -
Henrik Nergaard -
Johan Fabry -
Mariano Martinez Peck -
Offray Vladimir Luna Cárdenas -
phil＠highoctane.be -
sergio ruiz -
Sven Van Caekenberghe