OOH.. I like this idea. I’ll do a quick test on this and see if it works as expected.. For now, I want to store the data in the image. The trick is, I like to start with a fresh image every few days. So, I’d like to open a new Pharo, pull the source code, then pull the data. Thanks!

If you are keeping data in a Pharo image then you could just save the image instead of using STON. Alternatively, you could model your data in an Application class with three instance variables: theaters, movies, and events. Then you could dump the root application object and STON should preserve the relationships (I expect). I prefer this model since it allows me to create a test object that encapsulates all the data in one place (even if it is referenced from a class variable).

---- peace, sergio photographer, journalist, visionary Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x69B08F58923AB3A2 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV @sergio_101@mastodon.social https://sergio101.com http://www.codeandmusic.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101

+1 for Fuel, see here https://github.com/theseion/Fuel CheersDavide On Thursday, January 18, 2024 at 09:37:21 AM GMT+1, Todd Blanchard via Pharo-users <pharo-users@lists.pharo.org> wrote: You want to look at Fuel, a serialization library. Sent from my iPhone On Jan 17, 2024, at 9:49 PM, sergio ruiz <sergio.rrd@gmail.com> wrote:  I have been in the relational database world for decades. One of the things that is super simple is backing up and restoring data. Sometimes, it makes it makes sense to pull the production data into development. In the case of a catastrophe, it’s super simple to restore data from a backup. I am thinking of backup strategies for my data once again, and am wondering if things have changed since I was last using smalltalk. I am thinking of the following: STONI could dump all of my data into STON, but there is one slight twist. In this project, it’s just making a list of movie listings. There are three basic classes: Theater, Movie, Event. Each of these classes stores an OrderedCollection of items of that type on a class variable, as I’ll need to access each on their own.. so, if an event has a theater, I’ll need to access Theaters on their own, and Movies on their own. So, I would have to rebuild the list of Theaters from the Theaters in each event. Otherwise, I would lose the link between the Theaters on the Theater class variable and the Theater in the Event instance. Am I thinking about this incorrectly? GemstonesI am totally into the idea of using gemstones. Is there a backup and restore for objects here? do the relationships remain? VoyageThis seems like a good solution, but I would rather stay in the smalltalk object store. Ideas? Thanks! ---- peace, sergio photographer, journalist, visionary Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x69B08F58923AB3A2 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV@sergio_101@mastodon.socialhttps://sergio101.com http://www.codeandmusic.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101

Using this Will my links remain viable using this method? Meaning that an event has a theater. That theater is also in the OrderedCollection of Theaters. If the name of he theater changed in the Theaters collection, will that name change occur in the event theater object? After bringing these collections back in via STON, will all of these relationships remain intact?

On Jan 18, 2024, at 8:26 AM, Norbert Hartl <norbert@hartl.name> wrote:

If you have three classes with class side collections you can have a make e.g. a dictionary and put the collections from the class side into it like

model := { #theaters -> Theater instances. #movies -> Movie instances. #event -> Event instances. } asDictionary.

---- peace, sergio photographer, journalist, visionary Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x69B08F58923AB3A2 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV @sergio_101@mastodon.social https://sergio101.com http://www.codeandmusic.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101

Sergio, I would go with STON, as advised by Norbert (I don't have experience with Soil). One of the nice things of STON is that you can use whatever DVCS you like to versionate your data. For example, we locally use Fossil, while most people prefer Git. STON is super versatile! As an example, I used it with Grafoscopio, my Pharo powered interactive outliner/documenter, to store notebooks with Markdown stored inside STON, as you can see in [1]. And now I flipped that idea and I'm using STON to embed notebooks metadata inside Markdeep. In this way anyone can read notebooks directly in the web, without specialized software. [1a] is just the same document in [1] but with this flipped format idea (if you,re even more curious, look at the source code of the page at [1a] and you  will find that the STON representation of notebook cell/node medatada is pretty light and readable). [1] https://mutabit.com/repos.fossil/indieweb/file?name=indieweb.ston&ci=tip [1a] https://mutabit.com/repos.fossil/indieweb/doc/tip/indie-web--25zqu.md.html This is just an example of how STON allowed us to port lessons learned from Grafoscopio, made in 2016, to Lepiter, released in 2021, ensuring longevity of our community memory and data. Hopefully you will find STON pretty empowering in your own experiments. Cheers, Offray On 18/01/24 9:37, Norbert Hartl wrote:

...

Am 18.01.2024 um 14:54 schrieb sergio ruiz <sergio.rrd@gmail.com>:

Using this Will my links remain viable using this method?

Meaning that an event has a theater. That theater is also in the OrderedCollection of Theaters.

If the name of he theater changed in the Theaters collection, will that name change occur in the event theater object?

After bringing these collections back in via STON, will all of these relationships remain intact?

If with relationship you mean that one object event has a reference to a theater object then yes. STON serializes the graph as objects and all references will be reestablished on materialization.

Norbert

...

...

On Jan 18, 2024, at 8:26 AM, Norbert Hartl <norbert@hartl.name> wrote:

If you have three classes with class side collections you can have a make e.g. a dictionary and put the collections from the class side into it like

model := {  #theaters -> Theater instances.  #movies -> Movie instances.  #event -> Event instances. } asDictionary.

---- peace, sergio photographer, journalist, visionary

Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x69B08F58923AB3A2 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV @sergio_101@mastodon.social https://sergio101.com http://www.codeandmusic.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101

Can you sketch out the complexity aspects of your five projects? Is this all for learning, or are they “production” applications? Do you want to gain experience with different ways to persist data, or are you just thinking that you have to use different ways due to project complexity? It might be that Soil database could handle all your use cases, so you could avoid working through a bunch of databases, unless you want to. On Thu, Jan 18, 2024 at 8:56 AM sergio ruiz <sergio.rrd@gmail.com> wrote:

I figured Gemstones would be SUPER heavy handed for this application, but I have a list of five projects, each increasing in complexity that I want to launch and document. So, sooner or later, I will need a more robust solution.

Thanks!

On Jan 18, 2024, at 8:26 AM, Norbert Hartl <norbert@hartl.name> wrote:

GemStone is a huge thing to tame. I would not say the above and GemStone are alternatives. GemStone is a lot to install and learn but you get something that makes it hard to corrupt your data. So you have requirements that make it fit into GemStone or you have requirements that make it a fit for STON. Between both there is no overlap but rather a huge gap between the applicability of each.

---- peace, sergio photographer, journalist, visionary

Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x69B08F58923AB3A2 #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV @sergio_101@mastodon.social https://sergio101.com http://www.codeandmusic.com http://www.twitter.com/sergio_101 http://www.facebook.com/sergio101

Hi Sergio, I added a Movies demo, using a Soil database. There was a previous Accounting demo that had two versions (in memory database and Soil database) from which I based the new demo. I like the Movie demo because it is simple yet complex enough. I suspect the Accounting demo is too complicated to grasp as a first encounter. I’m going to use the Movie demo for a presentation, so it’ll continue to be tweaked. Mainly, I’ll be adding a way to use SoilTransaction with #commit, instead of #checkpointAndContinue. The code is at: [image: Seaside-Quasar.png] yannij/Seaside-Quasar: Pharo/Smalltalk code to use Quasar Framework in Seaside. Also, some demo apps using Magritte are provided. <https://github.com/yannij/Seaside-Quasar> github.com <https://github.com/yannij/Seaside-Quasar> Cheers, Yanni On Thu, Jan 18, 2024 at 11:11 AM sergio ruiz <sergio.rrd@gmail.com> wrote:

[…]

In the end, I’ll end up with:

- an application of some sort that scratches my own itch - a pile of articles outlining how i designed the tool set for the application.

and in reality, it’s gonna be thinly veiled Smalltalk evangelism.

I’m happy to answer any questions about Simple Persistence. It is a nice framework around (potentially any) serializer. It’s meant to be pluggable but currently uses Fuel out of the box. You just tell it what classes to persist and then create two methods per class to handle materialization/serialization. For the record, Ramon Leon created it in a blog post. I just ported it to Pharo and have been maintaining it. The stated reason he created it is that most solutions are complete overkill for 99% of projects that will never “become the next google”. It’s pretty basic, but at least two steps more sophisticated than simply saving the image because it keeps old versions and is easily reloadable into another image. That said, I’ve been using it almost exclusively for my personal projects and have yet to grow out of it.

Norbert, I'm very interested to investigate Soil. Do you have examples on how to use Soil? I currently use a STON persistence model. My current approach to my model is structured specifically to make STON full save and restores simple... - Wrapper Class which contains a collection of high level objects - high level objects contain collections of next level objects - next level objects contain collections of lower level objects - etc. While the database size (7.3Mb) is not terribly large, I am writing about 50 of those per day as each save simply saves off the entire old STON and writes a new one. Concurrency is generally not a problem as we have the application automatically save itself every 10 minutes, but every once in a while we do run up against a conflict that would be best handled by a concurrent capable model. I have roughly nine applications using the same class models, so it would be very helpful to adopt a more OO database approach to persistence... even though we're nowhere near Google :) Thoughts on where to begin? I'm on the Discord server as well, if you'd prefer to discuss there. Thanks, Russ On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl <norbert@hartl.name> wrote:

The approach as I understand is simple and valuable until you face concurrency. Nothing in SimplePersistence prevents your data from becoming corrupt in a concurrent situation. I agree that when you start a project you almost never face concurrency because the odds are way too low to have two things at the same time happening. This is because computers are really fast today and the amount of activity on a service is largely overestimated.

But there is still a huge gap between SimplePersistence and those overkill persistence solutions. For exactly that reason I've done Soil ( https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project with no dependency and setting it up is a no-brainer. It is way bigger than SimplePersistence but it deals with concurrency and adds query capabilties for your objects. But you have to use transactions to do this.

I started ApptiveGrid just by having the data persisted through saving the image. Next was to STON out the model. Then versioned STON files to add fallback. Then there was Soil (well, first voyage and then omnibase and then Soil) which is the next level to iterate.

So ApptiveGrid is not the next google but it also wouldn't work with SimplePersistence. So good that we have enough tools to improve our projects on.

Norbert

Am 15.02.2024 um 06:43 schrieb sean@clipperadams.com:

I’m happy to answer any questions about Simple Persistence. It is a nice framework around (potentially any) serializer. It’s meant to be pluggable but currently uses Fuel out of the box. You just tell it what classes to persist and then create two methods per class to handle materialization/serialization.

For the record, Ramon Leon created it in a blog post. I just ported it to Pharo and have been maintaining it.

The stated reason he created it is that most solutions are complete overkill for 99% of projects that will never “become the next google”. It’s pretty basic, but at least two steps more sophisticated than simply saving the image because it keeps old versions and is easily reloadable into another image.

That said, I’ve been using it almost exclusively for my personal projects and have yet to grow out of it.

-- Russ Whaley whaley.russ@gmail.com

I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel. Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it. Thanks for everyone for giving us so many options. Tim On Sat, 17 Feb 2024, at 6:46 PM, Russ Whaley wrote:

Norbert, I'm very interested to investigate Soil. Do you have examples on how to use Soil?

I currently use a STON persistence model. My current approach to my model is structured specifically to make STON full save and restores simple...

- Wrapper Class which contains a collection of high level objects - high level objects contain collections of next level objects - next level objects contain collections of lower level objects - etc.

While the database size (7.3Mb) is not terribly large, I am writing about 50 of those per day as each save simply saves off the entire old STON and writes a new one. Concurrency is generally not a problem as we have the application automatically save itself every 10 minutes, but every once in a while we do run up against a conflict that would be best handled by a concurrent capable model.

I have roughly nine applications using the same class models, so it would be very helpful to adopt a more OO database approach to persistence... even though we're nowhere near Google :)

Thoughts on where to begin? I'm on the Discord server as well, if you'd prefer to discuss there.

Thanks, Russ

On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl <norbert@hartl.name> wrote:

...

The approach as I understand is simple and valuable until you face concurrency. Nothing in SimplePersistence prevents your data from becoming corrupt in a concurrent situation. I agree that when you start a project you almost never face concurrency because the odds are way too low to have two things at the same time happening. This is because computers are really fast today and the amount of activity on a service is largely overestimated.

But there is still a huge gap between SimplePersistence and those overkill persistence solutions. For exactly that reason I've done Soil (https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project with no dependency and setting it up is a no-brainer. It is way bigger than SimplePersistence but it deals with concurrency and adds query capabilties for your objects. But you have to use transactions to do this.

I started ApptiveGrid just by having the data persisted through saving the image. Next was to STON out the model. Then versioned STON files to add fallback. Then there was Soil (well, first voyage and then omnibase and then Soil) which is the next level to iterate.

So ApptiveGrid is not the next google but it also wouldn't work with SimplePersistence. So good that we have enough tools to improve our projects on.

Norbert

...

Am 15.02.2024 um 06:43 schrieb sean@clipperadams.com:

I’m happy to answer any questions about Simple Persistence. It is a nice framework around (potentially any) serializer. It’s meant to be pluggable but currently uses Fuel out of the box. You just tell it what classes to persist and then create two methods per class to handle materialization/serialization.

For the record, Ramon Leon created it in a blog post. I just ported it to Pharo and have been maintaining it.

The stated reason he created it is that most solutions are complete overkill for 99% of projects that will never “become the next google”. It’s pretty basic, but at least two steps more sophisticated than simply saving the image because it keeps old versions and is easily reloadable into another image.

That said, I’ve been using it almost exclusively for my personal projects and have yet to grow out of it.

-- Russ Whaley whaley.russ@gmail.com

I wasn't particularly advocating any path - but have observed that in larger orgs its a more difficult discussion to tread a different path (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL based tools and equivalently support teams who know the same - and if in that world you may have to play their game (or not - if you have traction). So I'm just observing that we can work in any of those spaces - you can take your pick and apply what makes sense in your environment. Of course I love a rebel technology as much as anyone else - but sometimes their are other battles to fight - or more interesting niches to explore. Tim On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote:

Tim,

What is the thinking behind “Finally you might need something more enterprise and then Gemstone or Voyage…”?

Is it the maturity level of Soil codebase vs. these others? Or is it a belief that a database has to be a complex separate piece of engineering (therefore best outsourced).

Yanni

On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote:

...

__ I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel.

Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it.

Thanks for everyone for giving us so many options.

Tim

I think Voyage with its connection to Mongo is somewhat more palatable to some companies - however I did forget to mention Glorp of course - which does map directly to a SQL database. Not to dis Gemstone in anyway (I do wish OO db's had gained more traction in the past, they are still glorious) - and I do think they might have solutions to map to a SQL db to appease corp teams - although I'm not sure. Again - I think its useful to understand the patterns at play and how we can help folks understand where/how to play. Tim On Sun, 18 Feb 2024, at 10:40 PM, Yanni Chiu wrote:

Agreed that larger orgs and BI teams likely slant toward SQL. But GemStone and Voyage/Mongo wouldn’t address that either. An export from a Soil, GemStone or Mongo db, into a SQL db should address the BI tools.

On Sun, Feb 18, 2024 at 4:35 PM Tim Mackinnon <tim@testit.works> wrote:

...

__ I wasn't particularly advocating any path - but have observed that in larger orgs its a more difficult discussion to tread a different path (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL based tools and equivalently support teams who know the same - and if in that world you may have to play their game (or not - if you have traction). So I'm just observing that we can work in any of those spaces - you can take your pick and apply what makes sense in your environment.

Of course I love a rebel technology as much as anyone else - but sometimes their are other battles to fight - or more interesting niches to explore.

Tim

On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote:

...

Tim,

What is the thinking behind “Finally you might need something more enterprise and then Gemstone or Voyage…”?

Is it the maturity level of Soil codebase vs. these others? Or is it a belief that a database has to be a complex separate piece of engineering (therefore best outsourced).

Yanni

On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote:

...

__ I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel.

Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it.

Thanks for everyone for giving us so many options.

Tim