I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel. Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it. Thanks for everyone for giving us so many options. Tim On Sat, 17 Feb 2024, at 6:46 PM, Russ Whaley wrote: > Norbert, > I'm very interested to investigate Soil. Do you have examples on how to use Soil? > > I currently use a STON persistence model. My current approach to my model is structured specifically to make STON full save and restores simple... > > - Wrapper Class which contains a collection of high level objects > - high level objects contain collections of next level objects > - next level objects contain collections of lower level objects > - etc. > > While the database size (7.3Mb) is not terribly large, I am writing about 50 of those per day as each save simply saves off the entire old STON and writes a new one. Concurrency is generally not a problem as we have the application automatically save itself every 10 minutes, but every once in a while we do run up against a conflict that would be best handled by a concurrent capable model. > > I have roughly nine applications using the same class models, so it would be very helpful to adopt a more OO database approach to persistence... even though we're nowhere near Google :) > > Thoughts on where to begin? I'm on the Discord server as well, if you'd prefer to discuss there. > > Thanks, > Russ > > > On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl <norbert@hartl.name> wrote: >> The approach as I understand is simple and valuable until you face concurrency. Nothing in SimplePersistence prevents your data from becoming corrupt in a concurrent situation. I agree that when you start a project you almost never face concurrency because the odds are way too low to have two things at the same time happening. This is because computers are really fast today and the amount of activity on a service is largely overestimated. >> >> But there is still a huge gap between SimplePersistence and those overkill persistence solutions. For exactly that reason I've done Soil (https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project with no dependency and setting it up is a no-brainer. It is way bigger than SimplePersistence but it deals with concurrency and adds query capabilties for your objects. But you have to use transactions to do this. >> >> I started ApptiveGrid just by having the data persisted through saving the image. Next was to STON out the model. Then versioned STON files to add fallback. Then there was Soil (well, first voyage and then omnibase and then Soil) which is the next level to iterate. >> >> So ApptiveGrid is not the next google but it also wouldn't work with SimplePersistence. So good that we have enough tools to improve our projects on. >> >> Norbert >> >> >>> Am 15.02.2024 um 06:43 schrieb sean@clipperadams.com: >>> >>> I’m happy to answer any questions about Simple Persistence. It is a nice framework around (potentially any) serializer. It’s meant to be pluggable but currently uses Fuel out of the box. You just tell it what classes to persist and then create two methods per class to handle materialization/serialization. >>> >>> For the record, Ramon Leon created it in a blog post. I just ported it to Pharo and have been maintaining it. >>> >>> The stated reason he created it is that most solutions are complete overkill for 99% of projects that will never “become the next google”. It’s pretty basic, but at least two steps more sophisticated than simply saving the image because it keeps old versions and is easily reloadable into another image. >>> >>> That said, I’ve been using it almost exclusively for my personal projects and have yet to grow out of it. >>> >> > > > -- > Russ Whaley > whaley.russ@gmail.com

Tim, What is the thinking behind “Finally you might need something more enterprise and then Gemstone or Voyage…”? Is it the maturity level of Soil codebase vs. these others? Or is it a belief that a database has to be a complex separate piece of engineering (therefore best outsourced). Yanni On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote: > I think Ross (and what Norbert said) nicely alludes to the path people > follow - for really simple persistence, Fuel or simple image saving give > you an instant solution. The next step (assuming no real concurrency > issues) are what Sean has maintained - something that gives you rolling > snapshots and a simple UI/mechanism to recover old versions and then you > probably realise that you need something more transactional and Soil sounds > like it fits that perfectly. Finally you might need something more > enterprise and then Gemstone or Voyage are the direction to travel. > > Having so many options is terrific particularly if you can defer some of > the complexity the latter stages bring. I love it. > > Thanks for everyone for giving us so many options. > > Tim >

I wasn't particularly advocating any path - but have observed that in larger orgs its a more difficult discussion to tread a different path (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL based tools and equivalently support teams who know the same - and if in that world you may have to play their game (or not - if you have traction). So I'm just observing that we can work in any of those spaces - you can take your pick and apply what makes sense in your environment. Of course I love a rebel technology as much as anyone else - but sometimes their are other battles to fight - or more interesting niches to explore. Tim On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote: > Tim, > > What is the thinking behind “Finally you might need something more enterprise and then Gemstone or Voyage…”? > > Is it the maturity level of Soil codebase vs. these others? Or is it a belief that a database has to be a complex separate piece of engineering (therefore best outsourced). > > Yanni > > On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote: >> __ >> I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel. >> >> Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it. >> >> Thanks for everyone for giving us so many options. >> >> Tim

Agreed that larger orgs and BI teams likely slant toward SQL. But GemStone and Voyage/Mongo wouldn’t address that either. An export from a Soil, GemStone or Mongo db, into a SQL db should address the BI tools. On Sun, Feb 18, 2024 at 4:35 PM Tim Mackinnon <tim@testit.works> wrote: > I wasn't particularly advocating any path - but have observed that in > larger orgs its a more difficult discussion to tread a different path > (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL > based tools and equivalently support teams who know the same - and if in > that world you may have to play their game (or not - if you have traction). > So I'm just observing that we can work in any of those spaces - you can > take your pick and apply what makes sense in your environment. > > Of course I love a rebel technology as much as anyone else - but sometimes > their are other battles to fight - or more interesting niches to explore. > > > Tim > > On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote: > > Tim, > > What is the thinking behind “Finally you might need something more > enterprise and then Gemstone or Voyage…”? > > Is it the maturity level of Soil codebase vs. these others? Or is it a > belief that a database has to be a complex separate piece of engineering > (therefore best outsourced). > > Yanni > > On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote: > > > I think Ross (and what Norbert said) nicely alludes to the path people > follow - for really simple persistence, Fuel or simple image saving give > you an instant solution. The next step (assuming no real concurrency > issues) are what Sean has maintained - something that gives you rolling > snapshots and a simple UI/mechanism to recover old versions and then you > probably realise that you need something more transactional and Soil sounds > like it fits that perfectly. Finally you might need something more > enterprise and then Gemstone or Voyage are the direction to travel. > > Having so many options is terrific particularly if you can defer some of > the complexity the latter stages bring. I love it. > > Thanks for everyone for giving us so many options. > > Tim > > >

I think Voyage with its connection to Mongo is somewhat more palatable to some companies - however I did forget to mention Glorp of course - which does map directly to a SQL database. Not to dis Gemstone in anyway (I do wish OO db's had gained more traction in the past, they are still glorious) - and I do think they might have solutions to map to a SQL db to appease corp teams - although I'm not sure. Again - I think its useful to understand the patterns at play and how we can help folks understand where/how to play. Tim On Sun, 18 Feb 2024, at 10:40 PM, Yanni Chiu wrote: > Agreed that larger orgs and BI teams likely slant toward SQL. But GemStone and Voyage/Mongo wouldn’t address that either. An export from a Soil, GemStone or Mongo db, into a SQL db should address the BI tools. > > On Sun, Feb 18, 2024 at 4:35 PM Tim Mackinnon <tim@testit.works> wrote: >> __ >> I wasn't particularly advocating any path - but have observed that in larger orgs its a more difficult discussion to tread a different path (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL based tools and equivalently support teams who know the same - and if in that world you may have to play their game (or not - if you have traction). So I'm just observing that we can work in any of those spaces - you can take your pick and apply what makes sense in your environment. >> >> Of course I love a rebel technology as much as anyone else - but sometimes their are other battles to fight - or more interesting niches to explore. >> >> >> Tim >> >> On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote: >>> Tim, >>> >>> What is the thinking behind “Finally you might need something more enterprise and then Gemstone or Voyage…”? >>> >>> Is it the maturity level of Soil codebase vs. these others? Or is it a belief that a database has to be a complex separate piece of engineering (therefore best outsourced). >>> >>> Yanni >>> >>> On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon <tim@testit.works> wrote: >>>> __ >>>> I think Ross (and what Norbert said) nicely alludes to the path people follow - for really simple persistence, Fuel or simple image saving give you an instant solution. The next step (assuming no real concurrency issues) are what Sean has maintained - something that gives you rolling snapshots and a simple UI/mechanism to recover old versions and then you probably realise that you need something more transactional and Soil sounds like it fits that perfectly. Finally you might need something more enterprise and then Gemstone or Voyage are the direction to travel. >>>> >>>> Having so many options is terrific particularly if you can defer some of the complexity the latter stages bring. I love it. >>>> >>>> Thanks for everyone for giving us so many options. >>>> >>>> Tim >>

On Feb 19, 2024, at 3:17 AM, Tim Mackinnon <tim@testit.works> wrote: > > I do think [GemStone] might have solutions to map to a SQL db to appease corp teams - although I'm not sure. GemConnect (https://gemtalksystems.com/products/gemconnect/) is an add-on product for GemStone that allows you to interact with an Oracle relational database from GemStone. It is used by several customers to dump transactional data to Oracle for reporting. See also GemConnect-for-Postgres (https://github.com/GemTalk/GemConnect-for-Postgres). Less developed is a way to make GemStone look like a relational database. Years (decades!) ago there was a third-party add-on that allowed marketing to check a box in RFPs but it was never taken seriously. More recently (years, but not decades ago), I did a proof-of-concept that demonstrated connecting Microsoft’s ODBC to GemStone (so you could see GemStone data using SQL. See https://github.com/jgfoster/GemStone-ODBC with links to a silent video and slides. James