Easiest light weight cloud/web persistence for Pharo?
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it). Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there. I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?). Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me. Is there something that gives a little table of tradeoffs with some simple ways to get started? Tim
Maybe you can use SQlite3 in a persistent volume? On Mon, Oct 5, 2020, 19:23 Tim Mackinnon <tim@testit.works> wrote:
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it).
Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there.
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Is there something that gives a little table of tradeoffs with some simple ways to get started?
Tim
On Mon, Oct 05, 2020 at 11:23:25PM +0100, Tim Mackinnon wrote:
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Hi Tim, If you are familiar with Docker then starting a database server is easy that way. Assuming you pick a 'boring' product then what matters for your app is the quality of the database interface code on Pharo for that product. For PostgreSQL and MySQL no problem. I've not used Mongo itself nor Voyage so no comment. If you neither want to run a separate database server nor roll your own writing out to Fuel etc, then you're looking at 'embedded' databases. For pure Smalltalk, Omnibase or SimplePersistence. If you are ok with something that uses FFI, then SQLite, Unqlite and probably others. Personally I'd choose SQLite because it is boring - ubiquitous, high quality, and importantly unlike the pure Smalltalk and many embedded key-value databases there is a plethora of external tools that operate on SQLite databases which will help your debugging. For SQL databases you can also use an ORM like Glorp or ReStore. Finally there are also NoSQL-as-a-service like Google Firebase. Many options. Great time sinks. :-) Have fun. - PostgreSQL - https://github.com/svenvc/p3 - MySQL - https://github.com/pharo-rdbms/Pharo-MySQL - Omnibase - https://github.com/sebastianconcept/OmniBase - SimplePersistence - https://github.com/seandenigris/Simple-Persistence - SQLite - https://github.com/pharo-rdbms/Pharo-SQLite3 - Unqlite - https://github.com/mumez/PunQLite - Glorp - https://github.com/pharo-rdbms/glorp - ReStore - https://github.com/rko281/ReStoreForPharo - Firebase - https://github.com/psvensson/firebase-st - Choose Boring Technology - http://boringtechnology.club/ Pierce
Hi Pierce, On Mon, Oct 5, 2020 at 10:22 PM Pierce Ng <pierce@samadhiweb.com> wrote:
On Mon, Oct 05, 2020 at 11:23:25PM +0100, Tim Mackinnon wrote: Many options. Great time sinks. :-) Have fun.
<snipped options>
- Choose Boring Technology - http://boringtechnology.club/
This made my day. Thank you :-) Esteban A. Maringolo
- Choose Boring Technology - http://boringtechnology.club/ This made my day. Thank you :-)
Had never heard of this guy, but his site feels like home to me. Makes me a proud user of Smalltalk and DB2 after 25+ years ;-) He forgot to mention that boring technology can be quite interesting and cool, even if it is about to turn 50 years... And he's absolutely right about the fact that shipiing something useful for users makes you much more happy (and busy) than shipping something on top of an incredibly complicated and bloated technology stack... Joachim -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
On 6/10/20 2:25 p. m., jtuchel@objektfabrik.de wrote:
- Choose Boring Technology - http://boringtechnology.club/ This made my day. Thank you :-)
Had never heard of this guy, but his site feels like home to me. Makes me a proud user of Smalltalk and DB2 after 25+ years ;-)
He forgot to mention that boring technology can be quite interesting and cool, even if it is about to turn 50 years... And he's absolutely right about the fact that shipiing something useful for users makes you much more happy (and busy) than shipping something on top of an incredibly complicated and bloated technology stack...
Unfortunately software tech industry is co-opted by hype, trends and big numbers. In that sense, Pharo or Fossil or technology that doesn't fit that trio seems non boring for those used to evaluate safeness there. In my hackerspace the ones used to more popular tech look at Pharo as too strange (risky?), but those who don't have any preconception, mainly non-programmers (by education or trade) have no problem accepting Pharo, Fossil and more simple and agile technologies. We deliver value swiftly from the margins, without being trapped by hype o big numbers. I wonder how this organic, resilient and innovative growth can be more visible, despite of being so far away of the usual places where most tech related individuals, communities and investors look for. Offray
Hi Tim, I've been running Seaside applications on Hetzner cloud servers for more than a year now, with great pleasure and success: https://www.hetzner.com/cloud I guess their servers are similar to Digital Ocean, although I haven't followed the development of their products and solutions for quite a while. Setting up a new server at Hetzner is a breeze, and you can start already for as low as â¬2,49 per month! We're using Voyage on MongoDB for persistence. After learning some hard lessons (and I'm sure there are more to come ;-)), I really enjoy the unobtrusiveness of it. Most of the time, it doesn't require much attention and allows me to add persistence to real OO designs quickly and easily. I find it a welcome change from the relational database work I used to (need to) do, back when I was still doing Java. The 'everything an object' principle of Pharo/Smalltalk really makes it shine. I can't help you with a list of tradeoffs though. If you come across a set of arguments, I'd be happy to give feedback. By the way, I forked Sven's pharo-server-tools project (here: https://github.com/objectguild/pharo-server-tools) and have a routine going that suits me well enough. Still lots of room for improvement, but it's OK for my current needs. Future plans are to use the Hetzner API to provision a new server and use something like Chef or Ansible to install/configure it automatically to be ready to deploy a Seaside application. I'd like to integrate this into a full service CI/CD pipeline in the future, to be able to do automated production deployments without service interruption if possible. For this scenario, I would really also like to switch to using GemStone for persistence. Hope this helps! Let me know what you decide and I might be able to help with some technical stuff. Kind regards, Jonathan van Alteren Founding Member | Object Guild B.V. Sustainable Software for Purpose-Driven Organizations jvalteren@objectguild.com On 6 Oct 2020, 00:23 +0200, Tim Mackinnon <tim@testit.works>, wrote:
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it).
Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there.
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Is there something that gives a little table of tradeoffs with some simple ways to get started?
Tim
Some great answers everyone - really appreciate it. I think that all things considered, SimplePersistence seems like a very easy place to start, and then I can "upgrade" when my super awesome idea (not), exceeds its bounds. I wonder if these answers should go somewhere in a tips section somewhere, and get updated yearly. @jonathan I had forgotten about those server tools - I think the last time I did something it predated some earlier tips the Sven had given, so will check it out. Back then, I did have a GitLab build pipeline autodeploying for me, so I'm hoping I can re-incarnate all of that so I can get my little app running with no fuss... lets see. While I have a love/hate with Docker - I did wonder if there was something for Pharo that just let me build my image into a container and then it would just work with little system knowledge needed (as I keep forgetting all the voodoo between times when I need it). Maybe there is, or maybe it might come one day soon... Tim On Tue, 6 Oct 2020, at 9:40 AM, Jonathan van Alteren wrote:
Hi Tim,
I've been running Seaside applications on Hetzner cloud servers for more than a year now, with great pleasure and success: https://www.hetzner.com/cloud I guess their servers are similar to Digital Ocean, although I haven't followed the development of their products and solutions for quite a while. Setting up a new server at Hetzner is a breeze, and you can start already for as low as â¬2,49 per month!
We're using Voyage on MongoDB for persistence. After learning some hard lessons (and I'm sure there are more to come ;-)), I really enjoy the unobtrusiveness of it. Most of the time, it doesn't require much attention and allows me to add persistence to real OO designs quickly and easily. I find it a welcome change from the relational database work I used to (need to) do, back when I was still doing Java. The 'everything an object' principle of Pharo/Smalltalk really makes it shine.
I can't help you with a list of tradeoffs though. If you come across a set of arguments, I'd be happy to give feedback.
By the way, I forked Sven's pharo-server-tools project (here: https://github.com/objectguild/pharo-server-tools) and have a routine going that suits me well enough. Still lots of room for improvement, but it's OK for my current needs.
Future plans are to use the Hetzner API to provision a new server and use something like Chef or Ansible to install/configure it automatically to be ready to deploy a Seaside application. I'd like to integrate this into a full service CI/CD pipeline in the future, to be able to do automated production deployments without service interruption if possible. For this scenario, I would really also like to switch to using GemStone for persistence.
Hope this helps! Let me know what you decide and I might be able to help with some technical stuff.
Kind regards,
Jonathan van Alteren
Founding Member | Object Guild B.V. *Sustainable Software for Purpose-Driven Organizations*
jvalteren@objectguild.com On 6 Oct 2020, 00:23 +0200, Tim Mackinnon <tim@testit.works>, wrote:
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it).
Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there.
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Is there something that gives a little table of tradeoffs with some simple ways to get started?
Tim
Tim, Am 06.10.20 um 14:47 schrieb Tim Mackinnon:
Some great answers everyone - really appreciate it.
I think that all things considered, SimplePersistence seems like a very easy place to start, and then I can "upgrade" when my super awesome idea (not), exceeds its bounds.
Sigh. Forget about the idea that it will be easy to switch your persistence later. There are lots of little (and not so little) things to consider for each approach, be it an ORM, an object database or just a bunch of serialized objects. It starts with the bookkeeping for references between objects. Is it enough to remove an object from a Collection or will it somehow survive in your db and come back as a surprise after you started a new transction? Will a deleted object be deleted or just unreachable? Do I have to care about all possible paths to this object if I really want it to be deleted? Will my ORB possible re-insert it in the next transaction because I don't understand how an ORM works? Don't fool yourself into thinking there is something like a transparent persistence layer. If anybody ever says so, walk away, say thank you and forget about that person as fast as you can. I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. I am talking about experiences made with both OODBs and ORMs like Glorp. I've been through tough times with an architecture that I thought to be a good idea: keep persistence in a separate layer that just spits out objects and swallows new or changed ones. Works well in a prototype. But on top of Glorp and with a few more than a hand full of objects, you end up wanting to write real database queries. So you need to get a hold of something that represents the database, say a Session object. You end up either trying to wrapper the query building in your clever layer and end up with friggin slow performance or give up on your nice layering to make your application at least bearable. Sometimes you even give up navigating between objects and replace a simple getter with a DB query to get good performance (ever waited for 600 proxies in a collection to each deserialize itself when accessing the #items of your Stock object? - you better write a sql query fetching them all at once...) You need query by example? Good luck in an Object DB. You either use DB specific queries and hand-tuned indices, or you navigate your object net and have the user wait a little. Or you design your object model around the search paths (which is hard once your model has evolved a while). In Mongo/Voyage, you need to make a decision of what objects will always be tied closely together (referenced by a root) and can be retrieved on their own. You think in object trees rather than objects. We've thrown CouchDB out of our project. Not because it is a bad NoSQL database, but because we couldn't make it work well in our overall architecture in which the primary store is a relational Database. So please be aware that once your app gets a little more than just a prototype, you need to make a decision: relational, dictinaries, tree-like I guess this is not really helpful advice. I can't really tell you what is right. I am cursing at Glorp every few days, but I also love it for many great things it does for us. And I've had a few very "interesting" lessons along the way, and am still in the process of learning about its limits. Joachim
I wonder if these answers should go somewhere in a tips section somewhere, and get updated yearly.
@jonathan I had forgotten about those server tools - I think the last time I did something it predated some earlier tips the Sven had given, so will check it out. Back then, I did have a GitLab build pipeline autodeploying for me, so I'm hoping I can re-incarnate all of that so I can get my little app running with no fuss... lets see.
While I have a love/hate with Docker - I did wonder if there was something for Pharo that just let me build my image into a container and then it would just work with little system knowledge needed (as I keep forgetting all the voodoo between times when I need it). Maybe there is, or maybe it might come one day soon...
Tim
On Tue, 6 Oct 2020, at 9:40 AM, Jonathan van Alteren wrote:
Hi Tim,
I've been running Seaside applications on Hetzner cloud servers for more than a year now, with great pleasure and success: https://www.hetzner.com/cloud I guess their servers are similar to Digital Ocean, although I haven't followed the development of their products and solutions for quite a while. Setting up a new server at Hetzner is a breeze, and you can start already for as low as â¬2,49 per month!
We're using Voyage on MongoDB for persistence. After learning some hard lessons (and I'm sure there are more to come ;-)), I really enjoy the unobtrusiveness of it. Most of the time, it doesn't require much attention and allows me to add persistence to real OO designs quickly and easily. I find it a welcome change from the relational database work I used to (need to) do, back when I was still doing Java. The 'everything an object' principle of Pharo/Smalltalk really makes it shine.
I can't help you with a list of tradeoffs though. If you come across a set of arguments, I'd be happy to give feedback.
By the way, I forked Sven's pharo-server-tools project (here: https://github.com/objectguild/pharo-server-tools) and have a routine going that suits me well enough. Still lots of room for improvement, but it's OK for my current needs.
Future plans are to use the Hetzner API to provision a new server and use something like Chef or Ansible to install/configure it automatically to be ready to deploy a Seaside application. I'd like to integrate this into a full service CI/CD pipeline in the future, to be able to do automated production deployments without service interruption if possible. For this scenario, I would really also like to switch to using GemStone for persistence.
Hope this helps! Let me know what you decide and I might be able to help with some technical stuff.
Kind regards,
Jonathan van Alteren
Founding Member | Object Guild B.V. /Sustainable Software for Purpose-Driven Organizations/
jvalteren@objectguild.com On 6 Oct 2020, 00:23 +0200, Tim Mackinnon <tim@testit.works>, wrote:
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it).
Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there.
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Is there something that gives a little table of tradeoffs with some simple ways to get started?
Tim
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't.
Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph. Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help. NB I have maintained and extended the library, but it is the work of Ramon Leon ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Sean, thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use. I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-) In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart. The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet. Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make. But, hey, I said all of that before. So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost. I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes? I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds. But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture. Joachim Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone. I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case, however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do that⦠and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠however, move too soon and you get bogged down with the details and you lose site of an MVP. It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy). I suppose this is where the Rails scaffolding was/is? such a jumpstart, you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work. It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus on⦠Anyway, that login screen⦠oh crap I have to write one of those⦠Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
Oh.. and blimey , look what I just found in my spam bucket âthey are watching me⦠â - > "We are excited to announce the General Availability of DigitalOcean App Platform <https://pages.news.digitalocean.com/I0o2WEY06sliXV32090ID00>, a reimagined platform as a service (PaaS) that makes it much simpler and faster to build, deploy, and scale apps. App Platform does the heavy lifting of managing infrastructure, app runtimes, and dependencies â so you can focus on what matters the most: building awesome apps.â Possibly with its Docker support, perhaps Pharo might slot it, and they take care of everything for me⦠well hereâs hoping right? Tim
On 6 Oct 2020, at 21:41, Tim Mackinnon <tim@testit.works> wrote:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case, however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do that⦠and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠however, move too soon and you get bogged down with the details and you lose site of an MVP.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy). I suppose this is where the Rails scaffolding was/is? such a jumpstart, you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work. It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
Hi, I have been using STON and Fossil for persistance and I'm pretty happy with this combination as a future proof, simple and even reproducible solution for my storage needs, including web hosting. For example, for our last project about IndieWeb with pocket infrastructures[1], a STON powered Grafoscopio[2] notebook[2a] describes the computations in Brea[3] that create the web site at [1], everything else are just plain markdown files with YAML metadata blocks[3a] and mustache templates[4] that combined create the HTML files. Maybe this combination is too "static web site" oriented, but now that you are looking for a panoramic view on persistence, could be helpful to put this in the radar. [1] https://mutabit.com/repos.fossil/indieweb/ [2] https://mutabit.com/grafoscopio/en.html [2a] https://mutabit.com/repos.fossil/indieweb/file?name=indieweb.ston&ci=trunk [3a] https://mutabit.com/repos.fossil/indieweb/tree?ci=trunk&type=tree&name=docs/... [3] https://mutabit.com/repos.fossil/brea/ [4] https://mutabit.com/repos.fossil/indieweb/doc/trunk/docs/es/pagina.mus.html All my projects are run in a 5 USD Digital Ocean instance. I have not need for anything else, as computing, when necessary, is distributed in the desktops of the users that install and run Grafoscopio and Brea. The server only synchronizes and stores files via Fossil repositories. Cheers, Offray On 6/10/20 3:41 p. m., Tim Mackinnon wrote:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case, however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do that⦠and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠however, move too soon and you get bogged down with the details and you lose site of an MVP.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy). I suppose this is where the Rails scaffolding was/is? such a jumpstart, you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work. It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
On Tue, Oct 06, 2020 at 09:41:21PM +0100, Tim Mackinnon wrote:
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
For Seaside, there is TF-Login. And oh, it rolled its own file-based versioning object persistence. Version that I ported to Pharo 7 and Seaside 3.3: - https://github.com/PierceNg/TF-Login/tree/pharo7 Feature branch for better security: - https://github.com/PierceNg/TF-Login/tree/password Some of the classes are too tightly coupled IMO. Also I'm tending towards an API-first approach, which needs OAuth2 and OpenID Connect, meaning either more pieces to integrate or totally not using this. Pierce
Am 06.10.20 um 22:41 schrieb Tim Mackinnon:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case,
yes, sounds like it. You mentioned it is not a super-complex 30-year project you are planning. It sounds like you can load all the data into memory in a few msec and just go form there, Saving the whole model as a file also sounds like perfectly doable in such a project. So, heck, you could probably simply just save the image and be good. A chunk of JSON, Fuel, whatever object serialization is probably second best. The great thing about this: nothing external (to the image) to install or maintain. It is just you and the file system.
however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do thatâ¦
Oh no, don't fear this too much. That's not what I wanted to point out. Most projects will do well on either NoSQL or using an ORM or an OODB. Almost anything is possible on all of them. We do boring business stuff (accounting) using Glorp as an ORM and DB2. Another well-known project in the Pharo world does something extremely similar (travel cost) with Voyage/Mongo. It is not so much a question of lost opportunities. The problem here is that you have to deal with the very same problems (concurrent access and isolation, fast lookups, caching, ghost objects) on each of these technologies, but differently. It is the different approaches to the same problems that make switching from one to the other that makes the decision so hard. Example: we had a hard time finding out why some objects never went away although we had deleted them from the database. It took a while of logging all SQL statements until we found out first that they were really deleted in the Transaction when the user clicked OK. But they were inserted back in the next, possibly completely unrelated, transaction, because we had some dangling backpointers that tricked glorp into thinking "oops, there is this bunch of new objects I have to insert now". This is why I used object deletion as an example. Smalltalk has no concept of deleting an object. Databases do. So whatever approach you chose, deleting an object looks different. You either make the object completely (!) unreachable from the rest of the object model and thus just make it irrelevant, or you tell the databse to just throw it away. Sounds like no big deal. But it can be.
and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠The hard part here: the example with the reinserted objects is what is called an unknown unknown in the boringtech site ;-) You would need a test to check whether an object you deleted really isn't reoccuring after the next Transaction. You'd have to imagine that there might be a small chance that you delete an object successfully, but for some reason you distorted the ORMs internal bookkeeping and trocked it into inserting teh object back in some consecutive Transaction. A simple fact once you understood why it happened, but I am 99% sure almost nobody would come around the corner and say: well, we'll have to write a test for this scenario where an object still lingers somewhere and gets reinserted. The whole team would by this guy a beer and move to another table. however, move too soon and you get bogged down with the details and you lose site of an MVP.
move and live with your decision. Or be prepared for a much more complicated transition than you thought. The problem is that nobody (at least that I am aware of) has come up with an abstraction good enough to make the persistence implications irrelevant enough and provide good performance and feature richness at the same time. I know it's been tried.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy).
Not sure about the need for Docker. You just throw more tech at the problem. I mean, installing and setting up PostgreSQL or MySQL on a Linux distro these days is matter of a few commands. apt install, enter a db administrator password, answer a few questions and go. Same with Mongo or CouchDB.
I suppose this is where the Rails scaffolding was/is? such a jumpstart,
Don't get me started ;-) Scaffolding is great if you need to sell a technology to management on a few slides. Slip in some comment like "works on existing database schemas too, at no extra cost" and these guys are ready to write whatever it takes onto a cheque. I mean, come on, that guy showed us how to make the whole mapping and transaction management for a flight booking system in 15 minutes, how much harder can our project be? That's how most multi-million desasters start.
you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work.
I totally agree. These days, everything is easy. Just install NodeJS and some super sophisticated package manager and let it install everything for you in just 5 minutes and you're ready to go. Just make sure you have at least 12 GB of free memory and a good machine with fast internet. Your data will be stored in the *bling, stars and glitter * Cloud and you just forget about this detail. You want a login screen, sure, just type 'init -n -tfgr "login"' and provide your github credentials. We'll create a directory structure for you with a src, an html, a helpers and a .gitignore file. Don't worry, some of that you'll never touch. Oh, sure just make sure you have a Facebook account and consent with the cookie policy of that super-georgous technology company that provides this piece of code. Not sure if that is what is really needed, but it's where we're heading at the moment. Open any JS related book these days and read the first chapter. I bet it's going to step you through the installation process of at least three major super-cool open source, reliable and tested frameworks or infrastructure monsters before you can start. From there on, everything is a breeze. It's like a free lunch. But boy, somebody will have to run this stuff for a while. I think the Smalltalk vendors, both commercial and open source are making great progress here, and I like the fact that we are not followng the bloating trend. Maybe the "not invented here" meme makes sense to some degree. What we as developers need is some understanding of what we're doing and not so much the latest bells and whistles. Storing data in a file may be unsophisticated, but it does the job even under hard conditions. ..but I am getting slightly off-topic ;-) Joachim
It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
Joachim - thanks again for adding more insight, have learned (and been reminded) at lot from this thread - and certainly have a few paths forward with some suitable warnings too. As my project is a "spare time" one, I can at least enjoy the journey and test some of this out. Tim On Wed, 7 Oct 2020, at 5:04 AM, jtuchel@objektfabrik.de wrote:
Am 06.10.20 um 22:41 schrieb Tim Mackinnon:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case,
yes, sounds like it. You mentioned it is not a super-complex 30-year project you are planning. It sounds like you can load all the data into memory in a few msec and just go form there, Saving the whole model as a file also sounds like perfectly doable in such a project. So, heck, you could probably simply just save the image and be good. A chunk of JSON, Fuel, whatever object serialization is probably second best.
The great thing about this: nothing external (to the image) to install or maintain. It is just you and the file system.
however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do thatâ¦
Oh no, don't fear this too much. That's not what I wanted to point out. Most projects will do well on either NoSQL or using an ORM or an OODB. Almost anything is possible on all of them. We do boring business stuff (accounting) using Glorp as an ORM and DB2. Another well-known project in the Pharo world does something extremely similar (travel cost) with Voyage/Mongo. It is not so much a question of lost opportunities. The problem here is that you have to deal with the very same problems (concurrent access and isolation, fast lookups, caching, ghost objects) on each of these technologies, but differently. It is the different approaches to the same problems that make switching from one to the other that makes the decision so hard.
Example: we had a hard time finding out why some objects never went away although we had deleted them from the database. It took a while of logging all SQL statements until we found out first that they were really deleted in the Transaction when the user clicked OK. But they were inserted back in the next, possibly completely unrelated, transaction, because we had some dangling backpointers that tricked glorp into thinking "oops, there is this bunch of new objects I have to insert now". This is why I used object deletion as an example. Smalltalk has no concept of deleting an object. Databases do. So whatever approach you chose, deleting an object looks different. You either make the object completely (!) unreachable from the rest of the object model and thus just make it irrelevant, or you tell the databse to just throw it away. Sounds like no big deal. But it can be.
and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠The hard part here: the example with the reinserted objects is what is called an unknown unknown in the boringtech site ;-) You would need a test to check whether an object you deleted really isn't reoccuring after the next Transaction. You'd have to imagine that there might be a small chance that you delete an object successfully, but for some reason you distorted the ORMs internal bookkeeping and trocked it into inserting teh object back in some consecutive Transaction. A simple fact once you understood why it happened, but I am 99% sure almost nobody would come around the corner and say: well, we'll have to write a test for this scenario where an object still lingers somewhere and gets reinserted. The whole team would by this guy a beer and move to another table. however, move too soon and you get bogged down with the details and you lose site of an MVP.
move and live with your decision. Or be prepared for a much more complicated transition than you thought.
The problem is that nobody (at least that I am aware of) has come up with an abstraction good enough to make the persistence implications irrelevant enough and provide good performance and feature richness at the same time. I know it's been tried.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy).
Not sure about the need for Docker. You just throw more tech at the problem. I mean, installing and setting up PostgreSQL or MySQL on a Linux distro these days is matter of a few commands. apt install, enter a db administrator password, answer a few questions and go. Same with Mongo or CouchDB.
I suppose this is where the Rails scaffolding was/is? such a jumpstart,
Don't get me started ;-)
Scaffolding is great if you need to sell a technology to management on a few slides. Slip in some comment like "works on existing database schemas too, at no extra cost" and these guys are ready to write whatever it takes onto a cheque. I mean, come on, that guy showed us how to make the whole mapping and transaction management for a flight booking system in 15 minutes, how much harder can our project be? That's how most multi-million desasters start.
you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work.
I totally agree. These days, everything is easy. Just install NodeJS and some super sophisticated package manager and let it install everything for you in just 5 minutes and you're ready to go. Just make sure you have at least 12 GB of free memory and a good machine with fast internet. Your data will be stored in the *bling, stars and glitter * Cloud and you just forget about this detail. You want a login screen, sure, just type 'init -n -tfgr "login"' and provide your github credentials. We'll create a directory structure for you with a src, an html, a helpers and a .gitignore file. Don't worry, some of that you'll never touch. Oh, sure just make sure you have a Facebook account and consent with the cookie policy of that super-georgous technology company that provides this piece of code.
Not sure if that is what is really needed, but it's where we're heading at the moment. Open any JS related book these days and read the first chapter. I bet it's going to step you through the installation process of at least three major super-cool open source, reliable and tested frameworks or infrastructure monsters before you can start. From there on, everything is a breeze. It's like a free lunch.
But boy, somebody will have to run this stuff for a while.
I think the Smalltalk vendors, both commercial and open source are making great progress here, and I like the fact that we are not followng the bloating trend. Maybe the "not invented here" meme makes sense to some degree. What we as developers need is some understanding of what we're doing and not so much the latest bells and whistles. Storing data in a file may be unsophisticated, but it does the job even under hard conditions.
..but I am getting slightly off-topic ;-)
Joachim
It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
I have been using STON for a while now. It does a really nice job of keeping everything just the way I need it - and the read/write - even for larger files process very quickly. Limitations are full writes, no updates, but hey, it is incredibly easy to use and works in both v8 and v9. I built a flashcard application in Pharo as well, and the STON files load in a fraction of a second. Cheers! Russ On Wed, Oct 7, 2020 at 12:04 AM jtuchel@objektfabrik.de < jtuchel@objektfabrik.de> wrote:
Am 06.10.20 um 22:41 schrieb Tim Mackinnon:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case,
yes, sounds like it. You mentioned it is not a super-complex 30-year
project you are planning. It sounds like you can load all the data into
memory in a few msec and just go form there, Saving the whole model as a
file also sounds like perfectly doable in such a project. So, heck, you
could probably simply just save the image and be good. A chunk of JSON,
Fuel, whatever object serialization is probably second best.
The great thing about this: nothing external (to the image) to install
or maintain. It is just you and the file system.
however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do thatâ¦
Oh no, don't fear this too much. That's not what I wanted to point out.
Most projects will do well on either NoSQL or using an ORM or an OODB.
Almost anything is possible on all of them. We do boring business stuff
(accounting) using Glorp as an ORM and DB2. Another well-known project
in the Pharo world does something extremely similar (travel cost) with
Voyage/Mongo. It is not so much a question of lost opportunities. The
problem here is that you have to deal with the very same problems
(concurrent access and isolation, fast lookups, caching, ghost objects)
on each of these technologies, but differently. It is the different
approaches to the same problems that make switching from one to the
other that makes the decision so hard.
Example: we had a hard time finding out why some objects never went away
although we had deleted them from the database. It took a while of
logging all SQL statements until we found out first that they were
really deleted in the Transaction when the user clicked OK. But they
were inserted back in the next, possibly completely unrelated,
transaction, because we had some dangling backpointers that tricked
glorp into thinking "oops, there is this bunch of new objects I have to
insert now". This is why I used object deletion as an example. Smalltalk
has no concept of deleting an object. Databases do. So whatever approach
you chose, deleting an object looks different. You either make the
object completely (!) unreachable from the rest of the object model and
thus just make it irrelevant, or you tell the databse to just throw it
away. Sounds like no big deal. But it can be.
and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robustâ¦
The hard part here: the example with the reinserted objects is what is
called an unknown unknown in the boringtech site ;-) You would need a
test to check whether an object you deleted really isn't reoccuring
after the next Transaction. You'd have to imagine that there might be a
small chance that you delete an object successfully, but for some reason
you distorted the ORMs internal bookkeeping and trocked it into
inserting teh object back in some consecutive Transaction. A simple fact
once you understood why it happened, but I am 99% sure almost nobody
would come around the corner and say: well, we'll have to write a test
for this scenario where an object still lingers somewhere and gets
reinserted. The whole team would by this guy a beer and move to another
table.
however, move too soon and you get bogged down with the details and you lose site of an MVP.
move and live with your decision. Or be prepared for a much more
complicated transition than you thought.
The problem is that nobody (at least that I am aware of) has come up
with an abstraction good enough to make the persistence implications
irrelevant enough and provide good performance and feature richness at
the same time. I know it's been tried.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy).
Not sure about the need for Docker. You just throw more tech at the
problem. I mean, installing and setting up PostgreSQL or MySQL on a
Linux distro these days is matter of a few commands. apt install, enter
a db administrator password, answer a few questions and go. Same with
Mongo or CouchDB.
I suppose this is where the Rails scaffolding was/is? such a jumpstart,
Don't get me started ;-)
Scaffolding is great if you need to sell a technology to management on a
few slides. Slip in some comment like "works on existing database
schemas too, at no extra cost" and these guys are ready to write
whatever it takes onto a cheque. I mean, come on, that guy showed us how
to make the whole mapping and transaction management for a flight
booking system in 15 minutes, how much harder can our project be? That's
how most multi-million desasters start.
you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work.
I totally agree. These days, everything is easy. Just install NodeJS and
some super sophisticated package manager and let it install everything
for you in just 5 minutes and you're ready to go. Just make sure you
have at least 12 GB of free memory and a good machine with fast
internet. Your data will be stored in the *bling, stars and glitter *
Cloud and you just forget about this detail. You want a login screen,
sure, just type 'init -n -tfgr "login"' and provide your github
credentials. We'll create a directory structure for you with a src, an
html, a helpers and a .gitignore file. Don't worry, some of that you'll
never touch. Oh, sure just make sure you have a Facebook account and
consent with the cookie policy of that super-georgous technology company
that provides this piece of code.
Not sure if that is what is really needed, but it's where we're heading
at the moment. Open any JS related book these days and read the first
chapter. I bet it's going to step you through the installation process
of at least three major super-cool open source, reliable and tested
frameworks or infrastructure monsters before you can start. From there
on, everything is a breeze. It's like a free lunch.
But boy, somebody will have to run this stuff for a while.
I think the Smalltalk vendors, both commercial and open source are
making great progress here, and I like the fact that we are not followng
the bloating trend. Maybe the "not invented here" meme makes sense to
some degree. What we as developers need is some understanding of what
we're doing and not so much the latest bells and whistles. Storing data
in a file may be unsophisticated, but it does the job even under hard
conditions.
..but I am getting slightly off-topic ;-)
Joachim
It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects.
What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your
persistence later....I am not commenting on SimplePersistence here, I
don't even know what it
does or doesn't.
Joachim,
Thanks for this interesting perspective. I've never had the (mis?!)fortune
of a project growing enough to force me to make those tough choices! For
SimplePersistence I will say that I view it as a way to *delay* making *any*
choices until you are forced to. It's really just a layer of sugar on top of
Fuel (it used to use the old school Squeak equivalent serialization
mechanism - I forget the name and that might still work). You tell it what
classes to serialize. Implement two methods for each class that get and set
the data, and then it saves the whole thing as one object graph.
Tim,
If you use SimplePersistence, please keep me posted about your experience.
I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon
Leon
-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
-- Russ Whaley whaley.russ@gmail.com
Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure). I recall there is a git repo for Pharo tips/tricks - I will try and curate this list over there as a few people have commented how useful this has been, and appreciate everyone who has chipped in. Tim On Thu, 8 Oct 2020, at 1:43 AM, Russ Whaley wrote:
I have been using STON for a while now. It does a really nice job of keeping everything just the way I need it - and the read/write - even for larger files process very quickly. Limitations are full writes, no updates, but hey, it is incredibly easy to use and works in both v8 and v9. I built a flashcard application in Pharo as well, and the STON files load in a fraction of a second.
Cheers! Russ
On Wed, Oct 7, 2020 at 12:04 AM jtuchel@objektfabrik.de <jtuchel@objektfabrik.de> wrote:
Am 06.10.20 um 22:41 schrieb Tim Mackinnon:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case,
yes, sounds like it. You mentioned it is not a super-complex 30-year
project you are planning. It sounds like you can load all the data into
memory in a few msec and just go form there, Saving the whole model as a
file also sounds like perfectly doable in such a project. So, heck, you
could probably simply just save the image and be good. A chunk of JSON,
Fuel, whatever object serialization is probably second best.
The great thing about this: nothing external (to the image) to install
or maintain. It is just you and the file system.
however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do thatâ¦
Oh no, don't fear this too much. That's not what I wanted to point out.
Most projects will do well on either NoSQL or using an ORM or an OODB.
Almost anything is possible on all of them. We do boring business stuff
(accounting) using Glorp as an ORM and DB2. Another well-known project
in the Pharo world does something extremely similar (travel cost) with
Voyage/Mongo. It is not so much a question of lost opportunities. The
problem here is that you have to deal with the very same problems
(concurrent access and isolation, fast lookups, caching, ghost objects)
on each of these technologies, but differently. It is the different
approaches to the same problems that make switching from one to the
other that makes the decision so hard.
Example: we had a hard time finding out why some objects never went away
although we had deleted them from the database. It took a while of
logging all SQL statements until we found out first that they were
really deleted in the Transaction when the user clicked OK. But they
were inserted back in the next, possibly completely unrelated,
transaction, because we had some dangling backpointers that tricked
glorp into thinking "oops, there is this bunch of new objects I have to
insert now". This is why I used object deletion as an example. Smalltalk
has no concept of deleting an object. Databases do. So whatever approach
you chose, deleting an object looks different. You either make the
object completely (!) unreachable from the rest of the object model and
thus just make it irrelevant, or you tell the databse to just throw it
away. Sounds like no big deal. But it can be.
and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robustâ¦
The hard part here: the example with the reinserted objects is what is
called an unknown unknown in the boringtech site ;-) You would need a
test to check whether an object you deleted really isn't reoccuring
after the next Transaction. You'd have to imagine that there might be a
small chance that you delete an object successfully, but for some reason
you distorted the ORMs internal bookkeeping and trocked it into
inserting teh object back in some consecutive Transaction. A simple fact
once you understood why it happened, but I am 99% sure almost nobody
would come around the corner and say: well, we'll have to write a test
for this scenario where an object still lingers somewhere and gets
reinserted. The whole team would by this guy a beer and move to another
table.
however, move too soon and you get bogged down with the details and you lose site of an MVP.
move and live with your decision. Or be prepared for a much more
complicated transition than you thought.
The problem is that nobody (at least that I am aware of) has come up
with an abstraction good enough to make the persistence implications
irrelevant enough and provide good performance and feature richness at
the same time. I know it's been tried.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy).
Not sure about the need for Docker. You just throw more tech at the
problem. I mean, installing and setting up PostgreSQL or MySQL on a
Linux distro these days is matter of a few commands. apt install, enter
a db administrator password, answer a few questions and go. Same with
Mongo or CouchDB.
I suppose this is where the Rails scaffolding was/is? such a jumpstart,
Don't get me started ;-)
Scaffolding is great if you need to sell a technology to management on a
few slides. Slip in some comment like "works on existing database
schemas too, at no extra cost" and these guys are ready to write
whatever it takes onto a cheque. I mean, come on, that guy showed us how
to make the whole mapping and transaction management for a flight
booking system in 15 minutes, how much harder can our project be? That's
how most multi-million desasters start.
you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work.
I totally agree. These days, everything is easy. Just install NodeJS and
some super sophisticated package manager and let it install everything
for you in just 5 minutes and you're ready to go. Just make sure you
have at least 12 GB of free memory and a good machine with fast
internet. Your data will be stored in the *bling, stars and glitter *
Cloud and you just forget about this detail. You want a login screen,
sure, just type 'init -n -tfgr "login"' and provide your github
credentials. We'll create a directory structure for you with a src, an
html, a helpers and a .gitignore file. Don't worry, some of that you'll
never touch. Oh, sure just make sure you have a Facebook account and
consent with the cookie policy of that super-georgous technology company
that provides this piece of code.
Not sure if that is what is really needed, but it's where we're heading
at the moment. Open any JS related book these days and read the first
chapter. I bet it's going to step you through the installation process
of at least three major super-cool open source, reliable and tested
frameworks or infrastructure monsters before you can start. From there
on, everything is a breeze. It's like a free lunch.
But boy, somebody will have to run this stuff for a while.
I think the Smalltalk vendors, both commercial and open source are
making great progress here, and I like the fact that we are not followng
the bloating trend. Maybe the "not invented here" meme makes sense to
some degree. What we as developers need is some understanding of what
we're doing and not so much the latest bells and whistles. Storing data
in a file may be unsophisticated, but it does the job even under hard
conditions.
..but I am getting slightly off-topic ;-)
Joachim
It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects.
What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your
persistence later....I am not commenting on SimplePersistence here, I
don't even know what it
does or doesn't.
Joachim,
Thanks for this interesting perspective. I've never had the (mis?!)fortune
of a project growing enough to force me to make those tough choices! For
SimplePersistence I will say that I view it as a way to *delay* making *any*
choices until you are forced to. It's really just a layer of sugar on top of
Fuel (it used to use the old school Squeak equivalent serialization
mechanism - I forget the name and that might still work). You tell it what
classes to serialize. Implement two methods for each class that get and set
the data, and then it saves the whole thing as one object graph.
Tim,
If you use SimplePersistence, please keep me posted about your experience.
I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon
Leon
-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
-- Russ Whaley whaley.russ@gmail.com
Tim Mackinnon wrote
Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure).
It currently uses Fuel, but the serializer/materializer is abstracted, so STON could probably be plugged in easily ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
On 9 Oct 2020, at 12:13, Sean P. DeNigris <sean@clipperadams.com> wrote:
Tim Mackinnon wrote
Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure).
It currently uses Fuel, but the serializer/materializer is abstracted, so STON could probably be plugged in easily
As a textual format, STON has the advantage that you can read/edit the file (especially if you pretty print it). Wrapping a ZnBuffered[Read|Write]Stream around your text/character stream before handing it to STON improves speed.
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Tim, As I showed in my example Grafoscopio notebook in this thread, STON is a pretty well suited format for complex documents or simple objects and DVCS (Fossil / Git) friendly. Besides, the pretty print option make it really readable, even outside Pharo (see the referred notebook for example). Using Fossil repositories + STON as a future proof format for reproducible, agile data storytelling is one of those combinations that, could no be exactly what you're looking for, but maybe useful to have on the radar and put in your preliminary explorations. I'm pretty happy with it. Cheers, Offray On 9/10/20 6:07 a. m., Sven Van Caekenberghe wrote:
On 9 Oct 2020, at 12:13, Sean P. DeNigris <sean@clipperadams.com> wrote:
Tim Mackinnon wrote
Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure). It currently uses Fuel, but the serializer/materializer is abstracted, so STON could probably be plugged in easily As a textual format, STON has the advantage that you can read/edit the file (especially if you pretty print it).
Wrapping a ZnBuffered[Read|Write]Stream around your text/character stream before handing it to STON improves speed.
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Tim's original question is something that I ask myself from time to time. Having some "obvious" options for simple prototyping of ideas on Pharo would be a good tool for infiltrating Pharo in our workplaces and starting the slow adoption process. The Django web framework is a good example. The framework as a whole is quite complete, mature and professional, but it's adoption usually started with the auto-generated "admin" interface, an ill-suited tool for anything more serious. On Fri, Oct 9, 2020 at 12:53 PM Offray Vladimir Luna Cárdenas < offray.luna@mutabit.com> wrote:
Tim,
As I showed in my example Grafoscopio notebook in this thread, STON is a pretty well suited format for complex documents or simple objects and DVCS (Fossil / Git) friendly. Besides, the pretty print option make it really readable, even outside Pharo (see the referred notebook for example). Using Fossil repositories + STON as a future proof format for reproducible, agile data storytelling is one of those combinations that, could no be exactly what you're looking for, but maybe useful to have on the radar and put in your preliminary explorations. I'm pretty happy with it.
Cheers,
Offray
On 9/10/20 6:07 a. m., Sven Van Caekenberghe wrote:
On 9 Oct 2020, at 12:13, Sean P. DeNigris <sean@clipperadams.com>
wrote:
Tim Mackinnon wrote
Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure). It currently uses Fuel, but the serializer/materializer is abstracted,
so
STON could probably be plugged in easily As a textual format, STON has the advantage that you can read/edit the file (especially if you pretty print it).
Wrapping a ZnBuffered[Read|Write]Stream around your text/character stream before handing it to STON improves speed.
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- Eduardo de Oliveira Padoan â®)
Tim saving in STON is also nice :). S
On 6 Oct 2020, at 22:41, Tim Mackinnon <tim@testit.works> wrote:
Gosh - this is proving much more interesting than I had imagined, and Iâm getting lots of useful input, so I double appreciate the time and thoughts from everyone.
I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case, however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do that⦠and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust⦠however, move too soon and you get bogged down with the details and you lose site of an MVP.
It has been interesting hearing peoples thoughts on all of this⦠the turning tide on ORMâs, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy). I suppose this is where the Rails scaffolding was/is? such a jumpstart, you can get a full thing going quite easily, and deploying seems relatively easy too⦠for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work. It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus onâ¦
Anyway, that login screen⦠oh crap I have to write one of thoseâ¦
Tim
On 6 Oct 2020, at 20:56, jtuchel@objektfabrik.de wrote:
Sean,
thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects. What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.
I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)
In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.
The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.
Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.
But, hey, I said all of that before.
So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.
I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?
I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.
But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.
Joachim
Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:
jtuchel wrote
Sigh. Forget about the idea that it will be easy to switch your persistence later....I am not commenting on SimplePersistence here, I don't even know what it does or doesn't. Joachim, Thanks for this interesting perspective. I've never had the (mis?!)fortune of a project growing enough to force me to make those tough choices! For SimplePersistence I will say that I view it as a way to *delay* making *any* choices until you are forced to. It's really just a layer of sugar on top of Fuel (it used to use the old school Squeak equivalent serialization mechanism - I forget the name and that might still work). You tell it what classes to serialize. Implement two methods for each class that get and set the data, and then it saves the whole thing as one object graph.
Tim, If you use SimplePersistence, please keep me posted about your experience. I'm happy to help.
NB I have maintained and extended the library, but it is the work of Ramon Leon
----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
Hi Tim, I think that the persistence you choose will depend heavily on a few things: - The complexity of the objects graphs in your domain models - This architecture of your system - How these objects/data is acceded (Atomically/In clusters) In any case, for a lightweight solution you go the Fuel path, or something using SQLite. I have a GLORP based application (small, ~40 persisted classes), SQLite is so lightweight that I run my SUnit tests creating the whole database _on disk_ from scratch on setUp and destroying it on tearDown), and only run it against the production backend (PostgreSQL) before deploying, but even this is happening less frequently since I'm confident that things will work the same on both. OTOH I'm finding that ORMs are burden, and unless you need to work with an existing schema or you really want to use an RDBMS, you might be better off avoiding them. I've been using ORMs for almost two decades, so that's still my default choice, but I feel winds of change here :-) Regards! -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
If you want to use docker there's https://github.com/ba-st/docker-pharo and others that I don't remembar now that provides the basics. You have some examples in https://github.com/ba-st/docker-pharo/blob/master/docs/Examples.md . We're deploying our apps as docker containers so ask any question if you have doubts. On Tue, Oct 6, 2020 at 9:48 AM Tim Mackinnon <tim@testit.works> wrote:
Some great answers everyone - really appreciate it.
I think that all things considered, SimplePersistence seems like a very easy place to start, and then I can "upgrade" when my super awesome idea (not), exceeds its bounds.
I wonder if these answers should go somewhere in a tips section somewhere, and get updated yearly.
@jonathan I had forgotten about those server tools - I think the last time I did something it predated some earlier tips the Sven had given, so will check it out. Back then, I did have a GitLab build pipeline autodeploying for me, so I'm hoping I can re-incarnate all of that so I can get my little app running with no fuss... lets see.
While I have a love/hate with Docker - I did wonder if there was something for Pharo that just let me build my image into a container and then it would just work with little system knowledge needed (as I keep forgetting all the voodoo between times when I need it). Maybe there is, or maybe it might come one day soon...
Tim
On Tue, 6 Oct 2020, at 9:40 AM, Jonathan van Alteren wrote:
Hi Tim,
I've been running Seaside applications on Hetzner cloud servers for more than a year now, with great pleasure and success: https://www.hetzner.com/cloud I guess their servers are similar to Digital Ocean, although I haven't followed the development of their products and solutions for quite a while. Setting up a new server at Hetzner is a breeze, and you can start already for as low as â¬2,49 per month!
We're using Voyage on MongoDB for persistence. After learning some hard lessons (and I'm sure there are more to come ;-)), I really enjoy the unobtrusiveness of it. Most of the time, it doesn't require much attention and allows me to add persistence to real OO designs quickly and easily. I find it a welcome change from the relational database work I used to (need to) do, back when I was still doing Java. The 'everything an object' principle of Pharo/Smalltalk really makes it shine.
I can't help you with a list of tradeoffs though. If you come across a set of arguments, I'd be happy to give feedback.
By the way, I forked Sven's pharo-server-tools project (here: https://github.com/objectguild/pharo-server-tools) and have a routine going that suits me well enough. Still lots of room for improvement, but it's OK for my current needs.
Future plans are to use the Hetzner API to provision a new server and use something like Chef or Ansible to install/configure it automatically to be ready to deploy a Seaside application. I'd like to integrate this into a full service CI/CD pipeline in the future, to be able to do automated production deployments without service interruption if possible. For this scenario, I would really also like to switch to using GemStone for persistence.
Hope this helps! Let me know what you decide and I might be able to help with some technical stuff.
Kind regards,
Jonathan van Alteren
Founding Member | Object Guild B.V. *Sustainable Software for Purpose-Driven Organizations*
jvalteren@objectguild.com On 6 Oct 2020, 00:23 +0200, Tim Mackinnon <tim@testit.works>, wrote:
Hi everyone - Iâm wondering what is the recommended way to save some simple user data for a Pharo application I would like to run on the cloud (probably initially digital ocean, but could be AWS if it came to it).
Initially I thought I might try and run my little app in Digital ocean (I followed someoneâs steps a few years ago, and had a simple seaside app running quite well) - so I was thinking of starting there.
I know there is Svenâs P3 - but Iâm not sure Iâm ready to run and maintain a SQL database for a simple application, but could be persuaded it its simple to setup with little maintenance. Would mongo be a suggestion - is that easy to setup and run? (And is that Voyage?).
Possibly I could even use image persistence, and fuel out a Dictionary from time to time - but I think that might be a little bit too belt and braces for me.
Is there something that gives a little table of tradeoffs with some simple ways to get started?
Tim
participants (13)
-
Eduardo de Oliveira Padoan -
Esteban A. Maringolo -
Esteban Maringolo
-
Gabriel Cotelli -
Jonathan van Alteren -
jtuchel＠objektfabrik.de -
Offray Vladimir Luna Cárdenas -
Pierce Ng -
Russ Whaley -
Sean P. DeNigris -
Stéphane Ducasse -
Sven Van Caekenberghe -
Tim Mackinnon