Am 29.08.2024 um 14:46 schrieb Esteban Maringolo <emaringolo@gmail.com>:

Hi Norbert,

Excellent news.

What's the problem that's stopping it from working in Windows? I couldn't find a Github issue for that.

Because you assume windows support is something we want. I personally do not care about windows because I never used it and I think it is a really bad operating system that kills lots of things. The technical view is: - soil uses file range locks to lock portions of a file (think of row-level locking in relational databases) - soil uses a system call fsync() on POSIX systems that flushes bytes from OS to disk. This is needed to meet transaction requirements The architecture view is: - I've seen the implementation of file opening with additional flags in OmniBase and how the design got really bad in order to support all of this. This might be non-issue today but needs to prove me wrong first - windows might have comparable support for the requirements above but I'm not easily convinced to ruin the design in order to support it. Might also not be true... The maintainer view is: - I won't add windows support ever and Pablo was the only one offering his help so far (why should we implement something nobody needs?) - even if there would be someone doing it initially I still need to be convinced that there is always someone maintaining it because I won't. And I surely do not want to block progress of soil just because things cannot move. They will be removed then. So there are basically two options: - windows supports the requirements (current and future ones) comparable to unix systems so that the design does not need to be changed a lot in order to support windows. Then adding support is feasible - in the meantime I made the file locking and fsync configurable. So by switching them off it might just work out of the box without any change. On a desktop machine it should(!) work reliably. Locking and range locks are mostly needed for concurrency. But noone would be insane enough to use windows as a server or in a distributed environment, haha, no that would be really foolish...hahaha But we also try to make soil the de-facto database for pharo so in the end we have no choice then to add windows support. Hope this helps, Norbert

Regards!

Esteban A. Maringolo

On Wed, Aug 28, 2024 at 9:00 AM Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:

...

We are proud to announce the availability of Soil version 2.

This release is dated back to July because we made some bigger changes after that and all the bug fixes before that deserve a release. You can access the new release via

<Soil.png> ApptiveGrid/Soil at v2 github.com <https://github.com/ApptiveGrid/Soil/tree/v2>ApptiveGrid/Soil at v2 <https://github.com/ApptiveGrid/Soil/tree/v2> github.com <https://github.com/ApptiveGrid/Soil/tree/v2>

The changelog is available at

Soil/docs/changelog.md at v2 · ApptiveGrid/Soil github.com <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md>Soil/docs/changelog.md at v2 · ApptiveGrid/Soil <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> github.com <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> The new features need to mature a bit but we expect to have another release in a month or two.

Enjoy,

Norbert + Marcus <Soil.png>

Am 03.09.2024 um 03:04 schrieb sean@clipperadams.com:

Exciting news!

Two questions:

How is the stability? I am finally outgrowing SimplePersistence, but some of my systems are too critical to risk beta testing…

I think the only way finding out is to try. In ApptiveGrid we have 5000 instances and it is in production over one year. I wouldn't say there weren't problems in that time. But for me it is quite some time since I've found a conceptual problem that caused errors. This doesn't mean that it works for you and you won't bump into something the first time you use it. Stability is use case dependent.

I remember hearing (or reading?) that Fuel wasn’t used underneath because it wasn’t flexible or fast enough. Is that correct? If so, is there anywhere I can read about this analysis? Or would you touch on it here? It seemed like a lot of community resources were invested in Fuel so it’s disappointing that a rewrite felt necessary and I’m wondering if any of the lessons learned in Soil could help Fuel.

The new implementation is only questionable from a bird's eye view. Meaning when you put them under the coarse grained category serializers. That is not appropriate for many things because the way you use it is important. There is no written detailled analysis but let me give you a few points: Fuel is a whole graph serializer. Meaning you give it one object and then it produces an efficient serialized blob out of it. There are not many interception points. Soil supports graph partitioning, meaning that your graph will be sliced into smaller parts in order to have efficient control over the amount of data needs to be read/written. Adding that to fuel was hard and in the end did not give enough control to tweak Fuel has its own way of managing class names which needs pre-processing which is not suited if you want to use streaming objects to disk Fuel only supports management of class names because it assumes them to be there in the image. Soil stores class descriptions in the database. Adding that to Fuel wouldn't be rather difficult. Soil has class versions because as it is a database you can store data in one shape and then change the code. So we need proper migration support which in Fuel is only doable with a central registry and an external version managment Soil separates content and structure completely, meaning sub-graphs are stored together but everything referencing and addressing is done on the database level, meaning you can deal with the structure without having the code at hand etc. Soil uses byte type codes for special types to map it efficiently as type-length-value format onto disk. Fuel has no support for how you store objects on disk. ... Hope this gives a bit more insight than last time. I did not "just wanted to do my own", I considered using Fuel but it didn't work. So as usual there is no one-fits-all solution. And this is a good example. Norbert

Anyway thanks for creating and sharing this :)

Sean