Adam, SandstoneDb may be of interest to you: http://www.smalltalkhub.com/#!/~gnaritas/SandstoneDb "SandstoneDb is a lightweight Prevayler style embedded object database with an ActiveRecord API that doesn't require a command pattern and works for small apps that a single Squeak image can handle. The idea is to make a Squeak image durable and crash proof and suitable for use in small office applications. Data is kept in ram for speed and on disk for safety. All data is reloaded from disk on image startup. Since we're dealing with live objects in memory, concurrency is handled via optional record level critical sections rather than optimistic locking and commit failures. It's up to the developer to use critical sections at the appropriate points by using the critical method on the record. Saves are atomic for an ActiveRecord and all its non ActiveRecord children, for example, an order and its items. There is no atomic save across multiple ActiveRecords. A record is a cluster of objects that are stored in a single file together. After installing SandstoneDb make sure to restart your image before attempting to use it or running the tests." Fuel may also have something in stock: https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la... I also use Mapless http://sebastianconcept.github.io/Mapless/ HTH Phil On Sun, Sep 27, 2015 at 6:47 PM, Adam <adam@234.cz> wrote:
Hello,
I was wondering if it is possible for Pharo to load lots of data and query them without using external DB, just using object memory. Well, someting like GemStone/S, but not so sophisticated.
I did some research so here is details what I found. I would appreciate if someone can tell me if any points of this agenda can be allready done or need to be programmed, or if this whole idea is waste of time and it is better to use external DB.
1) Object memory limit: Current object memory of Pharo is limited to 4GB. Free version of GemStone/S is also limited to 4GB, so I would rather stick with Pharo until I reach this level. And in the future this limit grow due to 64bit version of VM. 2) Saving to disk: Is it possible to save object or group of objects to disk and ensure consistency? With some transaction mechanism and transaction log? 3) RAM limit: Is it possible to delete objects from memory, move them to the disk and not break its connections to other objects? For example by redirecting object internal ID to some kind of hollow object, that will renew receiver of message from disk if necessary... This approach can also possibly split object memory into more "remote" sources. 4) Indexing: What optimalization for searching current collections in Pharo do? For example: how to store instances of City if I would like to search through it by name and also by its postal code? Is there some library for creating fulltext search (maybe Spotter uses that - I did not study it yet).
Have a nice day, and thanks for any answer.
Adam