February 2012 - Pharo-users

Nearly limitless Image: revisited.

by ted f.a. van gaalen

"I first used image persistence but the image grow too large.." thanks for reminding, Norbert. Hi all, perhaps you'll find me an idiot, I don't care. (i know that I am: (class SelfReflection :o) ) Nevertheless, some brainstorming here. I've brought this up before, but it seems nobody is interested in this topic.. why? it's about image size. Currently, Pharo 1.3-13315 at my 4 Gigabyte Debian amd64 Linux machine reports memory 75,392,160 bytes old 61,370,144 bytes (81.4%) young 2,169,692 bytes (2.9000000000000004%) used 63,539,836 bytes (84.30000000000001%) free 11,852,324 bytes (15.700000000000001%) (nothing peculiar added, just the Seaside package and a few classes of my own. ) So, if i understand this well I have only 11,8 megabytes to play with? To begin with: why is only 75 megabyte available in Pharo (or Squeak as well) when there is 4 gigabyte physical memory available on my machine?? Anyway, IMHO this very limited storage violates the Smalltalk-all-in-one image ideal, because with this limited storage one cannot avoid the use of external data stores like databases (of course, exception: there always will be import an export of objects/data) External data storage is complex, unreliable and should be something of the past. I'd suggest: don't waste time with this. rather improve the Smalltalk internal data handling. I wish to have a very large image so e.q. I can have an entire company's administration (client and accounting data etc.) alive in it. I want to use Collections and descendants, not external databases. So, everything within the image. The beautiful (although opinions may vary) principle of a Smalltalk environment is Image Persistence, isn't it? IMHO this requires: - virtually unlimited image memory size. -no external databases like e.g. Mongo or DB2 whatever. -no persistent external storage at all. -You should see the (currently still) faster database performance as a challenge to improve the Collection and other related data handling classes -You would have to completely restructure the architecture. or just extend the address width: 64 bits? what about an image of virtually unlimited size > gigabytes so (for now, if it goes beyond the physical memory size (say 32 GB?) one could use a VM that works with virtual storage based VM Virtual storage was first in use successfully with IBM mainframes in the 1970s (and still is) ) on the other hand, I estimate holographic? physical memory going in Terabytes is about to be available in 5-10 years from now. blindingly fast, compared with today's standards. So, one has about 5 years to come up with such a system :o) if one does, Smalltalk is ahead of everything then.. Don't laugh. One didn't even dream of 4 gigabytes of memory during the Commodore 64 era. 1982-1994. In short: what would have to be changed to enable this? Some thoughts and speculations? Anyone for tennis? Thanks Ted -- Ted F.A. van Gaalen Danketsweiler 608 D-88263 Horgenzell Germany T: +49 750 491 48 38 M: +49 151 587 862 47

Feb. 28, 2012

Re: [Pharo-users] ProfStef - syntax highlighting

by Markus Schlager

On Sun, 26 Feb 2012, StÃ©phane Ducasse wrote: > so did you load it in the latest 1.4 and did it work now? Looks good, thanks! Markus

Feb. 28, 2012

Re: [Pharo-users] pharo & arduino

by StÃ©phane Ducasse

have a look at Pharo By Example. You can drop a mcz file on the image. Open a monticello browser, ad the folder as repository (+ repository) and select the package you want to load. STef > Hello, > > i have installed pharo in ubuntu 11.10 > > > i have two version pharocore-1.3 and pharo1.3 > > i test the second > > i went to http://www.squeaksource.com/Arduino.html > > > i'm not a developper i'm simple beginner's user > > > i have installed arduino with avr and edi > > > i have a lot of .mcz file. > > > How does i load a class > > > i have run in workspace the two code > > one for arduino and other for metacelo > > but it does'nt happen > > > i read the beginner's page of pharo par l'exemple. > > > thanks a lot for your time > > > > > > >

Feb. 28, 2012

Re: [Pharo-users] [Pharo-project] Who is using Fuel?

by Norbert Hartl

Am 28.02.2012 um 19:15 schrieb Yanni Chiu: > On 28/02/12 11:03 AM, Norbert Hartl wrote: >> >> I first used image persistence but the image grow to large. Then I added >> fuel as journal to be written before image save in order to be able to >> recover from an emergency. >> Now we redo the persistence part. An account object graph is dissected >> into configuration part and payload part. The payload is written >> directly to mongo db in a Json format. The configuration part will be >> written as fuel blob to mongo. Planned is that at startup a configured >> amount of account fuel blobs are loaded and started. Those >> configurations have their own proces ses that write back payload to >> mongo. Etc. > > That's similar to what I do, except no image save and no mongo db. > As soon as the fuel blobs are in mongo (or on disk) no image save is needed then and I will start fresh everytime which is more reliable. Mongo is quite nice because I have a few hundred megabytes of data. And via cron it runs a map/reduce task to create the statistics about the data that has been fetched. The statistical data is displayed in charts on the web site again. Norbert > Here's how it goes: > - on restart the image reconstructs a Pier kernel saved via Fuel to a file > - included in the Pier kernel is what you're calling the "configuration part" - there are custom (non-std Pier) components here. > - the components in the reconstructed image can read/write a "payload" part, which is serialized to files using Fuel and SandstoneDb. > - additionally, a "sub-tree" of the Pier kernel can be exported to a file using Fuel serialization, and later imported using Fuel deserialization. > > No URL available at the moment. > > Aside - before Fuel/SandstoneDb, I was using Glorp/PostgreSQL. I've gained a lot of simplicity, but lost sequence numbers and object caching (and maybe more things, that have not been a problem so far). I've written a seqno replacement, but the lack of object caching is not a problem at the moment (i.e. small data sets, so far). > >

Feb. 28, 2012

Re: [Pharo-users] Segmentation fault

by Bernat Romagosa

Thanks Igor, I'll get the latest one and report if the segfault still happens. Cheers, 2012/2/28 Igor Stasenko <siguctua(a)gmail.com> > On 28 February 2012 16:50, Bernat Romagosa > <tibabenfortlapalanca(a)gmail.com> wrote: > > Hi list, > > > > We're getting this segfault from time to > > time: http://hastebin.com/kuvefebuhe.sm > > > > The crashes appear to happen randomly, we haven't been able to find out > what > > kind of behavior triggers them. > > > > Here's the machine specs: > > > > Smalltalk vmVersion 'Croquet Closure Cog VM [CoInterpreter > > VMMaker-oscog-IgorStasenko.123]' > > Smalltalk version 'Pharo1.3' > > Smalltalk osVersion 'linux-gnu' > > > > > > The Pharo image is a standard One-Click, and the OS is a Debian testing. > > > > Hope you can help us see what's going on... > > We're going to update VMs soon, i am currently working on making > windows builds, and then we'll do automated 1-click. > > You can download more recent vm from here: > > https://ci.lille.inria.fr/pharo/view/Cog/job/Cog-Unix/ > which is built automatically, > as well as use one, build by Eliot: > http://www.mirandabanda.org/files/Cog/VM/ > > -- > Best regards, > Igor Stasenko. > > -- Bernat Romagosa.

Feb. 28, 2012

Re: [Pharo-users] [Pharo-project] Who is using Fuel?

by Norbert Hartl

Am 28.02.2012 um 17:23 schrieb Mariano Martinez Peck: > > > On Tue, Feb 28, 2012 at 5:03 PM, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck: > >> Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer. >> In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel >> >> Let me know if there is something I should add. >> > We are using it in > > http://www.2denker.de/product/apitester > > I first used image persistence but the image grow to large. Then I added fuel as journal to be written before image save in order to be able to recover from an emergency. > > I didn't undersantd this one. What is what you are serializing that you want to be able to recover in case of an emergency? All of my data. An account can log in and configure the service. The service collects data. All of these modifications should be persisted. I experience image corruption with big images. Every week or so the vm just didn't start the images anymore because it stated "premature end of image file". So I write down all my objects via fuel to disk before I save the image. In case of an emergency (image corruption) I take a fresh image and load back the saved fuel graph. And within 2 minutes I'm once read to go. Now the big parts of the data are in mongo so I don't expect it to happen real soon again. > > Now we redo the persistence part. An account object graph is dissected into configuration part and payload part. The payload is written directly to mongo db in a Json format. The configuration part will be written as fuel blob to mongo. Planned is that at startup a configured amount of account fuel blobs are loaded and started. Those configurations have their own processes that write back payload to mongo. Etc. > > > Just in case you want to know :) > > > Interested indeed :) I added it under "Custom applications" > thanks, Norbert

Feb. 28, 2012

Re: [Pharo-users] [Pharo-project] Who is using Fuel?

by Mariano Martinez Peck

On Tue, Feb 28, 2012 at 5:03 PM, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck: > > Of course, not the fuel you use to go to work every day ;) I am talking > about Fuel serializer. > In particular, I wonder if our page deserves an update: > rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel > > Let me know if there is something I should add. > > We are using it in > > http://www.2denker.de/product/apitester > > I first used image persistence but the image grow to large. Then I added > fuel as journal to be written before image save in order to be able to > recover from an emergency. > I didn't undersantd this one. What is what you are serializing that you want to be able to recover in case of an emergency? > Now we redo the persistence part. An account object graph is dissected > into configuration part and payload part. The payload is written directly > to mongo db in a Json format. The configuration part will be written as > fuel blob to mongo. Planned is that at startup a configured amount of > account fuel blobs are loaded and started. Those configurations have their > own processes that write back payload to mongo. Etc. > > > Just in case you want to know :) > > Interested indeed :) I added it under "Custom applications" Thanks! -- Mariano http://marianopeck.wordpress.com

Feb. 28, 2012

Re: [Pharo-users] Segmentation fault

by Igor Stasenko

On 28 February 2012 16:50, Bernat Romagosa <tibabenfortlapalanca(a)gmail.com> wrote: > Hi list, > > We're getting this segfault from time to > time:Â http://hastebin.com/kuvefebuhe.sm > > The crashes appear to happen randomly, we haven't been able to find out what > kind of behavior triggers them. > > Here's the machine specs: > > Smalltalk vmVersion 'Croquet Closure Cog VM [CoInterpreter > VMMaker-oscog-IgorStasenko.123]' > Smalltalk version 'Pharo1.3' > Smalltalk osVersion 'linux-gnu' > > > The Pharo image is a standard One-Click, and the OS is a Debian testing. > > Hope you can help us see what's going on... We're going to update VMs soon, i am currently working on making windows builds, and then we'll do automated 1-click. You can download more recent vm from here: https://ci.lille.inria.fr/pharo/view/Cog/job/Cog-Unix/ which is built automatically, as well as use one, build by Eliot: http://www.mirandabanda.org/files/Cog/VM/ -- Best regards, Igor Stasenko.

Feb. 28, 2012

Re: [Pharo-users] Who is using Fuel?

by Norbert Hartl

Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck: > Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer. > In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel > > Let me know if there is something I should add. > We are using it in http://www.2denker.de/product/apitester I first used image persistence but the image grow to large. Then I added fuel as journal to be written before image save in order to be able to recover from an emergency. Now we redo the persistence part. An account object graph is dissected into configuration part and payload part. The payload is written directly to mongo db in a Json format. The configuration part will be written as fuel blob to mongo. Planned is that at startup a configured amount of account fuel blobs are loaded and started. Those configurations have their own processes that write back payload to mongo. Etc. Just in case you want to know :) Norbert

Feb. 28, 2012

Segmentation fault

by Bernat Romagosa

Hi list, We're getting this segfault from time to time: http://hastebin.com/kuvefebuhe.sm The crashes appear to happen randomly, we haven't been able to find out what kind of behavior triggers them. Here's the machine specs: Smalltalk vmVersion 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog-IgorStasenko.123]' Smalltalk version 'Pharo1.3' Smalltalk osVersion 'linux-gnu' The Pharo image is a standard One-Click, and the OS is a Debian testing. Hope you can help us see what's going on... Thanks! -- Bernat Romagosa.

Feb. 28, 2012