October 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] binary serialization

by Robert Withers

On 10/14/2015 11:37 AM, Mariano Martinez Peck wrote: > > > On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers > <robert.w.withers(a)gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > > > On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote: > > Robert, > > As far as I can remember, the problem of substitutions at > materialziation time was because... as you may have read, Fuel first > serializes "objects" and then the references. At materialization, it > first creates the objects and then, on a second step, sets the > references betwen objects. So the problem was WHERE to place the > hook, > at which point in time. If we do it just after objects were created, > then the substitution code would NOT have access to any of the > instVars > of that object. If we do it AFTER objects creation and after objects > references, then there is no easy way to "substitute" which doesn't > involve #become: (because the graph is already constructed) And > you know > that #become: is still terrible slow because it scans full > memory. That > will change in Spur. > > > The trick I learned from Gemstone is to use forwarding proxies, > > > Well, Spur will/does something similar called lazy become. Basically it > lets a forwarding pointer object and then takes advantage of next GC > pass or whatever in order to resolve such proxy in lazy manner. > > looking up into the FLobjectId dictionary (decoder>>objects?) when > stitching the references. When you copy the proxies on substitution, > it stitches normally at reference time. > > > Yes, that could work. The problem is if you need instVars of the object > you want to substitute. Imagine you have a class called Client and > instVar 'age'. And you want to substitute Client with instances of > OldClient if 'age' is > 10. At that point in time, the reference to the > instVar 10 has not yet been filled. Yet, you need to replace the object > at graph construction time. > Ok, so a depth-first, node post-apply would allow ivars to substitute before they are accessed when the container may substitute, lazy as you point out. thanks so much ^^ Robert > > > As for Marea and Ghost, > > Ghost proxies paper: https://hal.inria.fr/hal-01081236/document > Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost > > Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf > Current repo: http://ss3.gemstone.com/ss/Marea.html > > And finally, my PhD thesis: > https://tel.archives-ouvertes.fr/tel-00764991/document > > > Nicely done, sir. I'll check them out, thankyou. > > > Thanks, feel free to ask questions. > > In Marea I needed custom clusters for my proxies because the > serializer > itself sends messages to the objects being serialized. My > proxies would > bring back graphs from a secondary memory. So if I was serializing a > graph that had proxies already, I didn't want that. So I hooked my > custom cluster for proxies that send only a few special messages > to the > proxy that these understand and answer rather than intercept those > messages. > > As for how to extend Fuel for this, I recommend to check the code of > Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. > I have a Marea one click here: > https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0 > > Cheers, > > > thanks so much ^^ > Robert > > > > On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers > <robert.w.withers(a)gmail.com <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> wrote: > > Good morning, Max. Thank you for the example. I got a little > confused, between migrations and substitutions. My issue > with no-arg > blocks, I believe, is the inability to access my scope > object to > maintain the object tables. > > I'm attempting to write my own Materializer, Decoder and > Materialization. At the moment, I'm just going to walk the > graph, > testing and do #becomes:. See how well that works when I > get a test. > > It's really good to know about that other list. > > thanks so much ^^ > Robert > > On 10/14/2015 04:15 AM, Max Leske wrote: > > BTW, there is a dedicated Fuel mailing list: > pharo-fuel(a)lists.gforge.inria.fr > <mailto:pharo-fuel@lists.gforge.inria.fr> > <mailto:pharo-fuel@lists.gforge.inria.fr > <mailto:pharo-fuel@lists.gforge.inria.fr>> > <mailto:pharo-fuel@lists.gforge.inria.fr > <mailto:pharo-fuel@lists.gforge.inria.fr> > <mailto:pharo-fuel@lists.gforge.inria.fr > <mailto:pharo-fuel@lists.gforge.inria.fr>>> > > Max > > > On 14 Oct 2015, at 09:45, Max Leske > <maxleske(a)gmail.com <mailto:maxleske@gmail.com> > <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>> > <mailto:maxleske@gmail.com > <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com > <mailto:maxleske@gmail.com>>>> wrote: > > > On 14 Oct 2015, at 04:39, Robert Withers > <robert.w.withers(a)gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>> wrote: > > > On 10/13/2015 09:43 PM, Mariano Martinez Peck > wrote: > > > > On Tue, Oct 13, 2015 at 10:33 PM, Robert > Withers > <robert.w.withers(a)gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>> wrote: > > Hi Mariano, > > This presents me with a big challenge, > then. I > read the docs and > explored the code and the only other > aspect not > mentioned, beyond > instance creation (#fuelNew #fuelNew:) and > postMaterialization > (#fuelAfterMaterialization), is migrations. > However, migration only > allows for instanceVar mappings, no code > blocks. > > > What do you mean that migrations only > allows instVar > mappings , and no > code blocks? I mean, what do you mean by > code blocks? > > > Sounds to me like this (see FuelOutStackDebuAction): > > serializeTestFailureContext: aContext toFileNamed: > aFilename > | serializer | > > serializer := FLSerializer newDefault. > self encodeDebugInformationOn: serializer. > serializer addPostMaterializationAction: [ > :materialization | > Smalltalk tools debugger > openOn: Processor activeProcess > context: materialization root > label: 'External stack' > contents: nil > fullView: false ]. > > serializer > " use the sender context, generally the current > context is not > interesting" > serialize: aContext > toFileNamed: aFilename > > This stores a block in the serialization which is > evaluated > after > materialization. The only requirement is that itâs > a clean > block (no > closure!). > > We also support class renames. This is here: > http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_… > > Which kind of migration example you have in > mind > that would not be > supported? An example would help. > > > Well, my pics will demonstrate. I am interested > in doing > more than > mappping ivars or a class rename. I want to do > a total > substitution, > then a further substitution on the receiving, > import side: > > Vat1: anObject (Class A) ---> On wire: desc > (Descriptor) > ---> Vat2: > aProxy (Class FarERef) > > A desc is substituted for a PassByProxy object, > then a > FarERef is > substituted for the desc. > > > #fuelAccept: is a serialization side method. > > If Fuel supports substitution on > serialization, I > don't understand > why no substitution support on > materialization. > > > There was a reason, which I cannot remember > completely. Maybe Martin or > Max can remember. > > > It seems your focus was pickling to disk then > back. My > focus is > distributed proxy graphs, which has different > use cases. > > > > I am definitely going to use the > world-class Fuel > binary > serialization system. However, I find myself > needing to extend Fuel > to support substitution on materialization. > Perhaps the solution is > a custom decoder. > > > I have made custom clusters for example for > my Ghost > proxies of Marea > system. It was a perfect example of how I could > extent Fuel besides the > common hooks. Fuel provides many places for > extending , like clusters, > analyzer, etc > > > Right on, exactly! Could you tell me more about > your > Ghost proxies > and Marea, please? As well, could you mention > how you > select a custom > cluster on the serialization side? > > > thanks so much ^^ > Robert > > > > No, a bit more. It looks like I need a new > FLSubstitutePointerObjectCluster, write > them on > serialization with > the substitute, then do unsubstitution on > materialization, since the > cluster controls materialization and not > the decoder. > > Does this approach seem sound to you, > from a you > know architecture > and design approach? > > > There was an issue. Hope other can > remember. If not, > I will try to > explan what I remember tomorrow. > > > thanks so much ^^ > Robert > > On 10/13/2015 04:49 PM, Mariano Martinez > Peck wrote: > > No, unfortunately, as far as I can > remember, > we do not have > that. There > are another hooks you may use but > only in > certain scenarios > (#fuelNew, > #fuelAfterMaterialization, global sends, > etc). But everything is > listed > in > http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Customizing… > so if you didn't find anything of > help in > there there are > chances > there isn't anything. > > Cheers, > > On Tue, Oct 13, 2015 at 5:30 PM, > Robert Withers > <robert.w.withers(a)gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>>> wrote: > > Yes, I meant dynamic > substitution on > materialization, to > use the > correct terminology. > > thanks, > Robert > > > On 10/13/2015 11:40 AM, Max > Leske wrote: > > > On 13 Oct 2015, at > 17:16, Robert > Withers > > <robert.w.withers(a)gmail.com <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>>> wrote: > > Every extra source > helps, thank > you. I see how to do > non-stream substitutions on > materializations, but the > documentation did not > indicate a > way to do non-stream > substitutions on > serialization. > Is it possible? > > > I donât understand what you > mean by > ânon-streamâ. Could > you give > an example? > > > thanks, > Robert > > On 10/13/2015 09:00 AM, > Mariano > Martinez Peck wrote: > > Hi Robert, > > As for the > documentation, > you have LOTS of > tests, you > have the chapter > Torsten pasted, you > have > this documentation: > http://rmod.inria.fr/web/software/Fuel > > But also, as for > internals, > there is a journal > paper we > wrote: > http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf > > Let us know how it > goes, > > > On Tue, Oct 13, > 2015 at 6:00 > AM, Torsten Bergmann > <astares(a)gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>> <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>>> > <mailto:astares@gmx.de > <mailto:astares@gmx.de> <mailto:astares@gmx.de > <mailto:astares@gmx.de>>> > <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>> <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>>>> > > <mailto:astares@gmx.de <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>> <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>>> > <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>> <mailto:astares@gmx.de > <mailto:astares@gmx.de> > <mailto:astares@gmx.de > <mailto:astares@gmx.de>>>>>> wrote: > > Hi Robert, > > Also checkout the > chapter on Fuel in Pharo > Enterprise book: > > https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/l… > > Bye > Torsten > > Gesendet: Dienstag, 13. Oktober 2015 um > > 09:44 Uhr > > Von: "Robert Withers" > > > <robert.w.withers(a)gmail.com <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>>>> > > An: pharo-dev(a)lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>>> > > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>>> > > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>>>> > > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>>> > > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org> > <mailto:pharo-dev@lists.pharo.org > <mailto:pharo-dev@lists.pharo.org>>>>> > > Betreff: Re: [Pharo-dev] binary > > serialization > > > Yes, I have to do object substitutions. > > Thanks > for the link! > > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > On 13 Oct 2015, at 09:40, > Robert Withers > > > <robert.w.withers(a)gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>>>> wrote: > > > Sven and Torsten, that's a binary > > serialization > library! It > will take time > to learn > it and how to use > mappers. > > > What is the format; is it language > > neutral? > > > For quick serialization you donât > > need to do > anything. It works > for (almost) all > objects. Only if you > want to > exclude things or > treat some > objects in a > special way, you > will need > to do some stuff. > > > Documentation: > > http://rmod.inria.fr/web/software/Fuel. > > > > > thanks, > Robert > > On 10/13/2015 01:21 AM, Sven Van > > Caekenberghe > wrote: > > Yes, it is called FUEL and > it is a > > standard > part of the > image. See > FLSerializer > and FLMaterializer. > > > On 13 Oct 2015, at > 06:59, Robert > > Withers > > <robert.w.withers(a)gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>> > > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com> > <mailto:robert.w.withers@gmail.com > <mailto:robert.w.withers@gmail.com>>>>>> wrote: > > > Does Pharo have stream > classes to > > binary > de/serialize an > object, such > that the > protocol accepts an > object as > an argument and > converts it to > a byteArray? > > > -- > thanks, > Robert > > > > > > > > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > <Exporting Vat.jpg><Importing Vat.jpg> > > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > > > -- > Mariano > http://marianopeck.wordpress.com

Oct. 14, 2015

Re: [Pharo-dev] what is NativeBoost?

by Esteban Lorenzano

it allows you to define and execute assembly code in the image. Esteban > On 14 Oct 2015, at 18:24, Robert Withers <robert.w.withers(a)gmail.com> wrote: > > I'd think there's just as much concern with all the globals. What does ASMJIT do? a Jit? > > thanks so much ^^ > Robert > > On 10/14/2015 11:18 AM, Esteban Lorenzano wrote: >> but bewareâ¦ ASMJIT part of NB (and wellâ¦ NB it self, that means) will fade awayâ¦ >> it will remain compatible (95% of the cases), but we will unload itâ¦ there is a lot of maintainability and security concerns around it. >> >> Esteban >> >> >>> On 14 Oct 2015, at 17:13, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> >>>> On 14 Oct 2015, at 17:01, Robert Withers <robert.w.withers(a)gmail.com> wrote: >>>> >>>> I ran across it browsing and it looks pretty interesting. It's purpose remains a mystery to to me. >>> >>> NativeBoost provides an FFI. Itâs used by Athens for example to render the fonts in Pharo using Cairo (http://cairographics.org) >>> >>> Cheers, >>> Max >>> >>>> >>>> -- >>>> thanks so much ^^ >>>> Robert >>>> >>> >>> >> >> >

Oct. 14, 2015

Re: [Pharo-dev] what is NativeBoost?

by Robert Withers

I'd think there's just as much concern with all the globals. What does ASMJIT do? a Jit? thanks so much ^^ Robert On 10/14/2015 11:18 AM, Esteban Lorenzano wrote: > but bewareâ¦ ASMJIT part of NB (and wellâ¦ NB it self, that means) will fade awayâ¦ > it will remain compatible (95% of the cases), but we will unload itâ¦ there is a lot of maintainability and security concerns around it. > > Esteban > > >> On 14 Oct 2015, at 17:13, Max Leske <maxleske(a)gmail.com> wrote: >> >> >>> On 14 Oct 2015, at 17:01, Robert Withers <robert.w.withers(a)gmail.com> wrote: >>> >>> I ran across it browsing and it looks pretty interesting. It's purpose remains a mystery to to me. >> >> NativeBoost provides an FFI. Itâs used by Athens for example to render the fonts in Pharo using Cairo (http://cairographics.org) >> >> Cheers, >> Max >> >>> >>> -- >>> thanks so much ^^ >>> Robert >>> >> >> > >

Oct. 14, 2015

Re: [Pharo-dev] Random forest in Pharo

by philippe.back＠highoctane.be

Not sure you would get enough performance on Pharo per se. Xe may be better off leveraging a multicore enabled external lib. Like caret and doMC on R. Le 14 oct. 2015 17:49, "Serge Stinckwich" <serge.stinckwich(a)gmail.com> a Ã©crit : > I don't think so. > > I followup your message on SciSmalltalk mailing-list. > This is something that might interested us ;-) > > > > On Wed, Oct 14, 2015 at 4:54 PM, Damien Cassou <damien.cassou(a)inria.fr> > wrote: > > Hi, > > > > did anyone implement a Random Forest algorithm in Pharo? > > > > https://en.wikipedia.org/wiki/Random_forest > > > > -- > > Damien Cassou > > http://damiencassou.seasidehosting.st > > > > "Success is the ability to go from one failure to another without > > losing enthusiasm." --Winston Churchill > > > > > > -- > Serge Stinckwich > UCBN & UMI UMMISCO 209 (IRD/UPMC) > Every DSL ends up being Smalltalk > http://www.doesnotunderstand.org/ > >

Oct. 14, 2015

Re: [Pharo-dev] Random forest in Pharo

by Serge Stinckwich

I don't think so. I followup your message on SciSmalltalk mailing-list. This is something that might interested us ;-) On Wed, Oct 14, 2015 at 4:54 PM, Damien Cassou <damien.cassou(a)inria.fr> wrote: > Hi, > > did anyone implement a Random Forest algorithm in Pharo? > > https://en.wikipedia.org/wiki/Random_forest > > -- > Damien Cassou > http://damiencassou.seasidehosting.st > > "Success is the ability to go from one failure to another without > losing enthusiasm." --Winston Churchill > -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/

Oct. 14, 2015

Re: [Pharo-dev] Corporate sponsored popularity

by Esteban A. Maringolo

2015-10-14 12:24 GMT-03:00 Ben Coman <btc(a)openinworld.com>: > Just a passing thought... When comparing the popularity of programming > languages, maybe technological features are less important than > sponsorship by a large tech corporation. > http://insights.dice.com/2015/10/13/apples-swift-is-killing-objective-c/ Interesting. I think it is not corporate sponsorship (as it was with Java or .Net a decade ago) but it is where it is going to run the software that you write. So it's not killing it, just displacing it. Swift is popular because if you want to write modern iOS apps you don't want to learn a language that is obsolete for the platform (it is, Objective-C). The same goes for Java in Android. JavaScript got popular not because of Mozilla or ECMA promoting it, but because it can run on almost any device today. And it is displacing popular "solutions" like Rails and alike, because it is becoming the language to run single-page-apps (Meteor/Ember/Angular, etc.) and to many server side solutions like Node that runs on embedded devices as well. So I don't think it is the promotion of it putting money into marketing but instead the interest of the software developer or company to write software that can be used in many devices as possible. Mobile software (iOS/Android) has taken the end user market share, followed by web apps. Native desktop apps are a niche these days, it's not going to disappear, but it will continue to shrink as the other platforms continue to add features. > Anyhow I don't mind that the masses haven't discovered Pharo. I enjoy it. I don't think that secrecy is something to be proud of, I also enjoy Pharo, but abundance of libraries and frameworks makes you more productive, and to have that you need a bigger community. Whether Pharo is ready to receive a HUGE adoption today, both as software, ecosystem (scm, module distribution, services, etc.) and also social community is a different story. Regards! Esteban A. Maringolo

Oct. 14, 2015

Re: [Pharo-dev] binary serialization

by Mariano Martinez Peck

On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers <robert.w.withers(a)gmail.com > wrote: > > On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote: > >> Robert, >> >> As far as I can remember, the problem of substitutions at >> materialziation time was because... as you may have read, Fuel first >> serializes "objects" and then the references. At materialization, it >> first creates the objects and then, on a second step, sets the >> references betwen objects. So the problem was WHERE to place the hook, >> at which point in time. If we do it just after objects were created, >> then the substitution code would NOT have access to any of the instVars >> of that object. If we do it AFTER objects creation and after objects >> references, then there is no easy way to "substitute" which doesn't >> involve #become: (because the graph is already constructed) And you know >> that #become: is still terrible slow because it scans full memory. That >> will change in Spur. >> > > The trick I learned from Gemstone is to use forwarding proxies, Well, Spur will/does something similar called lazy become. Basically it lets a forwarding pointer object and then takes advantage of next GC pass or whatever in order to resolve such proxy in lazy manner. > looking up into the FLobjectId dictionary (decoder>>objects?) when > stitching the references. When you copy the proxies on substitution, it > stitches normally at reference time. Yes, that could work. The problem is if you need instVars of the object you want to substitute. Imagine you have a class called Client and instVar 'age'. And you want to substitute Client with instances of OldClient if 'age' is > 10. At that point in time, the reference to the instVar 10 has not yet been filled. Yet, you need to replace the object at graph construction time. > > > As for Marea and Ghost, >> >> Ghost proxies paper: https://hal.inria.fr/hal-01081236/document >> Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost >> >> Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf >> Current repo: http://ss3.gemstone.com/ss/Marea.html >> >> And finally, my PhD thesis: >> https://tel.archives-ouvertes.fr/tel-00764991/document >> > > Nicely done, sir. I'll check them out, thankyou. > > Thanks, feel free to ask questions. > In Marea I needed custom clusters for my proxies because the serializer >> itself sends messages to the objects being serialized. My proxies would >> bring back graphs from a secondary memory. So if I was serializing a >> graph that had proxies already, I didn't want that. So I hooked my >> custom cluster for proxies that send only a few special messages to the >> proxy that these understand and answer rather than intercept those >> messages. >> >> As for how to extend Fuel for this, I recommend to check the code of >> Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. >> I have a Marea one click here: >> https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0 >> >> Cheers, >> > > thanks so much ^^ > Robert > > >> >> On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers >> <robert.w.withers(a)gmail.com <mailto:robert.w.withers@gmail.com>> wrote: >> >> Good morning, Max. Thank you for the example. I got a little >> confused, between migrations and substitutions. My issue with no-arg >> blocks, I believe, is the inability to access my scope object to >> maintain the object tables. >> >> I'm attempting to write my own Materializer, Decoder and >> Materialization. At the moment, I'm just going to walk the graph, >> testing and do #becomes:. See how well that works when I get a test. >> >> It's really good to know about that other list. >> >> thanks so much ^^ >> Robert >> >> On 10/14/2015 04:15 AM, Max Leske wrote: >> >> BTW, there is a dedicated Fuel mailing list: >> pharo-fuel(a)lists.gforge.inria.fr >> <mailto:pharo-fuel@lists.gforge.inria.fr> >> <mailto:pharo-fuel@lists.gforge.inria.fr >> <mailto:pharo-fuel@lists.gforge.inria.fr>> >> >> Max >> >> >> On 14 Oct 2015, at 09:45, Max Leske <maxleske(a)gmail.com >> <mailto:maxleske@gmail.com> >> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>>> >> wrote: >> >> >> On 14 Oct 2015, at 04:39, Robert Withers >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>> wrote: >> >> >> On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote: >> >> >> >> On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>> wrote: >> >> Hi Mariano, >> >> This presents me with a big challenge, then. I >> read the docs and >> explored the code and the only other aspect not >> mentioned, beyond >> instance creation (#fuelNew #fuelNew:) and >> postMaterialization >> (#fuelAfterMaterialization), is migrations. >> However, migration only >> allows for instanceVar mappings, no code blocks. >> >> >> What do you mean that migrations only allows instVar >> mappings , and no >> code blocks? I mean, what do you mean by code blocks? >> >> >> Sounds to me like this (see FuelOutStackDebuAction): >> >> serializeTestFailureContext: aContext toFileNamed: aFilename >> | serializer | >> >> serializer := FLSerializer newDefault. >> self encodeDebugInformationOn: serializer. >> serializer addPostMaterializationAction: [ :materialization | >> Smalltalk tools debugger >> openOn: Processor activeProcess >> context: materialization root >> label: 'External stack' >> contents: nil >> fullView: false ]. >> >> serializer >> " use the sender context, generally the current context is not >> interesting" >> serialize: aContext >> toFileNamed: aFilename >> >> This stores a block in the serialization which is evaluated >> after >> materialization. The only requirement is that itâs a clean >> block (no >> closure!). >> >> We also support class renames. This is here: >> >> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_… >> >> Which kind of migration example you have in mind >> that would not be >> supported? An example would help. >> >> >> Well, my pics will demonstrate. I am interested in doing >> more than >> mappping ivars or a class rename. I want to do a total >> substitution, >> then a further substitution on the receiving, import side: >> >> Vat1: anObject (Class A) ---> On wire: desc (Descriptor) >> ---> Vat2: >> aProxy (Class FarERef) >> >> A desc is substituted for a PassByProxy object, then a >> FarERef is >> substituted for the desc. >> >> >> #fuelAccept: is a serialization side method. >> >> If Fuel supports substitution on serialization, I >> don't understand >> why no substitution support on materialization. >> >> >> There was a reason, which I cannot remember >> completely. Maybe Martin or >> Max can remember. >> >> >> It seems your focus was pickling to disk then back. My >> focus is >> distributed proxy graphs, which has different use cases. >> >> >> >> I am definitely going to use the world-class Fuel >> binary >> serialization system. However, I find myself >> needing to extend Fuel >> to support substitution on materialization. >> Perhaps the solution is >> a custom decoder. >> >> >> I have made custom clusters for example for my Ghost >> proxies of Marea >> system. It was a perfect example of how I could >> extent Fuel besides the >> common hooks. Fuel provides many places for >> extending , like clusters, >> analyzer, etc >> >> >> Right on, exactly! Could you tell me more about your >> Ghost proxies >> and Marea, please? As well, could you mention how you >> select a custom >> cluster on the serialization side? >> >> >> thanks so much ^^ >> Robert >> >> >> >> No, a bit more. It looks like I need a new >> FLSubstitutePointerObjectCluster, write them on >> serialization with >> the substitute, then do unsubstitution on >> materialization, since the >> cluster controls materialization and not the >> decoder. >> >> Does this approach seem sound to you, from a you >> know architecture >> and design approach? >> >> >> There was an issue. Hope other can remember. If not, >> I will try to >> explan what I remember tomorrow. >> >> >> thanks so much ^^ >> Robert >> >> On 10/13/2015 04:49 PM, Mariano Martinez Peck >> wrote: >> >> No, unfortunately, as far as I can remember, >> we do not have >> that. There >> are another hooks you may use but only in >> certain scenarios >> (#fuelNew, >> #fuelAfterMaterialization, global sends, >> etc). But everything is >> listed >> in >> >> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Customizing… >> so if you didn't find anything of help in >> there there are >> chances >> there isn't anything. >> >> Cheers, >> >> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>>> wrote: >> >> Yes, I meant dynamic substitution on >> materialization, to >> use the >> correct terminology. >> >> thanks, >> Robert >> >> >> On 10/13/2015 11:40 AM, Max Leske wrote: >> >> >> On 13 Oct 2015, at 17:16, Robert >> Withers >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>>> wrote: >> >> Every extra source helps, thank >> you. I see how to do >> non-stream substitutions on >> materializations, but the >> documentation did not indicate a >> way to do non-stream >> substitutions on serialization. >> Is it possible? >> >> >> I donât understand what you mean by >> ânon-streamâ. Could >> you give >> an example? >> >> >> thanks, >> Robert >> >> On 10/13/2015 09:00 AM, Mariano >> Martinez Peck wrote: >> >> Hi Robert, >> >> As for the documentation, >> you have LOTS of >> tests, you >> have the chapter >> Torsten pasted, you have >> this documentation: >> http://rmod.inria.fr/web/software/Fuel >> >> But also, as for internals, >> there is a journal >> paper we >> wrote: >> >> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf >> >> Let us know how it goes, >> >> >> On Tue, Oct 13, 2015 at 6:00 >> AM, Torsten Bergmann >> <astares(a)gmx.de >> <mailto:astares@gmx.de> <mailto:astares@gmx.de >> <mailto:astares@gmx.de>> >> <mailto:astares@gmx.de <mailto:astares@gmx.de>> >> <mailto:astares@gmx.de >> <mailto:astares@gmx.de> <mailto:astares@gmx.de >> <mailto:astares@gmx.de>>> >> <mailto:astares@gmx.de >> <mailto:astares@gmx.de> <mailto:astares@gmx.de >> <mailto:astares@gmx.de>> >> <mailto:astares@gmx.de >> <mailto:astares@gmx.de> <mailto:astares@gmx.de >> <mailto:astares@gmx.de>>>>> wrote: >> >> Hi Robert, >> >> Also checkout the >> chapter on Fuel in Pharo >> Enterprise book: >> >> >> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/l… >> >> Bye >> Torsten >> >> Gesendet: Dienstag, 13. Oktober 2015 um >> >> 09:44 Uhr >> >> Von: "Robert Withers" >> >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>>>> >> >> An: pharo-dev(a)lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>>>> >> >> Betreff: Re: [Pharo-dev] binary >> >> serialization >> >> >> Yes, I have to do object substitutions. >> >> Thanks >> for the link! >> >> >> thanks, >> Robert >> >> On 10/13/2015 03:43 AM, Max Leske wrote: >> >> >> On 13 Oct 2015, at 09:40, Robert Withers >> >> >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>>>> wrote: >> >> >> Sven and Torsten, that's a binary >> >> serialization library! It >> will take time to learn >> it and how to use >> mappers. >> >> >> What is the format; is it language >> >> neutral? >> >> >> For quick serialization you donât >> >> need to do >> anything. It works >> for (almost) all >> objects. Only if you >> want to >> exclude things or >> treat some objects in a >> special way, you >> will need >> to do some stuff. >> >> >> Documentation: >> >> http://rmod.inria.fr/web/software/Fuel. >> >> >> >> >> thanks, >> Robert >> >> On 10/13/2015 01:21 AM, Sven Van >> >> Caekenberghe >> wrote: >> >> Yes, it is called FUEL and it is a >> >> standard >> part of the >> image. See FLSerializer >> and FLMaterializer. >> >> >> On 13 Oct 2015, at 06:59, Robert >> >> Withers >> >> <robert.w.withers(a)gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>> >> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com> >> <mailto:robert.w.withers@gmail.com >> <mailto:robert.w.withers@gmail.com>>>>> wrote: >> >> >> Does Pharo have stream classes to >> >> binary >> de/serialize an >> object, such that the >> protocol accepts an >> object as >> an argument and >> converts it to a >> byteArray? >> >> >> -- >> thanks, >> Robert >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> <Exporting Vat.jpg><Importing Vat.jpg> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > -- Mariano http://marianopeck.wordpress.com

Oct. 14, 2015

Corporate sponsored popularity

by Ben Coman

Just a passing thought... When comparing the popularity of programming languages, maybe technological features are less important than sponsorship by a large tech corporation. Anyhow I don't mind that the masses haven't discovered Pharo. I enjoy it. http://insights.dice.com/2015/10/13/apples-swift-is-killing-objective-c/ cheers -ben

Oct. 14, 2015

Re: [Pharo-dev] what is NativeBoost?

by Robert Withers

Max, if I were to call a blocking 'select' call for NIO, do you think it would lock the image? Thx --- robert > On Oct 14, 2015, at 11:13 AM, Max Leske <maxleske(a)gmail.com> wrote: > > >> On 14 Oct 2015, at 17:01, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> >> I ran across it browsing and it looks pretty interesting. It's purpose remains a mystery to to me. > > NativeBoost provides an FFI. Itâs used by Athens for example to render the fonts in Pharo using Cairo (http://cairographics.org) > > Cheers, > Max > >> >> -- >> thanks so much ^^ >> Robert > >

Oct. 14, 2015

Re: [Pharo-dev] what is NativeBoost?

by Robert Withers

;-) I knew you had fonts. Somewhere inside, i just had that feeling. Cheers, --- robert > On Oct 14, 2015, at 11:13 AM, Max Leske <maxleske(a)gmail.com> wrote: > > >> On 14 Oct 2015, at 17:01, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> >> I ran across it browsing and it looks pretty interesting. It's purpose remains a mystery to to me. > > NativeBoost provides an FFI. Itâs used by Athens for example to render the fonts in Pharo using Cairo (http://cairographics.org) > > Cheers, > Max > >> >> -- >> thanks so much ^^ >> Robert > >

Oct. 14, 2015