- Pharo-dev - lists.pharo.org

Re: [Pharo-project] [COTDC] 61 - InternetConfiguration

by laurent laffont

On Mon, May 9, 2011 at 8:34 PM, laurent laffont <laurent.laffont(a)gmail.com>wrote: > Today: InternetConfiguration > I read several parameters related to the OS network preferences, through the InternetConfigPlugin VM module. For example, to get the default path where downloaded files should be placed: InternetConfiguration getDownloadPath (That said, it's not used in the image). Laurent. > > Comment Of The Day Contest - One Day One Comment > Rules: > #1: Each day a not commented class is elected. Each day the best comment > will be integrated with name of the author(s). > #2: If you cannot comment it, deprecate it. > Results: http://code.google.com/p/pharo/wiki/CommentOfTheDayContest > > Laurent >

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Mariano Martinez Peck

On Tue, May 10, 2011 at 1:44 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > The fix in > http://code.google.com/p/pharo/issues/detail?id=4186 > > now one thing left: > we need to get rid of hanging methods with corrupted trailers. > > And we do it like this: > > (CompiledMethod allInstances select: [:m | > m isInstalled not and: [ m trailer kind = #VarLengthSourcePointer ]] > ) do: [:each | > each at: each size put: 0 ] > > This will replace a corrupted trailer with empty one. > > Excellent!!! that worked :) Thanks Igor. Now...it is supposed that when this issue fix is integrated the new generated image should now have these weird CMs so that script should not be necessary. I will test it when such image is build by hudson :) -- Mariano http://marianopeck.wordpress.com

May 10, 2011

[Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations

by Sven Van Caekenberghe

SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations HTTP client and server functionality has become an essential part of any Smalltalk implementation. There exist various open source solutions delivering this functionality. Users quickly find out that HTTPS is not as readily available as HTTP. Although a solution exists in the form of SqueakSSL, it is not easily portable to Pharo and certainly not to other Smalltalk implementations. Furthermore, SqueakSSL is incomplete and more a working proof of concept than a high quality, high performance, maintainable piece of code, partially because of its dependence on the internals of SocketStream. Hence, some people wanting HTTPS on Pharo as well as one organisation have come foreword with a bounty to force a solution. Here are the requirements, a plan of action and a number of steps to build such a solution. First of all, it would be stupid not to start from the work already done for SqueakSSL. Any new solution should start from the existing plugin, from the existing set of primitives and from the current implementation of these primitives for the Windows, Mac OS X and Linux platforms. The Smalltalk part of the solution should be as platform independent as possible. We therefor need a new, clean implementation of SocketStream, with proper internal buffering and optimised versions of the bulk input and output primitives. The code should be time and space efficient. At first, only binary I/O is needed. Encoding/decoding is best done by wrapping the stream. Maybe a bivalent stream could be an option. The current ASCII support in quite limited. Next a SecureSocketStream can be implemented using this clean base combined with the SSLSession primitives. Here too, time and space efficiency are important although SSL will by definition be less efficient. Eventually, HTTP frameworks can then use this SecureSocketStream to actually implement HTTPS. Although the primary requirement is client functionality, server functionality should be provided as well. Next the existing issues with certificates need to be resolved, on all 3 platforms. Finally, the C code of the plugin for each of the 3 platforms needs to be cleaned up and improved where necessary. This requires contributions from developers who know and understand a platform as well as how SSL is supposed to work. Furthermore they have to understand and be motivated to work on Smalltalk plugins. Up to 3 different developers might be needed. The current implementation on Mac OS X does not seem to (directly) use the standard OpenSSL library from Linux although this library exists on Mac OS X. This could be an opportunity to reduce the number of implementations of the plugin to 2 API's. Needless to say the whole implementation should be open source licensed (MIT), be documented and have a number of units tests to cover it's main functionality as well edge cases encountered in the field. Friendly cooperation with other open source projects in the Smalltalk world is a plus. This project should benefit the whole community. The bounty should be split in different parts. The first part is setting up the new project, delivering the new SocketStream, SSLSession and SecureSocketStream, including the HTTPS client proof of concept. Traction within the community has to be developed. Next the issues with the plugin have to be identified. Then the work on the plugin code for each platform has to be defined and 3 more bounties could be offered. The goal should be to include the plugin in VM's by default. With a proper solution to the plugin issues, SSL server functionality should be no problem. I offer to take on part one and help manage the coordination of this project. So is there anyone capable and willing to help out on the plugin on on one or more platforms for part of the bounty ? Are there others who wish to raise the bounty ? We will need a couple of good beta testers who understand this field as well. Sven

May 10, 2011

Re: [Pharo-project] Additional Test for Shared Queue (s)

by StÃ©phane Ducasse

tx! I want more of such tests. :) On May 8, 2011, at 12:59 AM, pharo-project-owner(a)lists.gforge.inria.fr wrote: > > While trying to track down a bug, I started thinking that there was a problem with SharedQueue. There isn't, but there were no tests that convinced me of that. So I wrote one that actually has two processes reading and writing many times to a Q. > > The same tests works for NSSharedQueue (except that the class under test is hard coded). I suggest that putting these tests in the appropriate packages/core would be a good idea. > > Tests attached. > > <NSSharedQueueTest-testCounting.st><SharedQueue2Test-testCounting.st> > > >

May 10, 2011

Re: [Pharo-project] Which are the limitations of MessageTally?? Re: new Cog VMs uploaded

by Eliot Miranda

On Sun, May 8, 2011 at 5:07 AM, Mariano Martinez Peck <marianopeck(a)gmail.com > wrote: > > > What approach do you suggest should I take? >>> >> >> I have no idea. You could try and break down the various activities and >> benchmark them separately I suppose. You could look at the profiles you're >> already generating in higher resolution to see where the time is going. You >> could also try and extract the SystemProfiler from OpenQwaq, which will run >> on Cog, and is better than MessageTally. Andreas wrote it to get around >> some of MessageTally's limitations. It should hopefully show primitives >> quite accurately. >> >> > Hi. I have already read this several times. From you now, from Alexandre > Bergel, from Henrik Johansen, etc. But no one (or maybe I didn't see it) > give a list of the real existent MessageTally limitations or problems. So > far, the only thing I read is what Eliot says here: "show primitives quite > accurately" > > So...which are the rest and what is solved by the SystemProfiler from > OpenQwaq ?? > MessageTally works by using delays to cause a high-priority process to interrupt the process being profiled. So the sample is taken only at a suspension point. The SystemProfiler support in the VM takes a sample either after a primitive has run or when the VM is checking for events, so it takes more accurate samples. Further, the SystemProfiler's VM support also samples the active process when collecting the sampled method, so it more accurately ascribes profiles to processes. MessageTally only profiles a single process. Both suffer from a bug my colleague Yaron Kashai observed this week at Cadence, which is that if GC pauses are long then both profilers will under-measure the GC costs because they both take a single sample before measurement. So e.g. if a GC pause lasts five times longer than the poll time only one sample will be takem. Ideally the SystemProfiler would be extended to maintain a list of sampled methods and processes. best, Eliot > Thanks in advance, > > > Mariano > > > > > >> HTH >> Eliot >> >> >>> >>> Cheers >>> Philippe >>> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com > >

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

Since code.google.com is broken, i putting it here. Embed source code for methods which get uninstalled from their classes. When compiled method is deinstalled from its class, its not managed anymore by development tools and it's hard to predict, how long a method could stay in the image, because if it contains blocks,they could still reference it. This change makes sure that when you deinstall method from its owning class, its source code are preserved in an embedded form inside a trailer. This is in addition to issue #... that one.. cant see it without google working, sorry :) -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

here an example how it works. first evaluate TTT blockValue so it assigns Block with closure returned by methodWithBlock. Now go to #methodWithBlock and change its implementation. And then try to debug TTT blockValue and you can see that when you evaluating a block , it will show you an original source code of method. (Except that there is some bug in debugger, which erasing the text area once you click on frame with block. But when you click it second time it show it normally). On 10 May 2011 16:46, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> oh , and forgot to note, that having a source code available at all >> times is quite useful in cases, >> when your methods using some kind of DSL. Because decompiler can only >> reconstruct a source code of smalltalk methods, >> but not the ones, which were authored using customized parser/compiler >> .. like using Helvetia. >> > > Btw, the change i proposing is _minimal_ and straightforward: > > basicAddSelector: selector withMethod: compiledMethod > Â Â Â Â "Add the message selector with the corresponding compiled method to the > Â Â Â Â receiver's method dictionary. > Â Â Â Â Do this without sending system change notifications" > > Â Â Â Â | oldMethodOrNil | > Â Â Â Â oldMethodOrNil := self lookupSelector: selector. > Â Â Â Â self methodDict at: selector put: compiledMethod. > Â Â Â Â compiledMethod methodClass: self. > Â Â Â Â compiledMethod selector: selector. > > Â Â Â Â "Now flush Pharo's method cache, either by selector or by method" > Â Â Â Â oldMethodOrNil ifNotNil: [ > Â Â Â Â Â Â Â Â oldMethodOrNil flushCache. > ++++ Â Â oldMethodOrNil dropSourcePointer. > Â Â Â Â Â Â Â Â ]. > Â Â Â Â selector flushCache. > > > :) > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

> > oh , and forgot to note, that having a source code available at all > times is quite useful in cases, > when your methods using some kind of DSL. Because decompiler can only > reconstruct a source code of smalltalk methods, > but not the ones, which were authored using customized parser/compiler > .. like using Helvetia. > Btw, the change i proposing is _minimal_ and straightforward: basicAddSelector: selector withMethod: compiledMethod "Add the message selector with the corresponding compiled method to the receiver's method dictionary. Do this without sending system change notifications" | oldMethodOrNil | oldMethodOrNil := self lookupSelector: selector. self methodDict at: selector put: compiledMethod. compiledMethod methodClass: self. compiledMethod selector: selector. "Now flush Pharo's method cache, either by selector or by method" oldMethodOrNil ifNotNil: [ oldMethodOrNil flushCache. ++++ oldMethodOrNil dropSourcePointer. ]. selector flushCache. :) -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] About CompiledMethod category returning nil sometimes

by Mariano Martinez Peck

Hi Stef. From what I can see, the category can be nil only when CompiledMethods are created for DoIts but not for regular CompiledMethods. Even those which are under 'as yet unclassified' answer that and not nil. (CompiledMethod allInstances select: [ :each | (each category isNil and: [(each selector beginsWith: #DoIt) not]) ]) size answers 0 in my image. (CompiledMethod allInstances select: [ :each | each category isNil ] thenCollect: [:each | each selector]) asSet -->> a Set(#DoIt #DoItIn:) So...only those guys answers nil. Is that important? If so, we already have the message: CompiledMethod >> isSelfEvaluating ^self methodClass notNil and: [(#(#DoIt #DoItIn: nil) includes: self selector) not] that answers false for those do its. Hence, we could patch to something like this: CompiledMethod >> category self isSelfEvaluating ifFalse: [^ 'evaluating']. ^self methodClass organization categoryOfElement:self selector There is another solution: the CM to answers its category delegates to "^self methodClass organization categoryOfElement:self selector" for do-its, the methodClass is always "UndefinedObject" So.....I can create a category called "evaluation" in UndefinedObject and put these two new methods: DoIt "When execution code from a workspace or similar places the Compiler creates a dummy CompiledMethod whose methodClass is UndefinedObject. When such CompiledMethods are ask its category, it answers nil, since the selector is search in the organization of the methodClass, which in this case is UndefinedObject. Hence, implementing this mehtod here will make such CompiledMethods to answer the category where this method is. " DoItIn: nonImportant "When execution code from a workspace or similar places the Compiler creates a dummy CompiledMethod whose methodClass is UndefinedObject. When such CompiledMethods are ask its category, it answers nil, since the selector is search in the organization of the methodClass, which in this case is UndefinedObject. Hence, implementing this mehtod here will make such CompiledMethods to answer the category where this method is. " So....now: (CompiledMethod allInstances select: [ :each | each isSelfEvaluating not ] thenCollect: [:each| each category ]) asSet --->a Set(#evaluation) I am not sure which of the both solution is the best nor the impact of them, Cheers Mariano On Mon, May 9, 2011 at 11:35 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > wrote: > Hi guys > > With cyrille we experience sometimes CompiledMethod with nil category and > this can break hard some problem. > We were thinking but not sure that it would be good to make sure that nil > is not returned but 'as yet unclassified' > What do you think? > Do you have any idea of potential problem? > > Stef > -- Mariano http://marianopeck.wordpress.com

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

On 10 May 2011 16:22, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 10 May 2011 13:33, Henrik Sperre Johansen > <henrik.s.johansen(a)veloxit.no> wrote: >> On 10.05.2011 12:44, Igor Stasenko wrote: >>> >>> On 10 May 2011 11:27, Mariano Martinez Peck<marianopeck(a)gmail.com> Â wrote: >>>> >>>> On Tue, May 10, 2011 at 10:59 AM, Igor Stasenko<siguctua(a)gmail.com> >>>> Â wrote: >>>>> >>>>> On 10 May 2011 09:48, Mariano Martinez Peck<marianopeck(a)gmail.com> >>>>> Â wrote: >>>>>> >>>>>> On Tue, May 10, 2011 at 9:41 AM, Marcus Denker<marcus.denker(a)inria.fr> >>>>>> wrote: >>>>>>> >>>>>>> On May 10, 2011, at 9:32 AM, Mariano Martinez Peck wrote: >>>>>>> >>>>>>>> what is even worst, is that even after removing those correct CM, and >>>>>>>> doing a GC etc...they still don't disappear. >>>>>>>> >>>>>>> There is >>>>>>> >>>>>>> Â Â Â Â ScriptLoader new cleanUpMethods >>>>>>> >>>>>>> This is called from #cleanUpForRelease and normally should make sure >>>>>>> there >>>>>>> are no old methods. >>>>>>> >>>>>> >>>>>> hehehehe if I do: ScriptLoader new cleanUpForRelease, then inspect >>>>>> ((CompiledMethod allInstances select: Â [:each | each trailer kind = >>>>>> #VarLengthSourcePointer] ) ) >>>>>> >>>>>> they are not GC'ed and if then I click on the first element in the >>>>>> inspector.... VM CRASH!!! Â with both, InterpreterVM and CogVM. >>>>>> >>>>>> :( >>>>>> >>>>> okay. it seems i found the offender. Its a CompiledMethod >>>>> class>>cleanUp. >>>>> It changing a source pointer of all non-installed methods to 0. >>>> >>>> Good catch!!! Â :) Â Anyway, why CompiledMethod class>>cleanUp would like >>>> to >>>> destroy source pointers??? >>>> If I understood correctly (please correct me), all compiled methods will >>>> loose the pointer to sources and hence they will be decompiled after when >>>> they are ask their source! >>> >>> Yes, if you wipe the sourcePointer, there is no way how you can get a >>> source code of method. >> >> I believe the "not getting the source code" was on purpose. >> It's done f.ex. as part of condenseChanges, as the source pointers would >> have been invalid anyways. >> The intent I believe is that you should get decompiled source if you inspect >> old(but still referenced) CompiledMethods, instead of a bogus display of >> some random part of the new changes file, and remove error handling in >> source lookup where pointer could potentially be beyond end of file. >> >> This of course worked much faster before trailers, when setSourcePointer: >> actually modified the instance inline, rather than become'ing a copy, >> nowadays I agree with Igor that it'd probably be more conceptually clean to >> the empty MethodTrailer in destroySourcePointer :) >> >> Just remember to make sure such instances decompile as expected to uphold >> why the zapping was done in the first place. >> If we're lucky, Â it won't crash the VM either ;) >> > > Actually , in #destroySourcePointer setting sourcePointer = 0 is pretty useless. > Since now, we can embed source code in trailer, we can replace a > senders of it with #dropSourcePointer. > Then you don't have to decompile the method, even if its no longer > installed and cannot be located in .sources or .changes files anymore. > > We could even do this at the method installation time: > Â - if a newly installed method replacing an old one, > we tell old one to drop its source pointer immediately. > > Then, by definition, you cannot have non-installed methods with wrong > source pointer, and all those methods will still be able to show you > an original source code. > > Sounds like an idea? > oh , and forgot to note, that having a source code available at all times is quite useful in cases, when your methods using some kind of DSL. Because decompiler can only reconstruct a source code of smalltalk methods, but not the ones, which were authored using customized parser/compiler .. like using Helvetia. >> Cheers, >> Henry >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig.

May 10, 2011