July 2009 - Pharo-dev - lists.pharo.org

[Pharo-project] proposed fix for issue 913

by Dale Henrichs

http://code.google.com/p/pharo/issues/detail?id=913 I wasn't able to directly reproduce the problem in my tests, but upon code inspection it was obvious that an empty Dictionary should be returned by BytecodeEncoder>>blockExtentsToTempRefs instead of nil. None of the senders would be adversely affected by the change. Of course the changes are in the Compiler package, so it's worth having an extra set of eyes take a look. Dale

July 3, 2009

Re: [Pharo-project] adding nil to sets

by Igor Stasenko

2009/7/3 Reinout Heeck <reinz(a)desk.org>: > > On Jul 3, 2009, at 8:53 AM, Ralph Boland wrote: >> >> Since I am here allow me to state my opinion on the matter. >> We should not add the ability to add nil to Sets. Â While I admit I >> don't >> know for sure, I believe the number of instances where we need to add >> nil to a set is rare. > > > I am of the opposite opinion (and quite strongly so :-) > > When writing code that uses sets I don't want to be bothered with the > need to think about special cases (like nil). > I agree that the number of instances where it *actually* happens is > small but it does force authors to think of it *every* time they write > code that adds objects to sets -- particularly hard to keep up when > those objects are handed in from yet another module not under their > control. > > Smalltalk being about abstractions and Pharo being about raising > squeak to a 'professional' level I think it is very important to > isolate the Pharo programmer from such library implementation > idiosyncrasies -- they should be allowed to write abstract (naive if > you will) code without fear that it breaks on such special cases. > > >> Â For the rare cases in which it is needed we should >> create a class Â SetSupportingNils which has two instance variables: >> Â Â containsNil: Â a boolean set to true iff Â the SetSupportingNils >> Â Â instance contains nil. > > > IMO such a set should be named 'Set' and the one that optimizes for > not holding nil should be the one that is given a special name. > > As a critique of the proposed implementation: If you put the sentinel > in the newly introduced ivar (instead of a boolean) the implementation > will be simplified (all nil checks become checks against the sentinel > as opposed to introducing 'case statements' in most of its > implementation). > > I think that Andres' suggestion of using self as sentinel might be > hitting a very sweet spot as a tradeoff between special cases and run- > time efficiency. > > >> >> >> Issue: >> Â Â I suppose we need a Â BagSupportingNils class too. > > Yes, and it should be named 'Bag' ;-) > > > Now there clearly are costs associated with doing this: > > Set instantiation requires overwriting nils put there by the VM, this > could be mitigated in the future by adding object instantiation > protocol that is handed the sentinel. Perhaps in the short run this > can be decoupled from the VM by using a plugin -- but I don't know if > a plugin is allowed to take over object allocation. > > Custom set subclasses will break (but only if they explicitly check > for nilled slots) -- but this seems like a mostly fail-fast scenario > to me so it requires a code maintenance investment that seems mostly > limited to a one-off cost. Anyway, finding all subclasses of Set > doesn't seem too hard to me ;-) > > In short: Set is cheating and it is doing it poorly, it gets caught > when code happens to stuff a very public object called nil in it. > It should be taught to cheat better so it doesn't pollute code at > other levels of abstraction with special cases. > > And yes: this requires an investment... > Thanks for supporting my point , Reinout . For those who is not aware, i wrote a GenericSet class, and it is showing the difference (how much more complex it comparing to current Set). I can also modify a code which avoids using extra instance var. Simply by encoding presence of nil in the 'tail' ivar: 0 or positive - nil is not included negative - nil is included > R > - > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

July 3, 2009

Re: [Pharo-project] adding nil to sets

by Cameron Sanders

On Jul 3, 2009, at 9:53 AM, Reinout Heeck wrote: > I think that Andres' suggestion of using self as sentinel might be > hitting a very sweet spot as a tradeoff between special cases and run- > time efficiency. The idea of using 'self' as a special flag sure seems like a special case; and while I have never created a self referential sets... it just seems like someday somebody will get burned badly discovering this implementation (as a matter of debugging their app). As someone still on the learning curve (less than two years of smalltalk scattered over time), I have come to assume that anything storing data with "holes" in it (a/k/a sparse) will fill those holes will nil, and therefore nil is NOT an acceptable element to be stored. That said, let me add that it seems like I have to create a NullDatum (a nil'ish singleton) type object with some frequency. I seem to remember the first axiom from modern Set Theory goes something like this: the empty set (nil) is a set (value)! So, in principle, why not allow nil on the public interface (which could be mapped to some singleton in the implementation)? There would be no speed penalty -- in the most minimal form of always remapping nil in the public interface to NullDatum (or whatever it would be called) would be insignificant. I suppose it could break some existing code... somewhere outside the collection classes. On the other hand, using a specialized subclass is OK too, especially given that I do not have the time to go change the all of the collection classes to make certain they are consistent in allowing nil. -Cam

July 3, 2009

Re: [Pharo-project] creation of ReadWriteStream with #on:

by Alexandre Bergel

Redefining on: on ReadWriteStream generate an error when I save with Monticello. I haven't got the time to dig into this however. Cheers, Alexandre On 30 Jun 2009, at 16:21, Henrik Johansen wrote: > Well, my point was that by copying that, you don't set writeLimit, so > attempting to write to the ReadWriteStream results in an error... > (which is why I gave a test that both reads and writes instead of just > reading) > In that case, what's the point? It's functionality is then just the > same as a ReadStream. (Which is doubly confusing, seeing as it > inherits from WriteStream) > > so doing it your way, we'd have to do: > > on: aCollection > readLimit := aCollection size. > writeLimit := aCollection size. > position := 0. > self reset. > > instead of a straight copy. > > Btw, your version has a dependency on the super as well, mainly that > it defines the correct instvars. :) > Which version to prefer comes down to taste, I guess. > > Cheers, > Henry > > On Jun 30, 2009, at 3:54 35PM, Alexandre Bergel wrote: > >>> You haven't set the writeLimit by just copying PositionableStream >>> though, have you? >> >> I suggested to copy PositionableStream>>on: in ReadWriteStream: >> on: aCollection >> >> collection := aCollection. >> readLimit := aCollection size. >> position := 0. >> self reset >> >> Your implementation works well too, it has no duplication, but has a >> dependency over the implementation of the superclass. >> >> Cheers, >> Alexandre >> >> >>> >>> >>> >>> On Jun 30, 2009, at 3:00 46PM, Alexandre Bergel wrote: >>> >>>>> Put the readLimit at 0 make sense for a WriteStream ( with witch >>>>> your not supposed to read ) but when you create a ReadWriteStream >>>>> on >>>>> a collection you would be able to read what is already in the >>>>> collection, no? >>>>> I wonder why #on: has not been overriden in the instance side of >>>>> ReadWriteStream ? >>>>> Maybe it's not a good idea to create a ReadWriteStream with a >>>>> collection that already includes elements ? >>>> >>>> >>>> It looks like on: must be overridden in ReadWriteStream. >>>> With ReadStream I have: >>>> (ReadStream on: (1 to: 20)) next => 1 >>>> (ReadStream on: (1 to: 20)) next; next => 2 >>>> >>>> However, with ReadWriteStream, nil is returned, for the very >>>> problem >>>> you mentioned: >>>> (ReadWriteStream on: (1 to: 20)) next => nil >>>> >>>> If I cut and paste PositionnableStream>>on: in ReadWriteStream, >>>> then I >>>> have the behavior I expected: >>>> (ReadWriteStream on: (1 to: 20)) next => 1 >>>> (ReadWriteStream on: (1 to: 20)) next; next => 2 >>>> >>>> Cheers, >>>> Alexandre >>>> >>>> -- >>>> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >>>> Alexandre Bergel http://www.bergel.eu >>>> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >>>> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>> >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> >> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

July 3, 2009

[Pharo-project] Problem with fork

by Mariano Martinez Peck

Hi all, this is my first post here, just joined this group so let me do a quick introduction. I have 10+ years of experience doing full-time development with VisualWorks (creating trading platforms for the European energy exchanges). Also I have near-zero experience with using Squeak or how its development process works. I'll slowly cut my teeth, starting by interacting on this mailing list. Multi threading is something I have spent a lot of quality time with, so I want to share some thoughts on the following: |semaphores tr| semaphores := Array new: 10. tr := ThreadSafeTranscript new. tr open. 1 to: 10 do: [ :index | semaphores at: index put: Semaphore new ]. 1 to: 10 do: [:i | [ tr nextPutAll: i printString, ' fork '; cr. (semaphores at: i) signal. ] fork ]. semaphores do: [:each | each wait ]. tr show: 'all forks proccesed'; cr. I have seen this pattern often (allocating a semaphore for every forked process), I usually interpret this as a signal that such code is still in its first 'make it work/make it right' stages. What a lot of people don't realize is that at its heart a semapore is a thread-safe counter/register (and if you look at the hierarchy it is implemented on you wouldn't guess that either, since the hierarchy stresses the implementation part that manages waiting processes rather than the counter aspect). So trying to take the code snippet toward 'make it abstract' territory this could be refactored to lean more on the counter aspect of semaphores and use only a single semaphore: |count sem tr| tr := ThreadSafeTranscript new. tr open. count := 10. sem := Semaphore new. 1 to: count do: [:i | [ tr nextPutAll: (i printString, ' fork\') withCRs. sem signal. ] fork]. count timesRepeat: [sem wait]. tr show: 'all forks proccesed'; cr. Now the above is about as far as you can go with the current Squeak and VisualWorks implementations so you can take it as a simple refactoring advise. However I want to press on a bit more (and go a bit off-topic for this list ;-) because I feel it still has a big problem: we need to maintain a 'count' and pass that between the two loops in the above example. In the current example this is not much of a problem but in more complex applications where the forking is done by yet other forked processes we will need to make 'count' thread-safe as well -- I find this very ugly, because you will need an extra semaphore just to make the original semaphore work as required. Furthermore you cannot add new forked processes once the second loop has started running. So here is an experiment I did a couple of years ago with VisualWorks: I altered the VM (just one line of its source ;-) so it would react properly to semaphores that have negative values in the 'excessSignals' instance variable, and I added a method #unsignal to Semaphore that would decrease the value of that ivar. In my experiments that yielded many opportunities to simplify multiprocessing code (not only for thread synchronization but also for passing around counts in a thread-safe register!). In the above code that would allow us to 'pre-load' the semaphore at the place where the threads are created with as result that the 'count' variable can be removed and the bottom loop can be removed too: |count sem tr| tr := ThreadSafeTranscript new. tr open. sem := Semaphore forMutualExclusion. "We need one excessSignal to balance the #wait below" 1 to: 10 do: [:i | sem unsignal. "outside the forked code" [ tr nextPutAll: (i printString, ' fork\') withCRs. sem signal. "balance the unsignal" ] fork]. sem wait. "no loop, no need to know the count!" tr show: 'all forks proccesed'; cr. Above was just a simple refactoring, but look at how I needed it: |sem tr| tr := ThreadSafeTranscript new. tr open. sem := Semaphore new. "no excessSignal this time" "set up a monitoring system first(!)" [ sem wait. tr show: 'all forks proccesed'; cr ] fork. "then create jobs (in my case I had only a single first job that would recursively create more jobs, not shown here)" 1 to: 10 do: [:i | sem unsignal. [ tr nextPutAll: (i printString, ' fork\') withCRs. sem signal. ] fork]. "Now that we are sure at least one job is entered balance the #wait we started out with" sem signal. Since we elided 'count' I can move the code that relied on it up in front of the thread creation code, I very much like this flavor of decoupling. I guess this illustrates that Semaphore is stuck in the 'make it work/ make it right' phase for thirty years now, and that moving it into 'make it abstract' territory will make lots of hairy multi-threading code much simpler to express... (And for those thinking this through: yes I did implement a thread- safe #add: and #valueWithReset on semaphore too ;-) I hope I didn't bore y'all and stray to far off-topic, but I did want to share this bit of insight I gained by tinkering with the semaphore implementation: semaphores are thread-safe counters at their heart. Cheers, Reinout ------- PS: big congrats with the license cleaning milestone, this is what finally pulled me into this project :-) _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 3, 2009

Re: [Pharo-project] bugs with SmallDictionary #scanFor:

by StÃ©phane Ducasse

May be cyrille the best way would be to: rewrite small dictionary using the double arrays and to do it from scratch. I do not know if you want to do that but it could be fun to see if it goes faster than a dictionary for a small sets of items. Since it was designed for that. lukas told me that this class is coming from the refactoringBrowser on VW. Stef > I think SmallDictionary should be removed from the core. There are > no users. > > Lukas > > 2009/7/3 Cyrille Delaunay <cy.delaunay(a)gmail.com>: >> SmallDictionary, unlike all subclasses of Dictionary, never used >> ( and >> initialize ) the instance variable: 'array'. >> Associations are stored with a key array ( called 'keys' ) and a >> value array >> ( 'values' ). >> Problem is that the method #scanFor: is define in Dictionary ( and >> not >> overriden ) with this code : >> scanFor: anObject >> | element start finish | >> finish := array size. >> .... >> 'array' is nil and it cause bugs for those methods : >> -> #includesAssociation: >> -> #associationAt: >> -> #associationAt:ifAbsent: >> -> #associationDeclareAt: >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 3, 2009

Re: [Pharo-project] FasterSets improvements

by Reinout Heeck

>> In >> VW, adding nil is a no-op. > Which is a bug that still needs fixing, it should raise an exception (like dictionary does in VW). > In that case I would not let nil to be a possible element and avoid > incompatibility. Oh, how compatible does Pharo need to be/intend to be with bugs of other platforms? R -

July 3, 2009

Re: [Pharo-project] adding nil to sets

by Reinout Heeck

On Jul 3, 2009, at 8:53 AM, Ralph Boland wrote: > > Since I am here allow me to state my opinion on the matter. > We should not add the ability to add nil to Sets. While I admit I > don't > know for sure, I believe the number of instances where we need to add > nil to a set is rare. I am of the opposite opinion (and quite strongly so :-) When writing code that uses sets I don't want to be bothered with the need to think about special cases (like nil). I agree that the number of instances where it *actually* happens is small but it does force authors to think of it *every* time they write code that adds objects to sets -- particularly hard to keep up when those objects are handed in from yet another module not under their control. Smalltalk being about abstractions and Pharo being about raising squeak to a 'professional' level I think it is very important to isolate the Pharo programmer from such library implementation idiosyncrasies -- they should be allowed to write abstract (naive if you will) code without fear that it breaks on such special cases. > For the rare cases in which it is needed we should > create a class SetSupportingNils which has two instance variables: > containsNil: a boolean set to true iff the SetSupportingNils > instance contains nil. IMO such a set should be named 'Set' and the one that optimizes for not holding nil should be the one that is given a special name. As a critique of the proposed implementation: If you put the sentinel in the newly introduced ivar (instead of a boolean) the implementation will be simplified (all nil checks become checks against the sentinel as opposed to introducing 'case statements' in most of its implementation). I think that Andres' suggestion of using self as sentinel might be hitting a very sweet spot as a tradeoff between special cases and run- time efficiency. > > > Issue: > I suppose we need a BagSupportingNils class too. Yes, and it should be named 'Bag' ;-) Now there clearly are costs associated with doing this: Set instantiation requires overwriting nils put there by the VM, this could be mitigated in the future by adding object instantiation protocol that is handed the sentinel. Perhaps in the short run this can be decoupled from the VM by using a plugin -- but I don't know if a plugin is allowed to take over object allocation. Custom set subclasses will break (but only if they explicitly check for nilled slots) -- but this seems like a mostly fail-fast scenario to me so it requires a code maintenance investment that seems mostly limited to a one-off cost. Anyway, finding all subclasses of Set doesn't seem too hard to me ;-) In short: Set is cheating and it is doing it poorly, it gets caught when code happens to stuff a very public object called nil in it. It should be taught to cheat better so it doesn't pollute code at other levels of abstraction with special cases. And yes: this requires an investment... R -

July 3, 2009

Re: [Pharo-project] Problem with fork

by Adrian Lienhard

On Jul 3, 2009, at 13:35 , Reinout Heeck wrote: [...] > PS: big congrats with the license cleaning milestone, this is what > finally pulled me into this project :-) Thanks, and welcome! Adrian ___________________ http://www.adrian-lienhard.ch/

July 3, 2009

Re: [Pharo-project] adding nil to sets

by StÃ©phane Ducasse

andres > Something that could be considered down the road is restructuring the > collection hierarchy to, among other things, allow pluggable hashed > storage strategies for hashed collections. This approach should > drastically reduce the number of classes in the hashed collection > class > hierarchy (c.f.: what happens in VW). andres do you mean that this is done in VW. >> Thanks ralph. >> I think that this is not a so good idea to have nil as element. >> >> BTW: just to make sure. Our is not to hijack people from the squeak >> community >> (because when I scanned some emails there I thought that we were >> considered as evil >> and bad guys - people have a short memory). >> Our goal is to build an agile Smalltalk with faster, better.... code. >> And to make sure that people can reinvent the future because they can >> use cool >> abstractions. >> >> Stef >> >> >> On Jul 3, 2009, at 8:53 AM, Ralph Boland wrote: >> >> >>> Please note the change of topic; adding the ability to add nil to >>> sets >>> won't make sets faster and may well make them slower so has nothing >>> to to with the FasterSets package. >>> >>> Since I am here allow me to state my opinion on the matter. >>> We should not add the ability to add nil to Sets. While I admit I >>> don't >>> know for sure, I believe the number of instances where we need to >>> add >>> nil to a set is rare. For the rare cases in which it is needed we >>> should >>> create a class SetSupportingNils which has two instance variables: >>> containsNil: a boolean set to true iff the SetSupportingNils >>> instance contains nil. >>> >>> set: an instance of Set or one of its subclasses and used to >>> store any object added to the SetSupportingNils instance except >>> of course nil. >>> >>> This means that SetSupportingNils would need to recognize >>> any message that Sets or its subclasses can understand. >>> Any message not understood by SetSupportingNils would be >>> forwarded to its set instance variable. >>> My proposal is not very efficient when adding nil to a set is >>> needed but it requires no modifications to Set or any of its >>> subclasses. >>> That a rarely used class is somewhat inefficient is acceptable. >>> >>> Issue: >>> I suppose we need a BagSupportingNils class too. >>> Are there any other classes we need? >>> Can they be combined into one class? >>> >>> Regards, >>> >>> Ralph Boland >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> . >> >> > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 3, 2009