September 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] 11635: Race condition in SequenceableCollection>>shuffle

by StÃ©phane Ducasse

On Sep 23, 2013, at 3:17 PM, Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> wrote: > Here is my 100% personal opinion: > > I don't like the copyShuffle. > To me, the rules are quite clear: > sort shuffle reverse etc... -> perform modification in place > sorted shuffled reversed etc... -> answer a copy > I hope the methods comments are clear. Does PBE tells about these conventions? It would be a good thing. I do not remember. Because probably that we wrote it before. I will log it and see for the next release of PBE. > And I don't like to have mutexes in base library, the less we have, the better. > If a user is going to modify the same object concurrently, he/she takes care of mutual exclusion. > > > 2013/9/23 Max Leske <maxleske(a)gmail.com> > Sven suggested posting this on the list for discussion, so here you go: > >> Maybe this should be discussed on the list, your are going to break API. >> >> Note that there is also #sort and #sorted with similar copy behavior. >> >> Also, I am not sure that basic operations should use mutexes to protect themselves by default: there is a cost when you are a single threaded user. Even in Java there are synchronized and non-synchronized versions of collections. IMHO, the protection should happen in your app, and basic collections do not have to be thread safe. >> >> Sven >> >>> #shuffle does not use Collection>>mutexForPicking as other users of #randomForPicking demonstrate. This can lead to race conditions (found in our application). >>> >>> In addition, there are now #shuffle, #shuffled, #shuffleBy: and #shuffledBy: where #shuffled and #shuffledBy: shuffle a copy and answers that. This is very confusing. >>> >>> I propose a fix where #shuffled and #shuffledBy: are renamed to #copyShuffle and #copyShuffledBy: and moved to the "copying" protocol. #shuffle and #copyShuffle will use the mutex to prevent race conditions. >

Sept. 23, 2013

Re: [Pharo-dev] 11635: Race condition in SequenceableCollection>>shuffle

by Henrik Johansen

On Sep 23, 2013, at 3:33 , Max Leske <maxleske(a)gmail.com> wrote: >> >> If a user is going to modify the same object concurrently, he/she takes care of mutual exclusion. > > Agreed. BUT: the Random object used by these methods is the same one that is used by #atRandom for instance, hence the race condition. There is no way anyone can safely use these methods without the mutex, single threaded or not. Calls to methods using that same Random object can be all over the place and also in the base system. It seems to me an existing Random instance is used in this case mostly* for performance. One could argue that since the Random in this case is used for a bulk operation, for which the object creation cost is largely amortized for collection sizes > 20, it's acceptable to instead use Random new by default, which wouldn't suffer from the same race conditions. While still slower than a mutex-protected version for single-threaded code, it would also scale correctly if the users (and vm) are actually multi-threaded. [#(1 2 3 4 5 6 7 8 9 10 11 12) shuffle] bench '208,000 per second.' '222,000 per second.' '223,000 per second.' [#(1 2 3 4 5 6 7 8 9 10 11 12) shuffleWithMutex] bench '188,000 per second.' '186,000 per second.''184,000 per second.' [#(1 2 3 4 5 6 7 8 9 10 11 12) shuffleNewRandom] bench '167,000 per second.' '166,000 per second.' '167,000 per second.' Cheers, Henry * Low seed entropy is another issue, but if purely random shuffling is a critical requirement, one shouldn't use the default Random generator anyways...

Sept. 23, 2013

Re: [Pharo-dev] Help with reviewing needed

by Camillo Bruni

These are all cases with the status "Fix Review Needed" => load the changes in a fresh 3.0 image => review the changes => leave feedback in the issue tracker - positive => set the status to "Fix to Include" - negative => set the status to "Work Needed" On 2013-09-23, at 12:02, "phil(a)highoctane.be" <phil(a)highoctane.be> wrote: > These are all closed things in Inbox. > > What's the help about? > > Phil > > > > On Mon, Sep 23, 2013 at 4:08 PM, Marcus Denker <marcus.denker(a)inria.fr>wrote: > >> Hi, >> >> It would really be helpful if more people would review: >> >> https://pharo.fogbugz.com/f/filters/36/Review >> >> because if it has to wait for me, it might wait for a very long time. >> >> Marcus >>

Sept. 23, 2013

Re: [Pharo-dev] Help with reviewing needed

by phil＠highoctane.be

These are all closed things in Inbox. What's the help about? Phil On Mon, Sep 23, 2013 at 4:08 PM, Marcus Denker <marcus.denker(a)inria.fr>wrote: > Hi, > > It would really be helpful if more people would review: > > https://pharo.fogbugz.com/f/filters/36/Review > > because if it has to wait for me, it might wait for a very long time. > > Marcus >

Sept. 23, 2013

Re: [Pharo-dev] [NB] Trying to implement a ReadStream on NBExternalAddress

by Camillo Bruni

Hi Jan, I think I will add the ByteArray accessor to NBExternalAddress today or tomorrow since I need it as well for another project. Concerning the Streams, I think it is the easiest solution to wrap around a ByteArray. Otherwise you could subclass one of the main stream classes and implement your own primitive methods with FFI to read and write bytes. In the worst case you will have to implement all primitive methods with FFI that you find in the StandardFileStream. On 2013-09-22, at 10:12, Jan van de Sandt <jvdsandt(a)gmail.com> wrote: > Hello, > > I'm trying to implement a binary ReadStream that gets its data not from a > Smalltalk collection but from some external memory address. I got this > address from a library call using NativeBoost. > > The NBExternalAddress has methods to access bytes, int's and longs but > there is no method to get a ByteArray starting at a specified offset and > with a specified length. I need a method like this to implement the > ReadStream>>next: method efficiently. > > Is it possible ti implement such a method or are there other ways to > implement a ReadStream efficiently? > > In [1] I read that there was already an idea to implement a nbAddressAt: > method. This would also be using in combination with NativeBoost > class>>#memCopy:to:size > > Regrads, > Jan. > > [1] > http://forum.world.st/Pharo-dev-NB-Review-amp-fixes-amp-ideas-td4698514.html

Sept. 23, 2013

Re: [Pharo-dev] Integration Process

by Esteban A. Maringolo

Please, this is very well received! I have no clue on how to contribute back, it is not a streamlined process. Thank you Camillo. Esteban A. Maringolo 2013/9/23 Camillo Bruni <camillobruni(a)gmail.com>: > > On 2013-09-23, at 05:13, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > >> >> On Sep 23, 2013, at 6:21 AM, Camillo Bruni <camillobruni(a)gmail.com> wrote: >> >>> I spent today 10mins explaining step by step on how to regular Pharo programmers how to create a bugfix for Pharo! >>> >>> THIS IS BAD! >>> >>> There are many more motivated Pharo programmers outside RMoD, and I am sure that quite some >>> of them would actually provide bug fixes. The current bug-report process has a way too steep >>> learning curve. >>> >>> >>> There are a few things we have to change here: >>> >>> 1. make bugs.pharo.org finally work with dots in the email message >>> 2. make bugs.pharo.org automatically send a password reset request (in the worst case we do a redirect/curl on the normal fogbugz site) >>> 3. make the anonymous inbox work on the smalltalkhub >> >> I would not say anonymous... public, yes :) > > yes this is what I meant, committing without having to be added manually to the contributors list.

Sept. 23, 2013

Help with reviewing needed

by Marcus Denker

Hi, It would really be helpful if more people would review: https://pharo.fogbugz.com/f/filters/36/Review because if it has to wait for me, it might wait for a very long time. Marcus

Sept. 23, 2013

Re: [Pharo-dev] Extending HelpTopic

by kilon

ok thank you for the clarifications :) -- View this message in context: http://forum.world.st/Extending-HelpTopic-tp4709645p4709786.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Sept. 23, 2013

Re: [Pharo-dev] Use pharo users mailing list

by kilon

The one I found curious is that when I go to "smalltalk" tag http://stackoverflow.com/questions/tagged/pharo+smalltalk <http://stackoverflow.com/questions/tagged/pharo+smalltalk> Almost all questions for smalltalk are about pharo. Are we by any chance more popular than we think ? :D Or maybe smalltalkers by large don't like stackoverflow. Its strange though. philippeback wrote > Google and a copy of the list in there is fine enough for me. > > SO is okay but frankly, I do not go there to look for stuff/answer > questions, but because my google search sends me there. > > Where we could have exposure would be on things like infoQ. > > Regards, > --- > Philippe Back > Dramatic Performance Improvements > Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 > Mail: > phil@ > | Web: http://philippeback.eu > Blog: http://philippeback.be | Twitter: @philippeback > Youtube: http://www.youtube.com/user/philippeback/videos > > High Octane SPRL > rue cour Boisacq 101 | 1301 Bierges | Belgium > > Pharo Consortium Member - http://consortium.pharo.org/ > Featured on the Software Process and Measurement Cast - > http://spamcast.libsyn.com > Sparx Systems Enterprise Architect and Ability Engineering EADocX Value > Added Reseller > > > > > On Mon, Sep 23, 2013 at 2:33 PM, Esteban A. Maringolo > < > emaringolo@ > >wrote: > >> The difference not only lies in the format, but also in the user base. >> >> I'm 30+ years old, and I grew up using bulletin boards, newsgroups and >> mailing list, but there is a big generation of "programmers" (emphasis >> in the quotes) that will never subscribe to a mailing list, some don't >> even know what a mailing list is. They grew up using sites like Yahoo >> Answers, and expect the same for their programming questions. >> >> I guess it is the same that happened in the transition from newsgroups >> to mailing lists, and as in the bbc to newsgroups, and newsgroups to >> mailing lists, every new generation seems to me like an order of >> magnitude bigger. >> >> I think BOTH options are okay, it is, a user list and StackOverflow >> like websites. The SEO of the later is really good. >> The flattening of the question/answer curve may have to do with the >> fact that most questions have been answered and they start to repeat. >> >> I use StackOverflow and SuperUser on a daily basis. And I do find most >> of the answers I need. >> >> To me the radical choice would be to merge both users and development >> mailing lists. >> The separation is okay from an organization point of view, but the >> volume today is no that big as to justify fragmentation, seems like an >> "early optimization". >> >> Regards, >> >> >> Esteban A. Maringolo >> >> >> 2013/9/23 H. Hirzel < > hannes.hirzel@ > >: >> > On 9/23/13, Stephan Eggermont < > stephan@ > > wrote: >> >> Kilon wrote >> >>>Here is a radical suggestion you probably don't want to hear. >> >>>Close down Pharo users mailing list, redirect everyone to >> stackoverflow. >> >> >> >> Definitely not. Stackoverflow is nearly dead, and seriously unsuitable >> for >> >> small languages. >> > >> > There are obstacles but one cannot say that it is "seriously >> > unsuitable". Stackoverflow question demand a certain format, typically >> > something like >> > >> > - statement of what you want to accomplish >> > - attempt a solution with code which has problems >> > - question >> > >> >>Good questions will get closed by clueless people. >> > Mostly if the form does not fit. >> > And if there are Pharo people (maybe earned in another area) with >> > enough reputation points this may be prevented >> > >> >> Good answers will get downvoted because they go against majority >> >> views. >> > >> > Why? >> > >> >>The ranking system is heavily skewed towards popular languages. >> > >> > It is based on the number of hits which is naturally less in less >> > popular languages. >> > However subcommunities may function well in stackoverflow. >> > >> > the tagging and referencing system is excellent. >> > >> >> Stackoverflow doesn't allow discussions (and interesting questions), >> > >> > No, but that is not the aim of the site. It is a question and answer >> > site for _coding_ and maybe _design_ problems to a certain extent. >> > >> > However it allows for wiki pages. Any answer may be turned into a wiki >> page. >> > >> >> which is a major reason to have these lists. >> > >> > List are good in addition. >> > >> > --Hannes >> > >> >> To see for yourself, take a look at the questions related to OODBs. >> >> >> >> Stephan >> >> >> >> >> > >> >> -- View this message in context: http://forum.world.st/Use-pharo-users-mailing-list-tp4709698p4709785.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Sept. 23, 2013

Re: [Pharo-dev] 11635: Race condition in SequenceableCollection>>shuffle

by Frank Shearar

On 23 September 2013 14:34, Max Leske <maxleske(a)gmail.com> wrote: > > On 23.09.2013, at 15:20, Frank Shearar <frank.shearar(a)gmail.com> wrote: > >> On 23 September 2013 14:17, Nicolas Cellier >> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>> Here is my 100% personal opinion: >>> >>> I don't like the copyShuffle. >>> To me, the rules are quite clear: >>> sort shuffle reverse etc... -> perform modification in place >>> sorted shuffled reversed etc... -> answer a copy >>> I hope the methods comments are clear. Does PBE tells about these >>> conventions? It would be a good thing. >>> >>> And I don't like to have mutexes in base library, the less we have, the >>> better. >>> If a user is going to modify the same object concurrently, he/she takes care >>> of mutual exclusion. >> >> Especially since locks don't compose. If you _really_ cared about >> accessing something concurrently, you'd share immutable data >> structures. > > I don't quite follow. Could you elaborate? Hopefully it's uncontroversial to assert that locks don't compose. If you only ever have one thread of execution, you don't have any concurrency issues, and locks serve no purpose. If you do have multiple threads of execution, then you have a few choices for sharing data: * you can use a lock around mutable data (but lock-using blocks of code don't compose, so you end up with loads of bugs or deadlocks or nests of locks, or all of the above) * you can share a _copy_ of data. In the latter case, you can share an actual copy, or share a pointer to a structure that can't change. If it can't change, you can't have a reader accidentally reading something from a structure halfway through the writer writing to it. Sharing some immutable chunk of state lets you save the memory taken up by a copy, but also prevents all the race condition things you usually get with shared mutable state. frank >> frank >> >>> 2013/9/23 Max Leske <maxleske(a)gmail.com> >>>> >>>> Sven suggested posting this on the list for discussion, so here you go: >>>> >>>> Maybe this should be discussed on the list, your are going to break API. >>>> >>>> Note that there is also #sort and #sorted with similar copy behavior. >>>> >>>> Also, I am not sure that basic operations should use mutexes to protect >>>> themselves by default: there is a cost when you are a single threaded user. >>>> Even in Java there are synchronized and non-synchronized versions of >>>> collections. IMHO, the protection should happen in your app, and basic >>>> collections do not have to be thread safe. >>>> >>>> Sven >>>> >>>> #shuffle does not use Collection>>mutexForPicking as other users of >>>> #randomForPicking demonstrate. This can lead to race conditions (found in >>>> our application). >>>> >>>> In addition, there are now #shuffle, #shuffled, #shuffleBy: and >>>> #shuffledBy: where #shuffled and #shuffledBy: shuffle a copy and answers >>>> that. This is very confusing. >>>> >>>> I propose a fix where #shuffled and #shuffledBy: are renamed to >>>> #copyShuffle and #copyShuffledBy: and moved to the "copying" protocol. >>>> #shuffle and #copyShuffle will use the mutex to prevent race conditions. >>> >>> >> > >

Sept. 23, 2013