December 2014 - Pharo-dev - lists.pharo.org

Announcements & inter-thread communication

by Ben Coman

_BACKGROUND_ In relation to... https://pharo.fogbugz.com/default.asp?13182 "Refactor MCFileRepositoryInspector >> versionSearchAccept:" I am reviewing... https://pharo.fogbugz.com/default.asp?13007 "Cannot integrate a slice with monticello (red square in the UI)" and in general considering how Monticello uses multiple threads to allow you to type in the filter while its downloading info from the remote repository (which would be good to have more of this sort thing). Looking back at build 30814 just prior to slice 13007 being integrated, I notice that MCFileRepositoryInspector>>setRepository:workingCopy: [1] forks and sends #refresh, which sends #changed: #packageList. and MCFileRepositoryInspector>>packageSearchAccept: [2] also forks and sends #changed: #packageList So there were three threads (including UI [3]) modifying object state, which was compounded by #packageList method doing a lot of work. Issue 13007 worked around this by moving the work of #packageList out to #packageListUpdate, and moved the #changed: sends from [1] & [2] also out into #packageListUpdate (and wrapped in a #defer:). _QUESTIONS_ Now I am considering doing the same for issue 13182 #versionSearchAccept:, and considering Pharo's goal is to replace all Dependents/#changed: mechanisms with Announcements. 1. So I am wondering if that would be a better direction to address this? Then I wonder about how Announcements interact with threads. I see that SubscriptionRegistry>>deliver: #protect's the generation of /interestedSubscriptions/, but the actual announcement delivery occurs in the open, in AnnoucemenSubscription>>delvier:, in the sender's thread. So for an example... action := [ :it | self inform: 'removing child'.]. announcer on: RemoveChild do: action. /action/ might be executed from different concurrent threads. Now you might say that the programmer should know enough about the structure of their threaded code to wrap the action in as critical section, but Announcements are a DECOUPLING mechanism, and maybe a THIRD PARTY library subscribes to an Announcement that later someone else wraps in a fork, and no-one notices possible race conditions created. 2. So, is there something reasonable we can do to make the common/easier/safer single-threaded case the default, and make people work a bit harder to shoot themselves in the foot with multiple threads? We could introduce Announcer>>onOtherThread:do: with the purpose of making use of inter-thread announcements more explicit in a self documenting way. Announcements accidentally passed between threads to Announcer>>on:do: would generate an exception advising "use #onOtherThread:do: " with the reasonable expectation that use of #onOtherThread:do: would follow the pattern... announcer onOtherThread: RemoveChild do: [ :it | myMutex critical: [ self inform: 'removing child' ]. ]. With this arrangement, a third party subscribed to someone else's announcement, that is quietly changed to be wrapped by forked thread, the third party's code now fails EARLY and OBVIOUS, rather than letting subtle race conditions occur at random times. _PROPOSED IMPLEMENTATION_ "------------------" Announcer>>subscribe: anAnnouncementClass do: aValuable ^ registry add: ( AnnouncementSubscription new announcer: self; process: Processor activeProcess announcementClass: anAnnouncementClass; valuable: aValuable) "------------------" AnnouncementSubscription>>deliver: anAnnouncement process ~= Processor activeProcess ifTrue: [ self error: [ 'Cross thread announcements should subscribe using #onOtherThread:do: ' ]. ^ (self handlesAnnouncement: anAnnouncement ) ifTrue: [ [action cull: anAnnouncement cull: announcer] on: UnhandledError fork: [:ex | ex pass ]] "------------------" Announcer>>threadedSubscribe: anAnnouncementClass do: aValuable ^ registry add: ( ThreadedAnnouncementSubscription new announcer: self; process: Processor activeProcess announcementClass: anAnnouncementClass; valuable: aValuable) "------------------" where ThreadedAnnouncementSubscription essentially functions the same as the existing AnnoucementSubscription .

Dec. 6, 2014

Re: [Pharo-dev] GetOpt

by Eliot Miranda

On Wed, Dec 3, 2014 at 5:29 AM, Ben Coman <btc(a)openinworld.com> wrote: > Eliot Miranda wrote: > >> One if these, which Ben Coman's recent changes above my 64-bit >> microsecond clock should have fixed is a spin loop of up to a second to >> synchronize the second and millisecond clocks. >> > > Is this in the image or VM ? I didn't notice this. In the image. See secondsWhenClockTicks in Pharo 2 and Pharo 3. Its also in Squeak :-(. It is called from DateAndTime class>>#startUp: via DateAndTime class>>#initializeOffsets. It throws away on average half a second on every launch. Looks like it is not used in Pharo 4. I found these either by profiling (clock sync; I can give you a change set >> that allows profiling across a snapshot, quit, resume) or using the VM >> simulator (GC to check for finalization facilities). >> > > That would be interesting to try. > > cheers -ben > > -- best, Eliot

Dec. 6, 2014

Re: [Pharo-dev] Monticello and Fuel

by Mariano Martinez Peck

On Sat, Dec 6, 2014 at 12:37 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > On Thu, Dec 4, 2014 at 11:35 PM, Max Leske <maxleske(a)gmail.com> wrote: > >> >> > On 05 Dec 2014, at 08:02, Norbert Hartl <norbert(a)hartl.name> wrote: >> > >> > >> > >> > >> >> Am 04.12.2014 um 23:31 schrieb Thierry Goubier < >> thierry.goubier(a)gmail.com>: >> >> >> >> Hi all, >> >> >> >> I'm just wondering. >> >> >> >> Would it work to have a package format based on Fuel? >> >> >> > I doubt it would work cross platform. I don't know how fuel serializes >> WideString, LargePositiveInteger, BoxedFloat64. These differ between >> smalltalk platforms. The source as string solves that. Strings are written >> as unicode string and numbers as certain number format etc. >> >> True. Cross dialect loading isnât something we encourage to do with Fuel. >> >> @Thierry >> You said something about partial loading: that will never be possible >> with Fuel because of its pickle format. To select any partial graph you >> have to first read the entire file and rebuild the graph first (ok, you >> donât strictly have to build the graph but you still need to read the >> entire file). >> > > Hi Max, you misunderstand what partial loading means in this context. I > designed and implemented partial loading for the Parcel system in VW, and > Fuel is essentially a clean reimplementation of parcels. The idea is that > one does indeed read the entire object graph, but then only the parts of > the graph that mate with the current class hierarchy are installed, and the > bits that don't fit are stored for later. Why? So that a component can > define extensions on classes in components that may not be present. Why? > So that one can maintain a single logical component in a single package > instead of decomposing it into independently loadable fragments. > > Lets take an example like Fuel itself. This may have specialised > marshalling and unmarshalling extensions defined on may classes, some of > which may be to do with the GUI. If we have partial loading we can load > Fuel into a headless image. The extensions on the GUI classes will simply > not be installed, *until* we load the GUI. Hence Fuel does not have to be > decomposed every time we factor the system into subcomponents. Without > partial loading Fuel must be decomposed into a series of fragments so that > it can load that part that fits into the headless base, and we have to > manage teh dependency to ensure the fragment that loads against the GUI is > loaded when the GUI is loaded. Worse still, if we cut the GUI into two > (e.g. development vs deployment tools) we have to visit the Fuel GUI > component (and potentially many other component) and decompose it into two > pieces. In practice this is extremely costly to maintain, verging on > chaos. With partial loading things are simple. > > Perhaps I should have called it partial installation, but you get the > idea. > > Eliot, I do agree with all you said. But let me say a few bits please to reduce misunderstandings. We consider Fuel itself completely isolated from what is exporting/importing a package. In fact, as you know, by default Fuel serializes classes and methods "globally". So...all this discussion should actually be at another level... say Tanker level (or whatever tool we build on top of Fuel to manage packages). This is a difference to Parcels, which even if it was able to also serialize regular objects graphs, it was more tight to packages. In addition, as you said, Max is right in the sense that at FUEL level, the whole graph would have to be materialized. IN THE CASE OF TANKER, or whatever tool for packages, we may NOT install all parts, as you said. But they will indeed be materialized. So... to sum out, I think the clearer way of saying it is: - Fuel must have to materialize the whole graph anyway (at least as it is now). - Tanker or whatever tool for managing packages will also have to materialize all, but should be able to "install" only a part of it. The example you said about Fuel is very clear. Soo... I think we all agree. Just that we use a different naming. Best regards, > >> > >> > Norbert >> > >> >> Would that make loading faster? >> >> >> >> Does it already exist? >> >> >> >> Thanks, >> >> >> >> Thierry >> > >> >> >> > > > -- > best, > Eliot > -- Mariano http://marianopeck.wordpress.com

Dec. 6, 2014

Re: [Pharo-dev] Pharo+Git cons?

by Eliot Miranda

On Fri, Dec 5, 2014 at 7:31 PM, stepharo <stepharo(a)free.fr> wrote: > Hi eliot > > I would like to check the ressources addition to MC. > In the past I tried to harvest the refactorings did by chris in MC but I > failed because of lack of patience and time. > Now can you tell a word about the version server? > I think you should ask Chris. I'm just a happy consumer. I don't know how it works, except that there's an automatically updated database accessible via a server than responds to the version info queries from images. I've asked Chris whether that server could be replicated on one's own machine (so that e.g. one can work without a network) and he says that,, yes of course this can be done. So this looks to me like a good scheme. But Chris is it's designer and implementor. Ask him... -- best, Eliot

Dec. 6, 2014

Re: [Pharo-dev] Monticello and Fuel

by Eliot Miranda

On Thu, Dec 4, 2014 at 11:35 PM, Max Leske <maxleske(a)gmail.com> wrote: > > > On 05 Dec 2014, at 08:02, Norbert Hartl <norbert(a)hartl.name> wrote: > > > > > > > > > >> Am 04.12.2014 um 23:31 schrieb Thierry Goubier < > thierry.goubier(a)gmail.com>: > >> > >> Hi all, > >> > >> I'm just wondering. > >> > >> Would it work to have a package format based on Fuel? > >> > > I doubt it would work cross platform. I don't know how fuel serializes > WideString, LargePositiveInteger, BoxedFloat64. These differ between > smalltalk platforms. The source as string solves that. Strings are written > as unicode string and numbers as certain number format etc. > > True. Cross dialect loading isnât something we encourage to do with Fuel. > > @Thierry > You said something about partial loading: that will never be possible with > Fuel because of its pickle format. To select any partial graph you have to > first read the entire file and rebuild the graph first (ok, you donât > strictly have to build the graph but you still need to read the entire > file). > Hi Max, you misunderstand what partial loading means in this context. I designed and implemented partial loading for the Parcel system in VW, and Fuel is essentially a clean reimplementation of parcels. The idea is that one does indeed read the entire object graph, but then only the parts of the graph that mate with the current class hierarchy are installed, and the bits that don't fit are stored for later. Why? So that a component can define extensions on classes in components that may not be present. Why? So that one can maintain a single logical component in a single package instead of decomposing it into independently loadable fragments. Lets take an example like Fuel itself. This may have specialised marshalling and unmarshalling extensions defined on may classes, some of which may be to do with the GUI. If we have partial loading we can load Fuel into a headless image. The extensions on the GUI classes will simply not be installed, *until* we load the GUI. Hence Fuel does not have to be decomposed every time we factor the system into subcomponents. Without partial loading Fuel must be decomposed into a series of fragments so that it can load that part that fits into the headless base, and we have to manage teh dependency to ensure the fragment that loads against the GUI is loaded when the GUI is loaded. Worse still, if we cut the GUI into two (e.g. development vs deployment tools) we have to visit the Fuel GUI component (and potentially many other component) and decompose it into two pieces. In practice this is extremely costly to maintain, verging on chaos. With partial loading things are simple. Perhaps I should have called it partial installation, but you get the idea. > > > > Norbert > > > >> Would that make loading faster? > >> > >> Does it already exist? > >> > >> Thanks, > >> > >> Thierry > > > > > -- best, Eliot

Dec. 6, 2014

Re: [Pharo-dev] Pharo+Git cons?

by stepharo

Hi eliot I would like to check the ressources addition to MC. In the past I tried to harvest the refactorings did by chris in MC but I failed because of lack of patience and time. Now can you tell a word about the version server? Stef

Dec. 6, 2014

Re: [Pharo-dev] Slow TreeListMorph

by Nicolas Cellier

2014-12-06 0:27 GMT+01:00 Nicolai Hess <nicolaihess(a)web.de>: > Anyone knows for what this is used: > > TreeListMorph>> > addSubmorphsAfter: parentMorph fromCollection: aCollection allowSorting: > sortBoolean > "Update the font on each morph." > > |answer| > answer := super addSubmorphsAfter: parentMorph fromCollection: > aCollection allowSorting: sortBoolean. > > "---- THIS TAKES TIME ----" > self scroller submorphsDo: [:i | > i > font: self font; > extent: i minWidth @ i minHeight]. > "---- XXXXX ----" > > ^answer > > > setting the font and extent on all scroller submorphs is extremly > slow. > Try to merge a package with many changes, it takes up to several minutes > until > the DiffMorph shows up. > Removing the above lines makes the DiffMorph much faster and I can not see > any visual difference. > > nicolai > > Yeah, try to merge in Squeak, it's quite fast... Polymorph is good looking, but if it's superslow, then it totally misses the point. I used to disallow this Preference (err... Setting) until it disappeared... I just don't understand how people can stand it. I can't, merge is so essential... Or I understand a bit more why some git tools are perceived as SO superior ;) But if you found the major contributor, it's a great news !

Dec. 6, 2014

Re: [Pharo-dev] Pharo+Git cons?

by Eliot Miranda

On Fri, Dec 5, 2014 at 5:02 AM, Ben Coman <btc(a)openinworld.com> wrote: > Thierry Goubier wrote: > >> >> 2014-12-04 23:59 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com >> <mailto:eliot.miranda@gmail.com>>: >> >> I'm also delighted that work like Chris Muller's version server is >> out there. This is very nicely integrated to allow me to find out >> which package versions contain versions of a particular method or >> class definition. >> >> >> I know that this version server exists, and I hoped that you would point >> it out, because I believe it is significant. Even if it is not usable in >> many professional settings, such as mine. >> > > My google fu didn't turn up anything on this in the first few pages. Could > you provide a pointer? > http://forum.world.st/ANN-New-source-squeak-org-prototype-td4707006.html http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-September/17306… -- cheers, Eliot

Dec. 6, 2014

Slow TreeListMorph

by Nicolai Hess

Anyone knows for what this is used: TreeListMorph>> addSubmorphsAfter: parentMorph fromCollection: aCollection allowSorting: sortBoolean "Update the font on each morph." |answer| answer := super addSubmorphsAfter: parentMorph fromCollection: aCollection allowSorting: sortBoolean. "---- THIS TAKES TIME ----" self scroller submorphsDo: [:i | i font: self font; extent: i minWidth @ i minHeight]. "---- XXXXX ----" ^answer setting the font and extent on all scroller submorphs is extremly slow. Try to merge a package with many changes, it takes up to several minutes until the DiffMorph shows up. Removing the above lines makes the DiffMorph much faster and I can not see any visual difference. nicolai

Dec. 5, 2014

TextModel>>#aboutToStyle:

by Sean P. DeNigris

While we're creating an API, it would be great to make #aboutToStyle: something more intuitive. Somehow this questionable (always confusing to me anyway) bit leaked out from the Morphic implementation underneath. ----- Cheers, Sean -- View this message in context: http://forum.world.st/TextModel-aboutToStyle-tp4794332.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Dec. 5, 2014