April 2018 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Projects using Magritte meta models

by Sean P. DeNigris

Rafael Luque wrote > gitlab://â¦ ZnUrl>>enforceKnownScheme I think you have to do `Iceberg enableMetacelloIntegration: true.` first to get gitlab:// URLs to workâ¦ ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

April 25, 2018

Re: [Pharo-users] Right repo for TaskIt and features?

by Andrew Glynn

What about using VertStix for remote execution? AndrewÂ On Tue, 2018-04-24 at 15:31 +0000, Santiago Bragagnolo wrote: > > > On Tue, 24 Apr 2018 at 16:18 Holger Freyther <holger(a)freyther.de> > wrote: > > > > > > > On 24. Apr 2018, at 20:16, Santiago Bragagnolo > > o(a)gmail.com> wrote: > > >Â > > > Hi Holger!Â > > > I respond in bold > > > > hehe. And in the reply I am back to non rich text. Let me see if I > > quote it correctly. > hahahaha, non rich?, how come?Â I will still bolding hoping that if > you need to ensure the content you will check in a rich text client > :DÂ > > > > > > > >Â > > >Â > > >Â > > > On Tue, 24 Apr 2018 at 12:00 Holger Freyther <holger(a)freyther.de> > > wrote: > > > Hey! > > >Â > > > > > > > I wondered if somebody thought of remote task execution? > > >Â > > > *If you mean something else, I would need more information :). > > > > > > > > > > > When you do [ action ] schedule / [ action ] future, both created > > tasks are scheduled into the default runner. The default runner is > > a working pool with a default 'poolSizeMax' on 4, meaning, limit 4 > > processes working over the tasks. (this is a dynamic configuration, > > you can change it byÂ > > > TKTConfiguration runner poolMaxSize: 20. ) > > > > Yes. But with more work than the workers can handle the queue will > > grow. Which means the (median/max) latency of the system will > > monotonically increase.. to the point of the entire system failing > > (tasks handled after the external deadlines expired, effectively no > > work being done). > > Normally the worker pool adjust to the minimal needed workers (there > is a watch dog checking how much idle processes are there, or more > workers are needed, and ensuring to spawn or stop process regarding > to the state).Â > So, the number poolMaxSize is just a maximal limit. This limit should > be set for ensuring that the tasks that are running concurrently are > not incurring into too much resource consumption or into too much > overhead leading to kind of trashing.Â > I am not really friend of setting only a number for such a complex > problematic, but so far is the only approach I found that it does not > lead to a complex design. If you have better ideas to discuss on this > subject, i am completely open. (the same to deal with priorities by > general system understanding rather than absolute numbers)Â > > Â > > > > For network connected systems I like to think in terms of "back > > pressure" (not read more from the socket than the image can handle, > > eventually leading to the TCP window shrinking) and one way of > > doing it is to have bounded queues (and/or sleep when scheduling > > work). > > > > I can see multiple parts of a solution (and they have different > > benefits and issues): > > > > * Be able to attach a deadline to a task (e.g. see context.Context > > in go) > > * Be able to have a "blocking until queue is less than X elements" > > scheduleÂ (but that is difficult as one task might be scheduled > > during the >>#value of another task). > > > I wouldn't mind to have a second type of queue with this behaviour, > with a mean of configuration for setting one or other queue with it's > specific management encapsulated. > > Personally, in my domains of usage ( crawling, querying and in > sensor/actuator) i personally wouldn't use it. But I suppose you have > a better domain for this case. It would be good to discuss it to have > a better understanding of the need.Â > Â > > > > > > > Are there ideas how to add remote task scheduling? Maybe use > > Seamless for it? > > > Since you speak about seamless here, i suppose two different > > images, doesn't matter where.Â > > > It's not a bad idea to go seamless, but i did not go through the > > first restriction of remote executions (if the remote image can or > > not execute the task and if both images share the same semantic for > > the execution), then i did not yet checked on which communication > > platform to use for itÂ > > > > Right it would need to be homogeneous images (and care taken that > > the external interface remains similar enough). > I would like to understand a bit better what you are trying to do.Â > I have the hunch that you are looking for a multiple images solution, > for load balance in between images. TaskIT is mean to plan tasks into > process, regarding to the local image needs. You seems to need > something for planning tasks of a general system, beyond one image, > and maybe taking in care a process / network topology. > > If it's more that side, we should discuss in what extensions we can > do into taskit to be suitable of usage in this case, but surely I > would be inclined to do a higher level framework or even middleware > that uses taskit, than to add all those complexities in taskit. The > good news is that i may be needing something similar, so I will be > able to help there.Â > > > > > > > > > Have workers connect to the scheduler? Other ideas? > > > what do you mean by connection to the scheduler? The workers we > > use do not know their pools, if that is what you mean.Â > > > > Let's assume scheduling a task is simple (read something from a > > socket) but the computation is expensive (database look-up, math, > > etc). Hopefully one will reach the point where one image can > > schedule more tasks than a single worker image can handle. At that > > point it could be neat to scale by just starting another image. By > > inverting the launch order (workers connect to the scheduler) > > scaling can become more easy. > hahaha, now i re-read this paragraph and is not a hunch anymore. You > are looking for multiple images :).Â > > > Â > > holger > > > > > > > >

April 24, 2018

Re: [Pharo-users] Right repo for TaskIt and features?

by Santiago Bragagnolo

On Tue, 24 Apr 2018 at 16:18 Holger Freyther <holger(a)freyther.de> wrote: > > > > On 24. Apr 2018, at 20:16, Santiago Bragagnolo < > santiagobragagnolo(a)gmail.com> wrote: > > > > Hi Holger! > > I respond in bold > > hehe. And in the reply I am back to non rich text. Let me see if I quote > it correctly. > *hahahaha, non rich?, how come? I will still bolding hoping that if you need to ensure the content you will check in a rich text client :D * > > > > > > > > > On Tue, 24 Apr 2018 at 12:00 Holger Freyther <holger(a)freyther.de> wrote: > > Hey! > > > > > > I wondered if somebody thought of remote task execution? > > > > *If you mean something else, I would need more information :). > > > > > > When you do [ action ] schedule / [ action ] future, both created tasks > are scheduled into the default runner. The default runner is a working pool > with a default 'poolSizeMax' on 4, meaning, limit 4 processes working over > the tasks. (this is a dynamic configuration, you can change it by > > TKTConfiguration runner poolMaxSize: 20. ) > > Yes. But with more work than the workers can handle the queue will grow. > Which means the (median/max) latency of the system will monotonically > increase.. to the point of the entire system failing (tasks handled after > the external deadlines expired, effectively no work being done). *Normally the worker pool adjust to the minimal needed workers (there is a watch dog checking how much idle processes are there, or more workers are needed, and ensuring to spawn or stop process regarding to the state). * *So, the number poolMaxSize is just a maximal limit. This limit should be set for ensuring that the tasks that are running concurrently are not incurring into too much resource consumption or into too much overhead leading to kind of trashing. * *I am not really friend of setting only a number for such a complex problematic, but so far is the only approach I found that it does not lead to a complex design. If you have better ideas to discuss on this subject, i am completely open. (the same to deal with priorities by general system understanding rather than absolute numbers) * > > For network connected systems I like to think in terms of "back pressure" > (not read more from the socket than the image can handle, eventually > leading to the TCP window shrinking) and one way of doing it is to have > bounded queues (and/or sleep when scheduling work). > > I can see multiple parts of a solution (and they have different benefits > and issues): > > * Be able to attach a deadline to a task (e.g. see context.Context in go) > * Be able to have a "blocking until queue is less than X elements" > schedule (but that is difficult as one task might be scheduled during the > >>#value of another task). > > *I wouldn't mind to have a second type of queue with this behaviour, with a mean of configuration for setting one or other queue with it's specific management encapsulated.* *Personally, in my domains of usage ( crawling, querying and in sensor/actuator) i personally wouldn't use it. But I suppose you have a better domain for this case. It would be good to discuss it to have a better understanding of the need. * > > > > Are there ideas how to add remote task scheduling? Maybe use Seamless > for it? > > Since you speak about seamless here, i suppose two different images, > doesn't matter where. > > It's not a bad idea to go seamless, but i did not go through the first > restriction of remote executions (if the remote image can or not execute > the task and if both images share the same semantic for the execution), > then i did not yet checked on which communication platform to use for it > > Right it would need to be homogeneous images (and care taken that the > external interface remains similar enough). > *I would like to understand a bit better what you are trying to do. * *I have the hunch that you are looking for a multiple images solution, for load balance in between images. TaskIT is mean to plan tasks into process, regarding to the local image needs. You seems to need something for planning tasks of a general system, beyond one image, and maybe taking in care a process / network topology.* *If it's more that side, we should discuss in what extensions we can do into taskit to be suitable of usage in this case, but surely I would be inclined to do a higher level framework or even middleware that uses taskit, than to add all those complexities in taskit. The good news is that i may be needing something similar, so I will be able to help there. * > > > Have workers connect to the scheduler? Other ideas? > > what do you mean by connection to the scheduler? The workers we use do > not know their pools, if that is what you mean. > > Let's assume scheduling a task is simple (read something from a socket) > but the computation is expensive (database look-up, math, etc). Hopefully > one will reach the point where one image can schedule more tasks than a > single worker image can handle. At that point it could be neat to scale by > just starting another image. By inverting the launch order (workers connect > to the scheduler) scaling can become more easy. > *hahaha, now i re-read this paragraph and is not a hunch anymore. You are looking for multiple images :). * > holger > > > >

April 24, 2018

Re: [Pharo-users] get.pharo.org broken?

by Paul DeBruicker

Ahh. Thanks Sven. I was just copying/pasting from the <code> block on https://get.pharo.org/64 and it didn't work. Adding the trailing slash like you suggest fixes it. Sven Van Caekenberghe-2 wrote > curl -L https://get.pharo.org/64 > > or https://get.pharo.org/64/ > >> On 24 Apr 2018, at 16:27, PAUL DEBRUICKER < > pdebruic@ > > wrote: >> >> Hi - >> >> >> curl https://get.pharo.org/64 | bash >> >> >> gives an error on MacOS X: >> >> >> >> paul@a:~/pharo/maf$ curl https://get.pharo.org/64 | bash >> % Total % Received % Xferd Average Speed Time Time Time >> Current >> Dload Upload Total Spent Left >> Speed >> 100 237 100 237 0 0 315 0 --:--:-- --:--:-- --:--:-- >> 315 >> bash: line 1: syntax error near unexpected token `newline' >> bash: line 1: `<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML >> 2.0//EN">' >> >> >> >> am I doing something wrong or is it broken for other people too? >> >> Thanks >> >> Paul -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

April 24, 2018

Re: [Pharo-users] Projects using Magritte meta models

by Rafael Luque

Stef, I've sent a pull request via GitHub with my reviews: https://github.com/SquareBracketAssociates/Booklet-Magritte/pull/2 2018-04-21 8:49 GMT+00:00 Stephane Ducasse <stepharo.self(a)gmail.com>: > Cool I will have a look when I go back to Magritte > Rafael if you see mistake in the booklet please report them to me. > I will do a pass in a couple of weeks I hope > > On Sat, Apr 21, 2018 at 4:07 AM, Sean P. DeNigris <sean(a)clipperadams.com> > wrote: > > Rafael Luque wrote > >> I wonder if there are other relevant projects I could study to discover > >> other possible > >> uses cases of Magritte. > > > > I use it in nearly all my personal projects, almost always via Morphic, > not > > Seaside. Here is a public one you can have a look at: > > https://github.com/seandenigris/Small-World > > > > Load in Pharo 6.1 via: > > Metacello new > > baseline: 'SmallWorld'; > > repository: 'github://seandenigris/ > SmallWorld:master/repository'; > > onConflict: [ :ex | ex allow ]; > > load. > > > > Browse senders of magritteDescription for classes prefixed with "Small". > > > > > > > > ----- > > Cheers, > > Sean > > -- > > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > > > >

April 24, 2018

Re: [Pharo-users] get.pharo.org broken?

by Sven Van Caekenberghe

curl -L https://get.pharo.org/64 or https://get.pharo.org/64/ > On 24 Apr 2018, at 16:27, PAUL DEBRUICKER <pdebruic(a)gmail.com> wrote: > > Hi - > > > curl https://get.pharo.org/64 | bash > > > gives an error on MacOS X: > > > > paul@a:~/pharo/maf$ curl https://get.pharo.org/64 | bash > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 100 237 100 237 0 0 315 0 --:--:-- --:--:-- --:--:-- 315 > bash: line 1: syntax error near unexpected token `newline' > bash: line 1: `<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">' > > > > am I doing something wrong or is it broken for other people too? > > Thanks > > Paul

April 24, 2018

get.pharo.org broken?

by PAUL DEBRUICKER

Hi - curl https://get.pharo.org/64 | bash gives an error on MacOS X: paul@a:~/pharo/maf$ curl https://get.pharo.org/64 | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 237 100 237 0 0 315 0 --:--:-- --:--:-- --:--:-- 315 bash: line 1: syntax error near unexpected token `newline' bash: line 1: `<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">' am I doing something wrong or is it broken for other people too? Thanks Paul

April 24, 2018

Re: [Pharo-users] Right repo for TaskIt and features?

by Holger Freyther

> On 24. Apr 2018, at 20:16, Santiago Bragagnolo <santiagobragagnolo(a)gmail.com> wrote: > > Hi Holger! > I respond in bold hehe. And in the reply I am back to non rich text. Let me see if I quote it correctly. > > > > On Tue, 24 Apr 2018 at 12:00 Holger Freyther <holger(a)freyther.de> wrote: > Hey! > > I wondered if somebody thought of remote task execution? > > *If you mean something else, I would need more information :). > When you do [ action ] schedule / [ action ] future, both created tasks are scheduled into the default runner. The default runner is a working pool with a default 'poolSizeMax' on 4, meaning, limit 4 processes working over the tasks. (this is a dynamic configuration, you can change it by > TKTConfiguration runner poolMaxSize: 20. ) Yes. But with more work than the workers can handle the queue will grow. Which means the (median/max) latency of the system will monotonically increase.. to the point of the entire system failing (tasks handled after the external deadlines expired, effectively no work being done). For network connected systems I like to think in terms of "back pressure" (not read more from the socket than the image can handle, eventually leading to the TCP window shrinking) and one way of doing it is to have bounded queues (and/or sleep when scheduling work). I can see multiple parts of a solution (and they have different benefits and issues): * Be able to attach a deadline to a task (e.g. see context.Context in go) * Be able to have a "blocking until queue is less than X elements" schedule (but that is difficult as one task might be scheduled during the >>#value of another task). > Are there ideas how to add remote task scheduling? Maybe use Seamless for it? > Since you speak about seamless here, i suppose two different images, doesn't matter where. > It's not a bad idea to go seamless, but i did not go through the first restriction of remote executions (if the remote image can or not execute the task and if both images share the same semantic for the execution), then i did not yet checked on which communication platform to use for it Right it would need to be homogeneous images (and care taken that the external interface remains similar enough). > Have workers connect to the scheduler? Other ideas? > what do you mean by connection to the scheduler? The workers we use do not know their pools, if that is what you mean. Let's assume scheduling a task is simple (read something from a socket) but the computation is expensive (database look-up, math, etc). Hopefully one will reach the point where one image can schedule more tasks than a single worker image can handle. At that point it could be neat to scale by just starting another image. By inverting the launch order (workers connect to the scheduler) scaling can become more easy. holger

April 24, 2018

Re: [Pharo-users] Projects using Magritte meta models

by Rafael Luque

Hi Sean, Thank you for your answer. I'm looking forward to read this project's code, however, when I try to load it following your instructions I get a ZnUnknownScheme when it is loading files from gitlab://SeanDeNigris/gitlab-smalltalk-ci:master/src: ZnUrl>>enforceKnownScheme ZnRequestLine>>uri: ZnRequest>>url: ZnClient>>url: [ client := self httpClient. client ifFail: [ :exception | (exception className beginsWith: 'Zn') ifTrue: [ MCRepositoryError signal: 'Could not access ' , self location , ': ' , exception printString ] ifFalse: [ exception pass ] ]; url: self locationWithTrailingSlash; queryAt: 'C' put: 'M;O=D'; "legacy that some servers maybe expect" get. Thank you! 2018-04-21 2:07 GMT+00:00 Sean P. DeNigris <sean(a)clipperadams.com>: > Rafael Luque wrote > > I wonder if there are other relevant projects I could study to discover > > other possible > > uses cases of Magritte. > > I use it in nearly all my personal projects, almost always via Morphic, not > Seaside. Here is a public one you can have a look at: > https://github.com/seandenigris/Small-World > > Load in Pharo 6.1 via: > Metacello new > baseline: 'SmallWorld'; > repository: 'github://seandenigris/SmallWorld:master/repository'; > onConflict: [ :ex | ex allow ]; > load. > > Browse senders of magritteDescription for classes prefixed with "Small". > > > > ----- > Cheers, > Sean > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >

April 24, 2018

Re: [Pharo-users] UFFI and autoRelease

by Esteban Lorenzano

hi, there is nothing like that and I do not recommend messing with the registry in general. butâ¦ you can always extend the classes for your use, is around FFIExternalResourceManager. cheers, Esteban > On 24 Apr 2018, at 14:48, Serge Stinckwich <serge.stinckwich(a)gmail.com> wrote: > > > I'm using autoRelease on some FFIOpaqueObject instances. > > I need to test some behavior when I delete explicitly one of these objects. âHow to remove these objects from the finalization list, in order they are not freed two times. > > Something like ignoreFinalization ? > > -- > Serge Stinckwich > UMI UMMISCO 209 (SU/IRD/UY1) > "Programs must be written for people to read, and only incidentally for machines to execute." > http://www.doesnotunderstand.org/ <http://www.doesnotunderstand.org/>

April 24, 2018