- Pharo-dev - lists.pharo.org

Slides of my Lecture this morning..

by Marcus Denker

Hi, I am just visiting Jordi Delgado in Barcelona and I gave a lecture in his course this morning. âAdvanced Reflection in Pharoâ, this is meant to be a lecture after an introduction to standard Smalltalk reflection. SlideShare http://www.slideshare.net/MarcusDenker/lecture-27685035 PDF http://marcusdenker.de/talks/13BarcelonaLecture/ Marcus

Oct. 29, 2013

Re: [Pharo-dev] Tests with multiple processes

by Norbert Hartl

Am 29.10.2013 um 02:08 schrieb David T. Lewis <lewis(a)mail.msen.com>: > I have some experience with this, not related to Smalltalk but maybe > still relevant. I have worked with systems that were designed around > networked message passing, implemented on platforms like MS-DOS with time > slicing kernels, Series I, OS/2, etc. These systems were designed to operate > with asynchronous message passing protocols. The asynchronous designs were > intended to support primitive OS environments, including MS-DOS with > software interrupts for multitasking. > > These are production-critical manufacturing applications, and in the course > of modernizing them I have reimplemented the network protocols in Java and > moved them to new computing platforms. Along the way I have found that the > asynchronous designs could always be replaced by bidirectional send-receive > transactions running in the context of a thread or process. Even the systems > that appeared to be most complex have turned out to work perfectly well > when modeled as synchronous send-receive transactions on a bidirectional > communication channel. The software is much simpler to understand and debug > when structured in this manner. > > My conclusion is that designing around asynchronous, loosely coupled > channels sounds like a good idea, but in practice it leads to unnecessary > complexity, and to systems that are difficult to debug and support. > I agree. That was the reason for me to look out for a simpler approach. At least for testing. Can you elaborate on the âbidirectional send-receive transactionâ. Iâm not sure I understand what happens there. I explained my case a bit more in my response to Igor. thanks, Norbert > $0.02 > > Dave > > > On Tue, Oct 29, 2013 at 01:19:17AM +0100, Igor Stasenko wrote: >> Maybe i'm off track, but here my thoughts: >> >> - one big misconception (and most problems which arising from it), that >> people tend to use a bidirectional communication models instead of >> unidirectional. >> In bidirectional you tend to think as if you using same channel for sending >> and receiving data. >> The truth is, that it is two unidirectional channels, wired in the way so >> they look like single bidirectional channel. >> >> And what it means, that coupling sending and receiving activities in a >> single thread is wrong: >> - you should not wait for data only if you sent something >> - you should not expect that activity from other end is possible only >> after you _send_ something. >> >> If you treat connection as two unidirectional, loosely coupled channels, >> instead of single bidirectional one, >> you will find that in such setup you don't have many problems. >> >> You should definitely go for (2), e.g. you should decouple a >> transport/communication layer >> from your program/logic flow: if some process wants to send data and block >> until some response to >> that message arrived, you should do so. >> You can simply put that process on semaphore, and adding it to the list of >> 'awaiting answer' >> processes in your scheduler which runs in separate process to handle >> incoming data. >> Then you can have different strategies, what to do if >> a) data seems taking too long to arrive (signal timeout) >> b) connection lost >> c) etc.. >> >> ohh.. with smalltalk it is so easy to operate with those things (the fact >> the VM is green-threaded doesn't >> changes much). >> >> >> >> On 28 October 2013 23:52, Norbert Hartl <norbert(a)hartl.name> wrote: >> >>> >>> Am 28.10.2013 um 23:50 schrieb Norbert Hartl <norbert(a)hartl.name>: >>> >>> Are there any approaches to simulate coroutines in a single thread >>> environment or approaches to deal with multiple processes within one >>> process? >>> >>> >>> That should have been >>> >>> Are there any approaches to simulate coroutines in a single thread >>> environment or approaches to deal with multiple processes within one test? >>> >>> Norbert >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko. >

Oct. 29, 2013

Re: [Pharo-dev] Tests with multiple processes

by Norbert Hartl

Igor, it seems I did a terrible job explaining it. Comments inline! Am 29.10.2013 um 01:19 schrieb Igor Stasenko <siguctua(a)gmail.com>: > Maybe i'm off track, but here my thoughts: > > - one big misconception (and most problems which arising from it), that people tend to use a bidirectional communication models instead of unidirectional. > In bidirectional you tend to think as if you using same channel for sending and receiving data. > The truth is, that it is two unidirectional channels, wired in the way so they look like single bidirectional channel. > > And what it means, that coupling sending and receiving activities in a single thread is wrong: > - you should not wait for data only if you sent something > - you should not expect that activity from other end is possible only after you _send_ something. > > If you treat connection as two unidirectional, loosely coupled channels, instead of single bidirectional one, > you will find that in such setup you don't have many problems. I donât know what gave you the impression it could be that way but it isnât. There is a component called NetworkService that Iâm using to do sends and where Iâm registering for incoming messages. I was not talking about the architecture of the application but about ways how to test those setups. For this I have a TestNetworkService that has an instVar otherService. On connection time the NetworkServices set themself via a setter in the other service and a send is done like send: someData otherService doReceive: someData This is the place where I change the execution model for the tests. What would be normally something asynchronous is now synchronous with the mentioned side effect that a send with response will look like A send1 B receive1 B send2 A receive2 A return from receive2 B return from send2 B return from receive1 A return from send1 I really like the tests to be single threaded. It makes debugging a lot easier. So I was evaluating a few possibilities. As long as all of the calls in the above chain are tail calls it should be the same of the calls are nested or not. But that would be a restriction and Iâm not sure yet if it is a good one or not. The next approach I do at the moment is that e.g. a send call goes roughly like this p := [ a block for preparing send data ] s := [ a block that sends the data ] s value: p value What I do is to rearrange flow. At the moment where s would be evaluated it is instead made the sender of p and the sender of s will be the former sender of p. So everything that happens in p will return before s is executed. For my execution model this mimicks parallel behaviour. But now the receive/sends are nested and my simple model doesnât work. I could improve it but it seems like I need to tweak a tweak of a workaround that probably wasnât the right choice in the first place. What my experience showed me is that often it is a good way to remove a hack instead of hacking a hack. So Iâm thinking about if this is the point to let go :) > You should definitely go for (2), e.g. you should decouple a transport/communication layer > from your program/logic flow: if some process wants to send data and block until some response to > that message arrived, you should do so. > You can simply put that process on semaphore, and adding it to the list of 'awaiting answer' > processes in your scheduler which runs in separate process to handle incoming data. > Then you can have different strategies, what to do if > a) data seems taking too long to arrive (signal timeout) > b) connection lost > c) etc.. > > ohh.. with smalltalk it is so easy to operate with those things (the fact the VM is green-threaded doesn't > changes much). > As I said the layers are separate. So one idea was to keep it in multiple processes and use semaphores for orchestration. I need to think about that because I donât have an immediate idea how this should be done. I mean the dispatcher/scheduler does the orchestration of the processes but I would need to make use of this inside a test. Need to think about that. Norbert > > > On 28 October 2013 23:52, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 28.10.2013 um 23:50 schrieb Norbert Hartl <norbert(a)hartl.name>: > >> Are there any approaches to simulate coroutines in a single thread environment or approaches to deal with multiple processes within one process? > > That should have been > > Are there any approaches to simulate coroutines in a single thread environment or approaches to deal with multiple processes within one test? > > Norbert > > > > -- > Best regards, > Igor Stasenko.

Oct. 29, 2013

Re: [Pharo-dev] [pharo-project/pharo-core] a20ada: 30532

by Marcus Denker

On 29 Oct 2013, at 11:37, GitHub <noreply(a)github.com> wrote: > Branch: refs/heads/3.0 > Home: https://github.com/pharo-project/pharo-core > Commit: a20adab058f13ca9dff1c6cc5ee92be5b5fc3c9d > https://github.com/pharo-project/pharo-core/commit/a20adab058f13ca9dff1c6cc… The link above goes to page with diffs (can be easily missed). Marcus

Oct. 29, 2013

Re: [Pharo-dev] Hungry Pharo

by Nicolas Cellier

The conclusion is: more work is required on both sides, VM and image. 2013/10/29 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > > A greater than, say, 1% base CPU load from an otherwise idle application > would indicate that it's doing some busy waiting somewhere... in my > Activity Monitor, I don't see too many processes doing that, so just using > an ObjC framework doesn't seem to be the guilty part. By the way, very old > versions of VisualWorks used to be bad in that way, but not any more; so > there is something that can be done about it. > > if I'm correct there are some busy waiting process in the VM instead of an > event based + semaphore architecture. > We already removed some of the polling behavior at the image level but > there are probably more and there was a discussion in the > VM mailing list. Now I do not know the conclusion. > > > A large virtual memory working set wouldn't help either for power > efficiency, though with today's RAM sizes it's probably less important. > > > >

Oct. 29, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30532 Home: https://github.com/pharo-project/pharo-core

Oct. 29, 2013

[pharo-project/pharo-core] a20ada: 30532

by GitHub

Branch: refs/heads/3.0 Home: https://github.com/pharo-project/pharo-core Commit: a20adab058f13ca9dff1c6cc5ee92be5b5fc3c9d https://github.com/pharo-project/pharo-core/commit/a20adab058f13ca9dff1c6cc… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2013-10-29 (Tue, 29 Oct 2013) Changed paths: A Files.package/SourceFileArray.class/instance/source code management/getPreambleFrom_at_.st M Graphics-Fonts.package/StrikeFont.class/instance/displaying/displayMultiString_on_from_to_at_kern_baselineY_.st M Kernel.package/CompiledMethod.class/instance/source code management/getPreambleFrom_at_.st M Kernel.package/ExtendedNumberParser.class/instance/parsing-private/readNumberWithFractionPartNumberOfTrailingZeroInIntegerPart_.st M Kernel.package/ExtendedNumberParser.class/instance/parsing-private/readNumberWithoutIntegerPartOrNil.st M Kernel.package/SqNumberParser.class/instance/parsing-private/makeIntegerOrScaledInteger.st M Kernel.package/SqNumberParser.class/instance/parsing-private/readNumberWithFractionPartNumberOfTrailingZeroInIntegerPart_.st R Kernel.package/SqNumberParser.class/instance/parsing-private/readScale.st A Kernel.package/SqNumberParser.class/instance/parsing-private/readScale_.st M Kernel.package/SqNumberParser.class/instance/parsing-public/nextScaledDecimal.st M KernelTests.package/NumberParsingTest.class/instance/tests - Integer/testIntegerReadFrom.st A KernelTests.package/SqNumberParserTest.class/instance/tests - ScaledDecimal/testScaledDecimalWithoutExplicitScale.st A KernelTests.package/SqNumberParserTest.class/instance/tests - ScaledDecimal/testScaledIntegerWithoutExplicitScale.st M Ring-Core-Kernel.package/RGMethodDefinition.class/instance/source pointers/getPreambleFrom_at_.st R Ring-Core-Kernel.package/extension/SourceFileArray/instance/getPreambleFrom_at_.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - scripts/script187.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - updates/update30532.st M ScriptLoader30.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 30532 12031 One should not ask for the glyphInfo of a fallbackFont https://pharo.fogbugz.com/f/cases/12031 12029 ScaledDecimal with unspecified scale should auto-adjust their scale https://pharo.fogbugz.com/f/cases/12029 12023 Remove duplicated code in 3 implementors of #getPreambleFrom:at: https://pharo.fogbugz.com/f/cases/12023 http://files.pharo.org/image/30/30532.zip

Oct. 29, 2013

Re: [Pharo-dev] Tests with multiple processes

by Norbert Hartl

Am 29.10.2013 um 10:56 schrieb Frank Shearar <frank.shearar(a)gmail.com>: > If you're going to even think of using callbacks like they do in > Javascript, strongly reconsider. Especially so since the Javascript > community are abandoning callbacks in favour of promises, especially > of the A+ sort. > Well, callbacks are still used but managed by promise objects. Without building composites of callbacks the programming style will turn into âfire & forgetâ. With using this style of promises you get back a sort of sequential programming model while being asynchron or even parallel. IMHO that is the same thing we are facing from time to time. We like to have things happen in parallel while the programming model is still single threaded. > (Squeak has such Promises in the base image, and if anyone's > interested I can break them out into a Pharo-consumable library.) > That would be really great! I would appreciate that a lot. And btw. I was glad Iâve found your blog post about delimited continuations. Good work! Norbert > frank > > On 29 October 2013 00:11, Sebastian Sastre <sebastian(a)flowingconcept.com> wrote: >> you can do stuff like that, just not sync (which when it's about networks is a good idea) >> >> sounds like you should design things javascript style. >> >> I mean, like if "everything" is a callback reacting to something.. >> >> yourAPIClient sendCommand: aCommand do: aBlockHandlingResponse >> >> my 2 cents >> >> >> >> >> On Oct 28, 2013, at 8:50 PM, Norbert Hartl <norbert(a)hartl.name> wrote: >> >>> Iâm working on project that deals with server to server communication. For the test suite it is necessary to simulate the two ends of the communication within one test. The problem is that in a single threaded environment you make a send call SC from server A to B but the response of B will happen before SC returned. That makes every state change the happens in A after SC bogus from a call flow perspective. >>> >>> There are two approaches that have their own problems. >>> >>> 1) Until now Iâm reordering parts of the call stack to simulate the return of a call before the actual call is sent. But now I have nested sends that make my simple approach not working anymore. I could elaborate my approach by using something like delimited continuations to make a less brittle approach to stack call reordering in order to cope with the problem. That would need some work and the stack in the debugger will look a little bit odd/confusing. >>> >>> 2.) The product will later have a dispatcher process that dispatches each send operation in its own process. That would solve one problem: the timing of when actions happen. But Iâm not sure if it will be easy to orchestrate actions in a way that I would call controlled in order to have reliable tests. >>> >>> Are there any approaches to simulate coroutines in a single thread environment or approaches to deal with multiple processes within one process? >>> >>> It is hard to explain and I hope my description of the problem is understandable >>> >>> thanks in advance, >>> >>> Norbert >> >> >

Oct. 29, 2013

Re: [Pharo-dev] Tests with multiple processes

by Frank Shearar

If you're going to even think of using callbacks like they do in Javascript, strongly reconsider. Especially so since the Javascript community are abandoning callbacks in favour of promises, especially of the A+ sort. (Squeak has such Promises in the base image, and if anyone's interested I can break them out into a Pharo-consumable library.) frank On 29 October 2013 00:11, Sebastian Sastre <sebastian(a)flowingconcept.com> wrote: > you can do stuff like that, just not sync (which when it's about networks is a good idea) > > sounds like you should design things javascript style. > > I mean, like if "everything" is a callback reacting to something.. > > yourAPIClient sendCommand: aCommand do: aBlockHandlingResponse > > my 2 cents > > > > > On Oct 28, 2013, at 8:50 PM, Norbert Hartl <norbert(a)hartl.name> wrote: > >> Iâm working on project that deals with server to server communication. For the test suite it is necessary to simulate the two ends of the communication within one test. The problem is that in a single threaded environment you make a send call SC from server A to B but the response of B will happen before SC returned. That makes every state change the happens in A after SC bogus from a call flow perspective. >> >> There are two approaches that have their own problems. >> >> 1) Until now Iâm reordering parts of the call stack to simulate the return of a call before the actual call is sent. But now I have nested sends that make my simple approach not working anymore. I could elaborate my approach by using something like delimited continuations to make a less brittle approach to stack call reordering in order to cope with the problem. That would need some work and the stack in the debugger will look a little bit odd/confusing. >> >> 2.) The product will later have a dispatcher process that dispatches each send operation in its own process. That would solve one problem: the timing of when actions happen. But Iâm not sure if it will be easy to orchestrate actions in a way that I would call controlled in order to have reliable tests. >> >> Are there any approaches to simulate coroutines in a single thread environment or approaches to deal with multiple processes within one process? >> >> It is hard to explain and I hope my description of the problem is understandable >> >> thanks in advance, >> >> Norbert > >

Oct. 29, 2013

Re: [Pharo-dev] About BitBlt current -> BitBlt (Fuel)

by Max Leske

Stef, what exactly is the influence on Fuel? I looked at the changes of your slice and couldnât find anythingâ¦ Max On 25.10.2013, at 16:26, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > Hi > > I just pushed a change cleaning some strange usage. > BitBlt was sometimes invoked via BitBlt current (which just returned the BitBlt class). > Apparently in Squeak they removed it too. > > Now it touches Fuel so Fuel guys can you merge on your side too? > > https://pharo.fogbugz.com/f/cases/12009/BitBlt-current-BitBlt > > Stef

Oct. 29, 2013