[Pharo-project] fuelized test failures
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
Wow! This is truly science fiction. Cheers, Doru On 20 Jul 2012, at 18:09, Camillo Bruni wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
-- www.tudorgirba.com "In a world where everything is moving ever faster, one might have better chances to win by moving slower."
On Sat, Jul 21, 2012 at 3:29 PM, Tudor Girba <tudor@tudorgirba.com> wrote:
Wow! This is truly science fiction.
hehehe yes, but there are still some problems when serializing contexs/debuggers. I would like to clarify the possible problems. When you serialize a context, we serialize all the context state together with its sender, and the sender has a sender, and a sender...so you end up serializing the whole list of contexts. That means that: 1) the object graph that you serialize, and hence, the stream size can be REALLY large depending on what the contexts have. Sometimes a context end up in the UI, so you end up serializing lots of morphs, colors, forms, etc. So..if everything is fine, it should be a couple hundred or thousands KB in the file. If it start to be in MB...then you may be serializing too much. 2) the same problem of 1) is that not only the graph is too big, but also that you incorporate objects (mostly when these are objects from the UI) that CHANGE while being serialized. This will cause Fuel to throw an error saying the graph has changed during serialization. 3) if 2) happens, then depending where you trigger the fuel serialization, you may end up in a loop. For example, say you want to serialize each error with Fuel. So you change SmalltalkImage>>logError:inContext: to write the context with Fuel. Now, if 2) happens and Fuel throws an error, you will try to log that again, causing again the serialization.... infinitive loop. So...some workarounds you may try to do (still, not sure if will help in all cases): 1) Deep copy the context before serializing it. 2) If you want to serialize particular contexts (for example, particular domain exceptions) then you may know WHERE to hook to make some instVars transient and therefore avoid serializing things you don't want and that may cause 2). 3) Copy the context, and actually serialize a PART of it. Cheers,
Cheers, Doru
On 20 Jul 2012, at 18:09, Camillo Bruni wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
-- www.tudorgirba.com
"In a world where everything is moving ever faster, one might have better chances to win by moving slower."
-- Mariano http://marianopeck.wordpress.com
I love that scenario :) This is great that powerful tools let us imagine and build solutions that would not be possible before. Power to imagination⦠It fits perfectly with one of the motto of Pharo: Inventing the future. On Jul 20, 2012, at 6:09 PM, Camillo Bruni wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
On Jul 22, 2012, at 10:00 AM, Stéphane Ducasse wrote:
I love that scenario :) This is great that powerful tools let us imagine and build solutions that would not be possible before. Power to imaginationâ¦
And it is an example for a building block that helps to build things. Things that are *impossible* without it. Another way to view Fuel is that of a scientific experiment: We empirically study the existing (ImageSegment). Without pre-concived notions to replace it! (We actually thought ImageSegments would turn out to be the greatest thing ever and just needed some documentation/more understandable implementation). -> You study the existing -> You claim that you can do better -> You do better. And then the next step is *extremeley* importnat: -> You prove that it is better for real by *replacing* ALL the existing subsystems that do the same. That last part is very essential: in the end, you often see that it is actually not that easy. So *replacing* is important. The other thing important is to realize that it enables things that are impossible without. There are people that claim that instead of Pharo, what we should have done is to just do our research on top of an unchanged Squeak. (While not even trying to improve Squeak itself). But I really wonder... just project Pharo some more years in the Future. Will that what we will be able to do even be *thinkable* in the context of an unchanged Squeak 3.8? Marcus -- Marcus Denker -- http://marcusdenker.de
nice explanation. Indeed I really liked IS but fuel is the way to go :) On Jul 22, 2012, at 11:11 AM, Marcus Denker wrote:
On Jul 22, 2012, at 10:00 AM, Stéphane Ducasse wrote:
I love that scenario :) This is great that powerful tools let us imagine and build solutions that would not be possible before. Power to imaginationâ¦
And it is an example for a building block that helps to build things. Things that are *impossible* without it.
Another way to view Fuel is that of a scientific experiment: We empirically study the existing (ImageSegment). Without pre-concived notions to replace it! (We actually thought ImageSegments would turn out to be the greatest thing ever and just needed some documentation/more understandable implementation). -> You study the existing -> You claim that you can do better -> You do better.
And then the next step is *extremeley* importnat:
-> You prove that it is better for real by *replacing* ALL the existing subsystems that do the same.
That last part is very essential: in the end, you often see that it is actually not that easy.
So *replacing* is important. The other thing important is to realize that it enables things that are impossible without.
There are people that claim that instead of Pharo, what we should have done is to just do our research on top of an unchanged Squeak. (While not even trying to improve Squeak itself).
But I really wonder... just project Pharo some more years in the Future. Will that what we will be able to do even be *thinkable* in the context of an unchanged Squeak 3.8?
Marcus
-- Marcus Denker -- http://marcusdenker.de
On Sun, Jul 22, 2012 at 11:11 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Jul 22, 2012, at 10:00 AM, Stéphane Ducasse wrote:
I love that scenario :) This is great that powerful tools let us imagine and build solutions that would not be possible before. Power to imaginationâ¦
And it is an example for a building block that helps to build things. Things that are *impossible* without it.
Another way to view Fuel is that of a scientific experiment: We empirically study the existing (ImageSegment). Without pre-concived notions to replace it! (We actually thought ImageSegments would turn out to be the greatest thing ever and just needed some documentation/more understandable implementation).
Indeed. I remember my first months in Douai for my PhD making drawings to understand the IS primitives :) And then Martin arrived hahaha.
-> You study the existing -> You claim that you can do better -> You do better.
And then the next step is *extremeley* importnat:
-> You prove that it is better for real by *replacing* ALL the existing subsystems that do the same.
That last part is very essential: in the end, you often see that it is actually not that easy.
That's a real good point. In Spanish I would say "Los pingos se ven en la cancha"
So *replacing* is important. The other thing important is to realize that it enables things that are impossible without.
+1 Fuel is getting more and more users. Even in BioInformatics: http://biosmalltalk.blogspot.fr/2012/07/custom-serialization-of-big-dna-or.h... http://rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel Cheers, -- Mariano http://marianopeck.wordpress.com
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com>wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test
Where can I find these .fuel of the tests?
- materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
-- Mariano http://marianopeck.wordpress.com
in the Workspace of the job -- Pavel On Mon, Jul 23, 2012 at 10:22 AM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test
Where can I find these .fuel of the tests?
- materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
-- Mariano http://marianopeck.wordpress.com
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com>wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
does it really work for you?? I am trying to materialize, for example, https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/ws/Arch... and I get: FLClassNotFound: Materialization error. Class named #HDTestReport not found That happens because during serialization the graph somehow reached the class HDTestReport (probably a sender at some point in the stack). But now, when materializing, such class is not present. So...is there in the workspace an image with the jenkins stuff? Anyway, the idea was to include this directly in Pharo, wasn't it? Thanks
The serialized stack trace is very limited (generally only 2 contexts) but they are enough to get a decent view on what the variables are
-- Mariano http://marianopeck.wordpress.com
On 2012-07-24, at 16:29, Mariano Martinez Peck wrote:
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com>wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
does it really work for you?? I am trying to materialize, for example, https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/ws/Arch...
and I get:
FLClassNotFound: Materialization error. Class named #HDTestReport not found
That happens because during serialization the graph somehow reached the class HDTestReport (probably a sender at some point in the stack). But now, when materializing, such class is not present. So...is there in the workspace an image with the jenkins stuff? Anyway, the idea was to include this directly in Pharo, wasn't it?
I suggest you use the image in the same directory, cause we install some Hudson tools (which IMO should be included by default). I don't really like that we have to load stuff for running the test, meaning that this might affect the outcome of the results... So for now either take the following image: https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/Archite... or manually install the hudson tools: "Load Hudson Build Tools" Gofer new url: 'http://ss3.gemstone.com/ss/CISupport'; package: 'HudsonBuildTools20'; load. then everything should work I think. best cami
On Tue, Jul 24, 2012 at 4:35 PM, Camillo Bruni <camillobruni@gmail.com>wrote:
On 2012-07-24, at 16:29, Mariano Martinez Peck wrote:
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
does it really work for you?? I am trying to materialize, for example,
https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/ws/Arch...
and I get:
FLClassNotFound: Materialization error. Class named #HDTestReport not
found
That happens because during serialization the graph somehow reached the class HDTestReport (probably a sender at some point in the stack). But
now,
when materializing, such class is not present. So...is there in the workspace an image with the jenkins stuff? Anyway, the idea was to include this directly in Pharo, wasn't it?
I suggest you use the image in the same directory, cause we install some Hudson tools (which IMO should be included by default).
+1 I remember there was an issue for that...I tried to find it but I fail. Do you remember?
I don't really like that we have to load stuff for running the test, meaning that this might affect the outcome of the results...
So for now either take the following image:
https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/Archite...
ok, thanks!
or manually install the hudson tools: "Load Hudson Build Tools" Gofer new url: 'http://ss3.gemstone.com/ss/CISupport'; package: 'HudsonBuildTools20'; load.
then everything should work I think.
yes, it works. And it is really cool :) Thanks! -- Mariano http://marianopeck.wordpress.com
THere is something I don't understand. If I see: https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/Archite... it seems it is independent of a particular build. What does it mean? that we only have the .fuel of the last build? if this is the case, then it is not very helpful at all. Thanks! On Tue, Jul 24, 2012 at 7:57 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
On Tue, Jul 24, 2012 at 4:35 PM, Camillo Bruni <camillobruni@gmail.com>wrote:
On 2012-07-24, at 16:29, Mariano Martinez Peck wrote:
On Fri, Jul 20, 2012 at 6:09 PM, Camillo Bruni <camillobruni@gmail.com wrote:
From now on all the test-failures are serialized on the build server.
This allows us to quickly find the cause of a failing test: - download the .fuel file for the failing test - materialize the .fuel in the image and debug the code | aContext | aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'. Debugger openContext: aContext label: 'This is the new debugger!' contents: nil
does it really work for you?? I am trying to materialize, for example,
https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/ws/Arch...
and I get:
FLClassNotFound: Materialization error. Class named #HDTestReport not
found
That happens because during serialization the graph somehow reached the class HDTestReport (probably a sender at some point in the stack). But
now,
when materializing, such class is not present. So...is there in the workspace an image with the jenkins stuff? Anyway, the idea was to include this directly in Pharo, wasn't it?
I suggest you use the image in the same directory, cause we install some Hudson tools (which IMO should be included by default).
+1 I remember there was an issue for that...I tried to find it but I fail. Do you remember?
I don't really like that we have to load stuff for running the test, meaning that this might affect the outcome of the results...
So for now either take the following image:
https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/Archite...
ok, thanks!
or manually install the hudson tools: "Load Hudson Build Tools" Gofer new url: 'http://ss3.gemstone.com/ss/CISupport'; package: 'HudsonBuildTools20'; load.
then everything should work I think.
yes, it works. And it is really cool :)
Thanks!
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Mariano Martinez Peck wrote
we only have the .fuel of the last build?
Why don't we zip any fuel files for failing tests in the artifact. This way, anyone can download and reconstruct the system. Also, the workspace is only available when we are logged in. Cheers, Sean -- View this message in context: http://forum.world.st/fuelized-test-failures-tp4640954p4641360.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On Tue, Jul 24, 2012 at 10:09 PM, Sean P. DeNigris <sean@clipperadams.com>wrote:
Mariano Martinez Peck wrote
we only have the .fuel of the last build?
Why don't we zip any fuel files for failing tests in the artifact.
I don't know much about Jenkins, but if that allows me to have the .fuel files for each version, then please go ahead :) Now...besides zipping all together, I would also like to have the individual .fuel files. CAmi/marcus anyone can take a look to this? This way,
anyone can download and reconstruct the system. Also, the workspace is only available when we are logged in.
+1
Cheers, Sean
-- View this message in context: http://forum.world.st/fuelized-test-failures-tp4640954p4641360.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
Mariano Martinez Peck wrote
but if that allows me to have the .fuel files for each version, then please go ahead :)
The zip artifact now includes all the fuel files. I didn't add the fuel files as separate artifacts because it seems there's no way to specify a target folder for artifacts without a plugin. -- View this message in context: http://forum.world.st/fuelized-test-failures-tp4640954p4641379.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 2012-07-25, at 00:50, Sean P. DeNigris wrote:
Mariano Martinez Peck wrote
but if that allows me to have the .fuel files for each version, then please go ahead :)
The zip artifact now includes all the fuel files. I didn't add the fuel files as separate artifacts because it seems there's no way to specify a target folder for artifacts without a plugin.
perfect guys!
And here I wrote what we are doing so that others know :) http://marianopeck.wordpress.com/2012/07/25/reviving-ci-test-failures-in-loc... Thanks Cami and Sean. You rock. Cheers, On Wed, Jul 25, 2012 at 6:45 AM, Camillo Bruni <camillobruni@gmail.com>wrote:
On 2012-07-25, at 00:50, Sean P. DeNigris wrote:
Mariano Martinez Peck wrote
but if that allows me to have the .fuel files for each version, then please go ahead :)
The zip artifact now includes all the fuel files. I didn't add the fuel files as separate artifacts because it seems there's no way to specify a target folder for artifacts without a plugin.
perfect guys!
-- Mariano http://marianopeck.wordpress.com
participants (7)
-
Camillo Bruni -
Marcus Denker -
Mariano Martinez Peck -
Pavel Krivanek -
Sean P. DeNigris -
Stéphane Ducasse -
Tudor Girba