Cami, Even worse, I don't get _any_ information about failures. When I run unit tests headless I get the following in my output: =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== =============================================================================== Notice: Errors in script loaded from /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st =============================================================================== No stacks in the debug log, so I am completely blind when I run Pharo3.0 builds ... been happening about 2 weeks ... I assume there was a radical change to the way the Pharo3.0 reports headless errors, either by accident or on purpose... Christophe will be checking into this and I am sure he'll et me running again ... Until I can see what's happening with the builds I can't get the last little bits fixed for Pharo3.0 ... very close though:) Dale ----- Original Message ----- | From: "Camillo Bruni" <camillobruni@gmail.com> | To: "Pharo Development List" <pharo-dev@lists.pharo.org> | Sent: Sunday, July 7, 2013 3:11:38 PM | Subject: Re: [Pharo-dev] filetree jenkins job | | I will change SUnit first produce proper error messages for the | #shouldnt:raise: methods since I cannot properly debug them. | Additionally at the point where the #:assert: happens the wrong | exception is not in the context which means it does not get properly | serialized by our test framework. All in all, with a little effort | we could debug much easier. | | BTW: There are so many strange methods on TAssertable that I do not | understand | - #shouldFix: | - #executeShould:inScopeOf: | | and some that I honestly think they hurt more than they help: | - #shouldnt:raise:whoseDescriptionIncludes:description: and friends? | | On 2013-07-07, at 23:47, "Dale K. Henrichs" | <dale.henrichs@gemtalksystems.com> wrote: | | > Cami, | > | > I've been having trouble with getting any useful information from | > Pharo3.0 failures on travis-ci[1] for a week or so now ... | > Christophe will be looking into this for me... | > | > Dale | > | > [1] | > https://github.com/dalehenrich/filetree/issues/80#issuecomment-20162125 | > ----- Original Message ----- | > | From: "Camillo Bruni" <camillobruni@gmail.com> | > | To: pharo-dev@lists.pharo.org | > | Sent: Sunday, July 7, 2013 2:00:10 PM | > | Subject: [Pharo-dev] filetree jenkins job | > | | > | after I tried to get filetree with the tests installed and I | > | didn't | > | get it | > | the first time I decided to create a jenkins job. | > | | > | https://ci.inria.fr/pharo-contribution/job/FileTree/ | > | | > | | > | | |

----- Original Message ----- | From: "Camillo Bruni" <camillobruni@gmail.com> | To: "Pharo Development List" <pharo-dev@lists.pharo.org> | Sent: Sunday, July 7, 2013 5:26:45 PM | Subject: Re: [Pharo-dev] filetree jenkins job | | On 2013-07-08, at 01:59, Dale K. Henrichs | <dale.henrichs@gemtalksystems.com> wrote: | > Cami, | > | > Even worse, I don't get _any_ information about failures. When I | > run unit tests headless I get the following in my output: | > | > =============================================================================== | > Notice: Errors in script loaded from | > /home/travis/dalehenrich-builderCI-3062fdf/builds/travisCI/travisCI.st | > =============================================================================== | > | > No stacks in the debug log, so I am completely blind when I run | > Pharo3.0 builds ... been happening about 2 weeks ... I assume | > there was a radical change to the way the Pharo3.0 reports | > headless errors, either by accident or on purpose... | | that's usually not a bug, it currently is a side-effect of Opal | giving proper feedback about undefined variables. | With the new classbuilder introduce we might have some side-effects | as well (there is a reported case that Martin and Christophe found). | | > Christophe will be checking into this and I am sure he'll et me | > running again ... | > | > Until I can see what's happening with the builds I can't get the | > last little bits fixed for Pharo3.0 ... very close though:) | | sometimes what helps is running in jenkins mode, failing tests | produce a serialized stack which can be drag n dropped onto an image | for easier debugging. I use that on a regular basis when figuring | out non-deterministic tests :) | | You can see such files here: | https://ci.inria.fr/pharo-contribution/job/FileTree/PHARO=30,VERSION=stable,... | Running on travisCI does not allow me to get access to serialized anything ... I get to see a log file (what do you expect for free:) ... a simple stack report printed is usually enough for me to figure out what's happening but the above is totally useless ... most cases reproduce locally, but the pharo3.0 issues that I'm trying to debug now are failing only on travis so I need to have something to point to what's wrong ... like I said I figured the headless operation had been jiggered in some way and will patiently wait for Christophe to come up with an acceptable solution for running on travis ... Dale

----- Original Message ----- | From: "Camillo Bruni" <camillobruni@gmail.com> | To: "Pharo Development List" <pharo-dev@lists.pharo.org> | Sent: Sunday, July 7, 2013 7:02:26 PM | Subject: Re: [Pharo-dev] filetree jenkins job | | > Running on travisCI does not allow me to get access to serialized | > anything ... I get to see a log file (what do you expect for | > free:) ... a simple stack report printed is usually enough for me | > to figure out what's happening but the above is totally useless | > ... most cases reproduce locally, but the pharo3.0 issues that I'm | > trying to debug now are failing only on travis so I need to have | > something to point to what's wrong ... like I said I figured the | > headless operation had been jiggered in some way and will | > patiently wait for Christophe to come up with an acceptable | > solution for running on travis ... | | | well that's what the fuel files are for... | Can't you run the tests and copy the files to a safe location where | you can analyze them offline? Nope. | | Depending on stack traces to debug is definitely history ;) I guess I won't be porting Metacello to Pharo3.0 then either... Dale

----- Original Message ----- | From: "Camillo Bruni" <camillobruni@gmail.com> | To: "Pharo Development List" <pharo-dev@lists.pharo.org> | Sent: Sunday, July 7, 2013 7:25:02 PM | Subject: Re: [Pharo-dev] filetree jenkins job | | > | well that's what the fuel files are for... | > | Can't you run the tests and copy the files to a safe location | > | where | > | you can analyze them offline? | > | > Nope. | | seems to be doable: | http://sleepycoders.blogspot.fr/2013/03/sharing-travis-ci-generated-files.ht... Like I said, I'm willing to be patient while Christophe supplies a solution to the problem... | | > | > | Depending on stack traces to debug is definitely history ;) | > | > I guess I won't be porting Metacello to Pharo3.0 then either... | | | Don't get me wrong, there still are stack traces, but most probably | the bug you experience is more hidden. In a local Pharo2.0 image, there are simply test failures and if I had a debug log of some sort, I would have fixed the problem 2 weeks ago:) My guess is that along with the changes that caused the debug log to disappear there were other changes related to running a headlesss remote vm in batch mode that I am ignorant of... | | What I am trying to say is that should use the fuel files to debug | failing tests instead of spending time deciphering dead text-based | stack traces. fuel files are not an option ... I'll tell you what though ... why don't you write come code that reads the fuel files into a pharo image and prints a stack trace to stdout ... that will get me exactly what I need and have for all of the other Smalltalk platforms ... When I am using travis, I am looking at the log file of the run just to see if the test passed or failed, so to have a "dead stack trace" right there in the log file that I am looking at is exactly what I WANT ... most of the time the stack trace is enough for me to debug the problem... | | And exactly in the case you describe theses fuel files can save you a | lot of time since you can basically debug the same way you debug a | local application. Except that I rarely write code in Pharo3.0, but I am interested in running tests and seeing the results of tests and seeing stack traces for failures for the code when it is run against Pharo3.0 ... These days I do the bulk of my development for FileTree, Metacello, etc. using tODE in GemStone, so I don't usually even have a Pharo development image (especially a fresh one) available to do testing ... so a stack trace actually saves me a whole lot of time... | | BTW: where do I see which tests fail under travis? clicking on the | status badges on the filetree page doesn't lead me to the direct | results. Here's the most recent pharo3.0 failure: https://travis-ci.org/dalehenrich/filetree/builds/8827089 I am actually merging code into the pharo3.0 branch even though I can't tell what's happening in the Pharo3.0 image ... the rest of the pharo, squeak and gemstone builds are all green so I expect that once Christophe has fixed the problem with Pharo3.0 I won't have a lot of work to do to get the tests passing ... Here's one from 18 days ago (the first one to go blind): https://travis-ci.org/dalehenrich/filetree/builds/8140902 Here's one from 22 days ago (the tests failed) which shows the logging information that used to work fine in Pharo3.0: https://travis-ci.org/dalehenrich/filetree/builds/8125607 | | In any case I see 4 failing tests on our jenkins build: | https://ci.inria.fr/pharo-contribution/job/FileTree/PHARO=30,VERSION=stable,... | | are these the same you have on travis? | I haven't seen the test results from travis for 22 days, so I don't now what is going on with pharo3.0 ... keep in mind that Christophe and Thierry have been doing Pharo2.0 development and they are seeing tests pass in their local environent, but the totally opaque and useless error messages I'm getting on travis give me no clue as to what might be wrong up there ... 22 days ago I was getting adequate information from Pharo3.0 Dale

On 7 July 2013 23:11, Camillo Bruni <camillobruni@gmail.com> wrote:

I will change SUnit first produce proper error messages for the #shouldnt:raise: methods since I cannot properly debug them. Additionally at the point where the #:assert: happens the wrong exception is not in the context which means it does not get properly serialized by our test framework. All in all, with a little effort we could debug much easier.

I found this a big improvement: TestCase >> shouldnt: aBlock raise: anExceptionalEvent ^ [ aBlock value ] on: anExceptionalEvent do: [:e | self fail: 'Block raised ', e className, ': ', e messageText]. TestCase >> fail: aString ^self assert: false description: aString. frank

BTW: There are so many strange methods on TAssertable that I do not understand - #shouldFix: - #executeShould:inScopeOf:

and some that I honestly think they hurt more than they help: - #shouldnt:raise:whoseDescriptionIncludes:description: and friends?

On 2013-07-07, at 23:47, "Dale K. Henrichs" <dale.henrichs@gemtalksystems.com> wrote:

...

Cami,

I've been having trouble with getting any useful information from Pharo3.0 failures on travis-ci[1] for a week or so now ... Christophe will be looking into this for me...

Dale

[1] https://github.com/dalehenrich/filetree/issues/80#issuecomment-20162125 ----- Original Message ----- | From: "Camillo Bruni" <camillobruni@gmail.com> | To: pharo-dev@lists.pharo.org | Sent: Sunday, July 7, 2013 2:00:10 PM | Subject: [Pharo-dev] filetree jenkins job | | after I tried to get filetree with the tests installed and I didn't | get it | the first time I decided to create a jenkins job. | | https://ci.inria.fr/pharo-contribution/job/FileTree/ | |