Re: [Pharo-dev] [regression reporter]regression occurred
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
OS X 10.11.3 ./pharo Pharo.image test --junit-xml-output '^(?!Metacello)(?!NativeBoost)[M-Z].*' 4146 run, 4122 passes, 17 failures, 7 errors. Output from the CI build: 4146 run, 4121 passes, 17 failures, 8 errors.
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
one idea could be to add this to the filter of the CI runner. It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting. cheers -ben On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
On 24 Feb 2016, at 10:50, Ben Coman <btc@openinworld.com> wrote:
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting.
These networking problems are issues specific to the CI infrastructure. But these are real bugs: we don't know why they happen, we have no idea. It is not that there is no network, else almost everything would fail immediately. It should not be too much asked in the 21st century for a server to be able to access any website, right ?
cheers -ben
On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
The problem is that managing a CI server for a project like Pharo would be one full time engineer in a company, we do not have the manpower. So we need to find solutions that are cheap to do.
On 24 Feb 2016, at 10:50, Ben Coman <btc@openinworld.com> wrote:
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting. cheers -ben
On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
Hereâs an idea: 1. exclude Zinc tests from the validation tests 2. after the build, trigger a Travis build on Github via API (I just set that up for Fuel, so I can provide help there) 3. the Travis build only runs the Zinc tests 4. read build results from Travis Very ugly, I know. But itâs done rather quickly and should solve all the network problems. Max
On 24 Feb 2016, at 11:01, Marcus Denker <marcus.denker@inria.fr> wrote:
The problem is that managing a CI server for a project like Pharo would be one full time engineer in a company, we do not have the manpower.
So we need to find solutions that are cheap to do.
On 24 Feb 2016, at 10:50, Ben Coman <btc@openinworld.com> wrote:
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting. cheers -ben
On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
On 24 Feb 2016, at 11:09, Max Leske <maxleske@gmail.com> wrote:
Hereâs an idea:
1. exclude Zinc tests from the validation tests 2. after the build, trigger a Travis build on Github via API (I just set that up for Fuel, so I can provide help there) 3. the Travis build only runs the Zinc tests 4. read build results from Travis
Very ugly, I know. But itâs done rather quickly and should solve all the network problems.
I don't think the current problem is severe enough to put much work in it, right now. But would it not be much better to run everything (all tests) on Travis ? At first, maybe just as a limited experiment ? I would love to see that.
Max
On 24 Feb 2016, at 11:01, Marcus Denker <marcus.denker@inria.fr> wrote:
The problem is that managing a CI server for a project like Pharo would be one full time engineer in a company, we do not have the manpower.
So we need to find solutions that are cheap to do.
On 24 Feb 2016, at 10:50, Ben Coman <btc@openinworld.com> wrote:
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting. cheers -ben
On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
On 24 Feb 2016, at 11:21, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 24 Feb 2016, at 11:09, Max Leske <maxleske@gmail.com> wrote:
Hereâs an idea:
1. exclude Zinc tests from the validation tests 2. after the build, trigger a Travis build on Github via API (I just set that up for Fuel, so I can provide help there) 3. the Travis build only runs the Zinc tests 4. read build results from Travis
Very ugly, I know. But itâs done rather quickly and should solve all the network problems.
I don't think the current problem is severe enough to put much work in it, right now.
But would it not be much better to run everything (all tests) on Travis ?
At first, maybe just as a limited experiment ? I would love to see that.
Sure. That wouldnât be hard (although I think support for Windows is missing for the Smalltalk language, but Iâm sure Fabio Niephaus would help us out there). We could use the Jenkins job as a trigger (or create a second job for experimenting first). In the long run, we probably would want to use the push / pull request hook to trigger the build but for quickly hacking things together I suggest using the API trigger. I donât have enough permissions on Github and Jenkins to pull this off alone but Iâd be happy to help setting up the Travis stuff and providing the trigger script.
Max
On 24 Feb 2016, at 11:01, Marcus Denker <marcus.denker@inria.fr> wrote:
The problem is that managing a CI server for a project like Pharo would be one full time engineer in a company, we do not have the manpower.
So we need to find solutions that are cheap to do.
On 24 Feb 2016, at 10:50, Ben Coman <btc@openinworld.com> wrote:
Maybe there can be a pre-test run at the shell level to flag that the required network connectivity is available to run that test inside the image. Pharo startup could read them in while starting. cheers -ben
On Wed, Feb 24, 2016 at 5:38 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
one idea could be to add this to the filter of the CI runner.
It seems it fails due to network setup problems that are specific to the CI server...
On 24 Feb 2016, at 09:07, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
On Wed, Feb 24, 2016 at 4:07 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The following test seems to be failing a lot lately on the CI infrastructure,
On a related topic, its hard for an individual person to notice when this sort of commonly failing test occurs. All they see is the one issue they are working on. Having a daily email of the top 5 failing tests would help quicker squash such cases. But probably just an idea for the back burner. cheers -ben
yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
On 24/02/2016 09:07, Sven Van Caekenberghe wrote:
The following test seems to be failing a lot lately on the CI infrastructure, yet it always succeeds for me on my machine. Is there anybody who sees this fail on their machines ?
On 24 Feb 2016, at 08:36, no-reply@ci.inria.fr wrote:
https://ci.inria.fr/pharo/job/Pharo-5.0-Update-Step-2.1-Validation-M-Z/label...
1 regressions found. Zinc.Zodiac.ZnHTTPSTests.testAmazonAWS
It does not fail all the time but on my Mac with a stable connection with: 20 timesRepeat: [ Transcript show: Time now asString, ': '. Transcript show: (ZnHTTPSTests new setTestSelector: #testAmazonAWS; run); cr ] I got: 12:19:52.230023 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:19:53.028736 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:19:53.92617 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:19:55.23689 pm: 1 run, 0 passes, 0 skipped, 0 expected failures, 0 failures, 1 errors, 0 unexpected passes 12:20:35.851549 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:36.991826 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:37.955399 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:38.930401 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:39.81128 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:40.775596 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:42.524211 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:43.326577 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:44.291269 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:45.313826 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:46.949006 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:48.581509 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:49.546702 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:50.435629 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:51.403718 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 12:20:53.225974 pm: 1 run, 1 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes So there was one test that failed. -- Cyril Ferlicot http://www.synectique.eu 165 Avenue Bretagne Lille 59000 France
participants (5)
-
Ben Coman -
Cyril Ferlicot Delbecque -
Marcus Denker -
Max Leske -
Sven Van Caekenberghe