2018-03-05 14:14 GMT+01:00 Stephane Ducasse <stepharo.self@gmail.com>:
Hi pavel
when I'm back can you explain to me because I did not get it :).
:) it is simple. If you want to ignore issues that the slice is not adding, you need to know which of them to ignore. That's why the original monkey run all the validations twice - first time to collect a list of failing tests in the fresh unchanged image and then with the slice or configuration loaded. So it doubled the issue validation time. There are several alternative strategies like to cache the failing test results for every build and use them for validations but the best strategy is simply to keep the amount of failing tests in the clean image on the zero level and force people to keep the system clean. The original monkey hasn't exposed the list of ignored tests. So it was possible that some test was ignored because of a temporal network issue but for the second time it failed for o good reason and you even didn't know. Cheers, -- Pavel
Stef
On Mon, Mar 5, 2018 at 11:07 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-03-05 10:54 GMT+01:00 Marcus Denker <marcus.denker@inria.fr>:
On 5 Mar 2018, at 10:27, Alistair Grant <akgrant0710@gmail.com> wrote:
Hi Marcus,
On 5 March 2018 at 09:23, Marcus Denker <marcus.denker@inria.fr> wrote:
On 5 Mar 2018, at 09:16, Alistair Grant <akgrant0710@gmail.com> wrote:
Hi Esteban & Marcus,
I'm getting repeated validation failures for:
https://pharo.manuscript.com/f/cases/21431
It's the same set of tests that fail each time, and as far as I can tell they have nothing to do with the patch I submitted.
Do you know if this is happening on other tests?
I saw that Saturday but decided to wait till Monday (weekends are important..).
So: no, I have *no* idea what happened. From one CI run to the next, suddenly around 160 tests related to Calypso started failing due to a missing method.
Now starting from sometime today, this problem stoped. The last failing PR checks fail due to different reasonsâ¦
And I have no idea why.
(And yes, we al know that 1) the PR checks need more compute power, too slow 2) we *need* to track down the reason why still *a lot* of times the PR fails even though it should not.
The problem is that just keeping a build alive of this kind is a full time job.. that we have nobody doing, so many many people do as much as they can and we hope it will get betterâ¦.)
Thanks for the update.
Oh, and it was completely unrelated. Your change is for Pharo6...
I took a look at the failures and it appears that
BehaviorTest>>testBehaviorRespectsPolymorphismWithTraitBehavior ClassDescriptionTest>>testClassDescriptionRespectsPolymorphismWithTraitDescription ClassTest>>testClassRespectsPolymorphismWithTrait
are all failing due to changes in Fuel - methods were changed from traits to local methods.
Yes, the problem is that the monkey (the contribution checker) fails as soon as there are errors even in the main image.
The last Pharo6 has these tests failing, so now all contribution checks for Pharo6 fail.
What needs to be done?
-> your change can be accepted as we know it does not fail more fixes -> then we need fix the tests in Pharo6 -> in a perfect world we would update the slice checker to only fail for now test failing⦠(it used to be lille thatâ¦).
I must say that It made the validation two times slower, fragile and led to the hiding of problems instead of solving them...
-- Pavel
As I said: this is a full time jobâ¦
Marcus