On Thu, Aug 24, 2017 at 2:16 PM, Ben Coman <btc@openinworld.com> wrote:
On Tue, Aug 22, 2017 at 11:51 PM, Guillermo Polito < guillermopolito@gmail.com> wrote:
Hi Marcus,
Please, do not relaunch the build if there is a test failure. That simply does not fix the problem, it just adds noise. There are two problems:
1) we have some sporadic test failures (sometimes network, sometimes some mutex or delay issues). They do not happen all the time and that's why sometimes there is one or two failing tests. We should detect these tests and fix them to be more robust
Note, is there some brave soul that would like to take action here?
2) sometimes the vm crashes while running the tests. We should detect the bug and fix it.
Note, is there some brave soul that would like to take action here?
Now, if we are integrating a fix, this means that the PR was **already validated as green**. That means that the integrated commit can be bootstrapped and tested (and all tests are green).
Moreover, running the tests after integration (and its build color) does not change the integration in itself: the commit was already merged and pushed into the main branch, so even if it fails always, it's too late.
I thought the idea of CI was to merge into the main branch *after* the tests pass.
It is the way we do it, nothing should be merged if tests are not green. BUT, we are also running the tests after merging, just because :). And it is in that second test-run when you may have some failing tests...
Why is the PR-branch not checked out to run the tests on...? https://help.github.com/articles/checking-out-pull-requests-locally/
cheers -ben
-- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13