Shouldn't Jenkins help us here?
We have about three options: 1) use Jenkins for updates checking - maybe it was the original plan but it doesn't work because we do not want to rollback updates
I think the original plan was to only use images that completed all the tests. Much like the VM builds.
2) create a fake update server and create Jenkins jobs that will check the updates and will not pass the dirty ones to mainstream
I guess that goes along with #1, we should be able to rollback changes easily. And yes, we should have a 2 level publish system, first internally for checking and only then publicly..
3) improve the Monkey do do more checks on fixes of issues - to run more tests etc. And do not allow fixes that do not pass.
the monkey should filter out as much as possible. since at that point we didn't change anything yet, so it's the ideal moment to fix things :) - do strict testing (of everything) - do some code critics (undeclared vars ...) - provide simple heuristics about the complexity of the fix