On 02 Feb 2014, at 14:31, btc@openinworld.com wrote:
For diligence and curiosity leading up to the Pharo3 release, I downloaded build image 30733 (with PharoLauncher)
Getting an image green is a lot of little, trivial, hard work⦠and green always means âGreen in this contextâ. Just headless vs. non-headless changes a lot, for example. Another wonderful thing is that tests are fine when run individually, but fail when running all tests. We need to think about automatic UI testing⦠Loading an external package changes *everything*. We do not have a a release criterium that all tests are green after loading external packages.
and immediately ran all tests, which reported 7 failures and 1 error (see attached snapshot). Now presumably these did not occur in the CI build since otherwise it would not have been available to PharoLauncher, so this raises a few questions/ideas:
1. Why the discrepancy / how did these slip through ? Perhaps some difference in local environment / difference in VM? Can anyone else reproduce this for 30733? I've logged an new case [1] for this where I plan to either link to existing cases or add a subcase for each failure. Please add your results there, particularly if you see a known issue.
Here are my failures... BlockClosureTest>>#testTrace
Not failing for me
JobTest>>#testCurrent Not failing for me
MetacelloRepositorySqueakCommonTestCase>>#testAsRepositorySpecFor MetacelloRepositorySqueakCommonTestCase>>#testDirectoryRepository MetacelloRepositorySqueakCommonTestCase>>#testFileTreeRepository Not failing fo rme
ReleaseTest>>#testObsoleteClasses
Not failing for me
SimmulateMouseSpecification>>#testSimulateRightClick yes, that is failing
and errors... OpenToolTest>>#testOpenBrowseOnInstalledTraitMethod
yes. The last two are examples of UI tests that are not run on the Build server in a way that they could fail.
3. Assuming Pharo3 will go through a Release Candidate phase,
Normally the idea is that we will just release. In the past we used an elaborate process, but the reality is that nobody looks at release candidates. You can tell 5 times that the release candidate will be released unchanged and people should check: they will not. They will download the release, though, and then complain that the âobviousâ thing X is not fixed that that the people who managed the release process did everything wrong. Because âreleaseâ means bug free. By magic. Marcus