On Fri, 28 Sep 2018 at 10:23, Ben Coman <btc@openinworld.com> wrote:
Just a random idea... when running Testrunner locally and a random test causes the image to freeze, its a bit of an effort to identify which test was being run. You need to restart and then start bisecting the tests and perhaps it is intermittent so the bisect doesn't work...
but now that PharoLauncher has been mainstreamed maybe it could provide a centralised Reporting Centre i.e. open a port for launched images to report the test about-to-commence as well as the run time. Then it would be simple to see which test froze the system. Also it could keep a record individual test runtimes, to alleviate fears when the system is not frozen but just a long running test.
Extending the idea, while CI testing is very useful, we get these sporadic errors that are hard to replicate locally. A local PharoLauncher driven system for repetitively launching an image auto-running tests might be good to gather statistics on sporadic failures to then drill down on the troublemakers.
Plus with the goal of having an object based logging system, a viewer is needed and perhaps PharoLauncher would be a logical place to provide a logging service to gather and especially to view log-objects reporting by launched images.
side thought, such a system might be good for watching test progress on headless IoT devices.
of course, resources required to do it, but just thought I share the idea.
cheers -ben