Need help to get a list of Pharo projects with tests
Hello Pharo community, I am currently working on detecting rotten tests in Pharo projects. Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run. To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1]. In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development. We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests. To do that, we need your help. You can help us in two ways: 1. Answer this email with links to one or many open-source Pharo projects containing tests. 2. If you want, provide us private projects or, at least, run our analyser on these private projects and give us back the results. If you want to help us with the second option, send me a mail directly at julien.delplanque@inria.fr <mailto:julien.delplanque@inria.fr>. Thanks in advance. Regards, Julien Delplanque Links: [1]: https://hal.archives-ouvertes.fr/hal-01819302 <https://hal.archives-ouvertes.fr/hal-01819302> [2]: https://github.com/juliendelplanque/RottenTestsFinder/ <https://github.com/juliendelplanque/RottenTestsFinder/> --- Julien Delplanque Doctorant à lâUniversité de Lille http://juliendelplanque.be/phd.html Equipe Rmod, Inria Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq Numéro de téléphone: +333 59 35 86 40
On Thu, Jul 5, 2018 at 3:24 PM Julien <julien.delplanque@inria.fr> wrote:
Hello Pharo community,
I am currently working on detecting rotten tests in Pharo projects.
Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run. To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].
In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.
We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.
To do that, we need your help. You can help us in two ways: 1. Answer this email with links to one or many open-source Pharo projects containing tests.
âWe have 774 tests in PolyMath: https://github.com/PolyMathOrg/PolyMath -- Serge Stinckwich UMI UMMISCO 209 (SU/IRD/UY1) "Programs must be written for people to read, and only incidentally for machines to execute."http://www.doesnotunderstand.org/
Fuel: https://github.com/theseion/Fuel Seaside: https://github.com/SeasideSt/Seaside On 5 Jul 2018, at 16:31, Serge Stinckwich wrote:
On Thu, Jul 5, 2018 at 3:24 PM Julien <julien.delplanque@inria.fr> wrote:
Hello Pharo community,
I am currently working on detecting rotten tests in Pharo projects.
Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run. To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].
In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.
We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.
To do that, we need your help. You can help us in two ways: 1. Answer this email with links to one or many open-source Pharo projects containing tests.
âWe have 774 tests in PolyMath: https://github.com/PolyMathOrg/PolyMath
-- Serge Stinckwich UMI UMMISCO 209 (SU/IRD/UY1) "Programs must be written for people to read, and only incidentally for machines to execute."http://www.doesnotunderstand.org/
Hi, can you please include in the README at the very least how it should be launched? Looking through the code I eventually found that there's a world menu entry... but that's not exactly the best way to discovery. I've ran it on couple of projects, but it didn't show anything apart from self skip, but if you find something else, I'd be interested to see what it can detect. https://github.com/peteruhnak?utf8=%E2%9C%93&tab=repositories&q=&type=source... Also when I filter packages and select one, the test cases list contains some random unrelated tests. When the filtering is not on, the test cases are correct. Cheers, Peter On Thu, Jul 5, 2018 at 7:16 PM, Max Leske <maxleske@gmail.com> wrote:
Fuel: https://github.com/theseion/Fuel Seaside: https://github.com/SeasideSt/Seaside
On 5 Jul 2018, at 16:31, Serge Stinckwich wrote:
On Thu, Jul 5, 2018 at 3:24 PM Julien <julien.delplanque@inria.fr> wrote:
Hello Pharo community,
I am currently working on detecting rotten tests in Pharo projects.
Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run. To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].
In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.
We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.
To do that, we need your help. You can help us in two ways: 1. Answer this email with links to one or many open-source Pharo projects containing tests.
âWe have 774 tests in PolyMath: https://github.com/PolyMathOrg/PolyMath
-- Serge Stinckwich UMI UMMISCO 209 (SU/IRD/UY1) "Programs must be written for people to read, and only incidentally for machines to execute."http://www.doesnotunderstand.org/
Thanks for all your answers, Iâm collecting these projects for the experiment. :-) Peter,
Le 5 juil. 2018 à 21:12, Peter Uhnák <i.uhnak@gmail.com> a écrit :
Hi,
can you please include in the README at the very least how it should be launched? Looking through the code I eventually found that there's a world menu entry... but that's not exactly the best way to discovery.
Yeah, the GitHub repository is not yet in a good state, the previous mail was not an announcement for the project in fact, I just put the link to show it exists, itâs real⦠:-) Anyway, you can run it (without the GUI, because there are some bugs in it and I do not focus too much on it for now) using this: [RottenTestsFinder analyze: (RPackage organizer packageNamed: 'Collections-Tests')] on: Warning do: #resume.
I've ran it on couple of projects, but it didn't show anything apart from self skip, but if you find something else, I'd be interested to see what it can detect.
In Collection tests you have some rotten tests. :-) BUT, the result are not totally correct right now, it is still under development so do not expect it to work for now.
https://github.com/peteruhnak?utf8=%E2%9C%93&tab=repositories&q=&type=source... <https://github.com/peteruhnak?utf8=%E2%9C%93&tab=repositories&q=&type=source...>
Also when I filter packages and select one, the test cases list contains some random unrelated tests. When the filtering is not on, the test cases are correct.
Yeah, I know this bug, the filter of the Spec widget does not work as I expected. If you do not use the filters, it should work well. You can select all tests in the second list using Cmd+a. Cheers, Julien
participants (4)
-
Julien -
Max Leske -
Peter Uhnák -
Serge Stinckwich