How coverage is computed with SmalltalkCI and coverall
Hi I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it. S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... and it's superclass. On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
Thanks I could not find it. I read the code and it is still unclear what is the logic. I could not figure out what we get lines here. I would like to resurrect the AST interpreter to get some information on AST level coverage. What is see is that so far this is a heuristic that is used in SmalltalkCI. S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... <https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph...> and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr <mailto:stephane.ducasse@inria.fr>> wrote: Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr <http://stephane.ducasse.free.fr/> / http://www.pharo.org <http://www.pharo.org/> 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
I think the coverage is done by method using something like method wrappers. And the whole method is marked as covered if hit, nothing fancy like branch or statement coverage. On Sun, Dec 20, 2020, 14:28 Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Thanks I could not find it. I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage. What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
First, configuration looks for classes and packages ( look at .smalltalk.ston file ) Second, you take all the methods inside the classes and packages. If a method is an extension ( for instance, method A of class C is under package P. But class C is under package P2), then select all the methods of the class of the method ( this logic is under discussion in two issues of SmallTalkCI. Third, SmallTalkCI wraps each method ( install a wrapper in the method's class that is called instead of the real method ). When a wrapper is called, marks a Boolean to true, and calls the original method. Finally, the coverage is the number of covered methods marked as true compare to the total number of covered method. ( Note : the wrapping is not done using reflectivity whereas it can be a good idea ? ) Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse <stephane.ducasse@inria.fr> a écrit :
Thanks I could not find it. I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage. What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
Ok I could not see the MW or methods as object use but now I get it. So the granularity is already good. I would really like the support for coverage in Pharo ecosystem and not only on build system. I would really like that we push tests to the next level :). S
On 20 Dec 2020, at 19:08, Benoît Verhaeghe <badetitou@gmail.com> wrote:
First, configuration looks for classes and packages ( look at .smalltalk.ston file ) Second, you take all the methods inside the classes and packages. If a method is an extension ( for instance, method A of class C is under package P. But class C is under package P2), then select all the methods of the class of the method ( this logic is under discussion in two issues of SmallTalkCI.
Third, SmallTalkCI wraps each method ( install a wrapper in the method's class that is called instead of the real method ). When a wrapper is called, marks a Boolean to true, and calls the original method.
Finally, the coverage is the number of covered methods marked as true compare to the total number of covered method.
( Note : the wrapping is not done using reflectivity whereas it can be a good idea ? )
Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse <stephane.ducasse@inria.fr <mailto:stephane.ducasse@inria.fr>> a écrit : Thanks I could not find it. I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage. What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com <mailto:g.cotelli@gmail.com>> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... <https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph...> and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr <mailto:stephane.ducasse@inria.fr>> wrote: Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr <http://stephane.ducasse.free.fr/> / http://www.pharo.org <http://www.pharo.org/> 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr <http://stephane.ducasse.free.fr/> / http://www.pharo.org <http://www.pharo.org/> 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
Just in case you don't find out, the wrapping is done in *SCICoverageWrapper* in SmalltalkCI-Coverage-Core package. If Pharo provides some native facility to calculate code coverage it should be pretty easy to adapt smalltalkCI to use it (if it will work at the method level). In case the coverage strategy is more focused like branch or statement based it will require a bit of change on smaltalkCI reporting support. On Sun, Dec 20, 2020 at 3:52 PM Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Ok I could not see the MW or methods as object use but now I get it. So the granularity is already good.
I would really like the support for coverage in Pharo ecosystem and not only on build system. I would really like that we push tests to the next level :).
S
On 20 Dec 2020, at 19:08, Benoît Verhaeghe <badetitou@gmail.com> wrote:
First, configuration looks for classes and packages ( look at .smalltalk.ston file ) Second, you take all the methods inside the classes and packages. If a method is an extension ( for instance, method A of class C is under package P. But class C is under package P2), then select all the methods of the class of the method ( this logic is under discussion in two issues of SmallTalkCI.
Third, SmallTalkCI wraps each method ( install a wrapper in the method's class that is called instead of the real method ). When a wrapper is called, marks a Boolean to true, and calls the original method.
Finally, the coverage is the number of covered methods marked as true compare to the total number of covered method.
( Note : the wrapping is not done using reflectivity whereas it can be a good idea ? )
Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse <stephane.ducasse@inria.fr> a écrit :
Thanks I could not find it. I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage. What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Ph... and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed. I read all the code of SmalltalkCi and I could not find it.
S. -------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
participants (3)
-
Benoît Verhaeghe -
Gabriel Cotelli -
Stéphane Ducasse