How coverage is computed with SmalltalkCI and coverall
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
AFAIR is implemented in
https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class
and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Thanks I could not find it.
I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage.
What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli g.cotelli@gmail.com wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr mailto:stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr http://stephane.ducasse.free.fr/ / http://www.pharo.org http://www.pharo.org/
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
I think the coverage is done by method using something like method
wrappers. And the whole method is marked as covered if hit, nothing fancy
like branch or statement coverage.
On Sun, Dec 20, 2020, 14:28 Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Thanks I could not find it.
I read the code and it is still unclear what is the logic. I could not
figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on
AST level coverage.
What is see is that so far this is a heuristic that is used in
SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli g.cotelli@gmail.com wrote:
AFAIR is implemented in
https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class
and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
First, configuration looks for classes and packages ( look at
.smalltalk.ston file )
Second, you take all the methods inside the classes and packages.
If a method is an extension ( for instance, method A of class C is under
package P. But class C is under package P2), then select all the methods of
the class of the method ( this logic is under discussion in two issues of
SmallTalkCI.
Third, SmallTalkCI wraps each method ( install a wrapper in the method's
class that is called instead of the real method ).
When a wrapper is called, marks a Boolean to true, and calls the original
method.
Finally, the coverage is the number of covered methods marked as true
compare to the total number of covered method.
( Note : the wrapping is not done using reflectivity whereas it can be a
good idea ? )
Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse stephane.ducasse@inria.fr
a écrit :
Thanks I could not find it.
I read the code and it is still unclear what is the logic. I could not
figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on
AST level coverage.
What is see is that so far this is a heuristic that is used in
SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli g.cotelli@gmail.com wrote:
AFAIR is implemented in
https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class
and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Ok I could not see the MW or methods as object use but now I get it.
So the granularity is already good.
I would really like the support for coverage in Pharo ecosystem and not only on build system.
I would really like that we push tests to the next level :).
S
On 20 Dec 2020, at 19:08, Benoît Verhaeghe badetitou@gmail.com wrote:
First, configuration looks for classes and packages ( look at .smalltalk.ston file )
Second, you take all the methods inside the classes and packages.
If a method is an extension ( for instance, method A of class C is under package P. But class C is under package P2), then select all the methods of the class of the method ( this logic is under discussion in two issues of SmallTalkCI.
Third, SmallTalkCI wraps each method ( install a wrapper in the method's class that is called instead of the real method ).
When a wrapper is called, marks a Boolean to true, and calls the original method.
Finally, the coverage is the number of covered methods marked as true compare to the total number of covered method.
( Note : the wrapping is not done using reflectivity whereas it can be a good idea ? )
Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse <stephane.ducasse@inria.fr mailto:stephane.ducasse@inria.fr> a écrit :
Thanks I could not find it.
I read the code and it is still unclear what is the logic. I could not figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on AST level coverage.
What is see is that so far this is a heuristic that is used in SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli <g.cotelli@gmail.com mailto:g.cotelli@gmail.com> wrote:
AFAIR is implemented in https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse <stephane.ducasse@inria.fr mailto:stephane.ducasse@inria.fr> wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr http://stephane.ducasse.free.fr/ / http://www.pharo.org http://www.pharo.org/
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr http://stephane.ducasse.free.fr/ / http://www.pharo.org http://www.pharo.org/
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Just in case you don't find out, the wrapping is done in
SCICoverageWrapper in SmalltalkCI-Coverage-Core package.
If Pharo provides some native facility to calculate code coverage it should
be pretty easy to adapt smalltalkCI to use it (if it will work at the
method level). In case the coverage strategy is more focused like branch or
statement based it will require a bit of change on smaltalkCI reporting
support.
On Sun, Dec 20, 2020 at 3:52 PM Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Ok I could not see the MW or methods as object use but now I get it.
So the granularity is already good.
I would really like the support for coverage in Pharo ecosystem and not
only on build system.
I would really like that we push tests to the next level :).
S
On 20 Dec 2020, at 19:08, Benoît Verhaeghe badetitou@gmail.com wrote:
First, configuration looks for classes and packages ( look at
.smalltalk.ston file )
Second, you take all the methods inside the classes and packages.
If a method is an extension ( for instance, method A of class C is under
package P. But class C is under package P2), then select all the methods of
the class of the method ( this logic is under discussion in two issues of
SmallTalkCI.
Third, SmallTalkCI wraps each method ( install a wrapper in the method's
class that is called instead of the real method ).
When a wrapper is called, marks a Boolean to true, and calls the original
method.
Finally, the coverage is the number of covered methods marked as true
compare to the total number of covered method.
( Note : the wrapping is not done using reflectivity whereas it can be a
good idea ? )
Le dim. 20 déc. 2020 à 18:29, Stéphane Ducasse stephane.ducasse@inria.fr
a écrit :
Thanks I could not find it.
I read the code and it is still unclear what is the logic. I could not
figure out what we get lines here.
I would like to resurrect the AST interpreter to get some information on
AST level coverage.
What is see is that so far this is a heuristic that is used in
SmalltalkCI.
S.
On 20 Dec 2020, at 13:27, Gabriel Cotelli g.cotelli@gmail.com wrote:
AFAIR is implemented in
https://github.com/hpi-swa/smalltalkCI/tree/master/repository/SmalltalkCI-Pharo-Coverage-Core.package/SCIPharoCodeCoverage.class
and it's superclass.
On Sun, Dec 20, 2020, 05:44 Stéphane Ducasse stephane.ducasse@inria.fr
wrote:
Hi
I would like to know exactly how the test coverage is computed.
I read all the code of SmalltalkCi and I could not find it.
S.
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France
Stéphane Ducasse
http://stephane.ducasse.free.fr / http://www.pharo.org
03 59 35 87 52
Assistant: Aurore Dalle
FAX 03 59 57 78 50
TEL 03 59 35 86 16
S. Ducasse - Inria
40, avenue Halley,
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France