[Pharo-project] code coverage?
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used? Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called. No doubt from the other side it would be interesting to see from a VM platform perspective if the Smalltalk Sunits actually execute a reasonable set of the slang/objective-c/C code. -- =========================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
No doubt from the other side it would be interesting to see from a VM platform perspective if the Smalltalk Sunits actually execute a reasonable set of the slang/objective-c/C code.
-- =========================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
I used Spy on a number of projects, including Moose, Glamour, Mondrian, Seaside, ... But checking it on the whole image is worth trying. Give me a couple of hours... Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On 2009-12-03, at 7:33 AM, Alexandre Bergel wrote:
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
I used Spy on a number of projects, including Moose, Glamour, Mondrian, Seaside, ... But checking it on the whole image is worth trying.
Give me a couple of hours...
Thank you, allow me to give some clues about what internal plugins are in each macintosh VM (3.x, 4.x) series. ADPCMCodecPlugin_exports, class ADPCMCodec AsynchFilePlugin_exports, class AsynchFile B2DPlugin_exports, class BallonEngine BitBltPlugin_exports, class BitBlt,WarpBlt DSAPrims_exports, class SHA1,SecureHashAlgorithm FFTPlugin_exports, class FFT FilePlugin_exports, class FileDirectory,StandardFileStream FloatArrayPlugin_exports, class BallonBuffer,FloatArray,MatrixTransform2x3 HostWindowPlugin_exports, class HostWindowProxy + subclasses InternetConfigPlugin_exports, class InternetConfiguration JoystickTabletPlugin_exports, class InputEventSensor JPEGReaderPlugin_exports, class JPEGReadWriter JPEGReadWriter2Plugin_exports, class JPEGReadWriter2 LargeIntegers_exports, Large positive/negative number usage... MacMenubarPlugin_exports, class HostSystemMenus + subclasses Matrix2x3Plugin_exports, class MatrixTransform2x3 MIDIPlugin_exports, class MidiPrimTester,SimpleMIDIPort MiscPrimitivePlugin_exports, class BitMap, ByteArray, ByteString,ByteSymbol,SampledSound SecurityPlugin_exports, class SecurityManager SerialPlugin_exports, class SerialPort SocketPlugin_exports, class NetNameResolver, Socket,SocketAddress SoundCodecPrims_exports, class GSMCodec SoundGenerationPlugin_exports, class FMSound,LoopedSampledSound,PluckedSound,ReverbSound,SampledSound SoundPlugin_exports, class SoundPlayer,SoundRecorder UUIDPlugin_exports, class UUID ZipPlugin_exports, class DeflateStream,FastInflateStream,NSAbstractDeflateStream+subclasses,ZipEncoder, ZipWriteStream These plugins are built in, but not used directly by Pharo 1.0-10496-rc1dev09.11.4.image B3DAcceleratorPlugin_exports, Requires Ballon 3d http://www.squeaksource.com/Balloon3D.html Squeak3D_exports, BMPReadWriterPlugin_exports, Oddly the smalltalk usage of this plugin was deleted by the author. GeniePlugin_exports, Requires http://www.squeaksource.com/genie.html Klatt_exports, Requires http://www.squeaksource.com/Speech.html RePlugin_exports, I think this is for the http://www.squeaksource.com/Regex.html StarSqueakPlugin_exports, Etoys SurfacePlugin_exports, Internal usage and exploited by various plugins like QuickTime for Sophie -- =========================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
hi John, I have been working on a code instrumentation mechanism that use the "object as method" trick to get the runtime info. The "hello world" of the test coverage of Spy could be: 1- open a workspace, and doit: Gofer new squeaksource: 'Spy'; addPackage: 'ConfigurationOfSpy'; load. (Smalltalk at: #ConfigurationOfSpy) perform: #loadDefault. 2- doit: CoveragePackageSpy viewCoverageForPackagesMatching: 'Mondrian*' 3- after few seconds, a mondrian window will open. Pink inner boxes methods that are not covered by unit tests. The expression above (i)instruments all methods belonging to Mondrian, (ii) run all the test, (iii) open a simple mondrian visualization that tells whether a method is executed or not when running th test. So far, it is good enough for small and controlled experiments. I would to do the coverage for the whole image, but I need some time for this. In 2 weeks I will be in holidays, I will therefore have time to work on this. I also worked on a mechanism to extract type information from SUnit tests. Try: (TypingPackageSpy runTestAndGatherForPackagesMatching: 'Mondrian*') viewBasic Cheers, Alexandre On 3 Dec 2009, at 17:20, John M McIntosh wrote:
On 2009-12-03, at 7:33 AM, Alexandre Bergel wrote:
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
I used Spy on a number of projects, including Moose, Glamour, Mondrian, Seaside, ... But checking it on the whole image is worth trying.
Give me a couple of hours...
Thank you, allow me to give some clues about what internal plugins are in each macintosh VM (3.x, 4.x) series.
ADPCMCodecPlugin_exports, class ADPCMCodec AsynchFilePlugin_exports, class AsynchFile B2DPlugin_exports, class BallonEngine BitBltPlugin_exports, class BitBlt,WarpBlt DSAPrims_exports, class SHA1,SecureHashAlgorithm FFTPlugin_exports, class FFT FilePlugin_exports, class FileDirectory,StandardFileStream FloatArrayPlugin_exports, class BallonBuffer,FloatArray,MatrixTransform2x3 HostWindowPlugin_exports, class HostWindowProxy + subclasses InternetConfigPlugin_exports, class InternetConfiguration JoystickTabletPlugin_exports, class InputEventSensor JPEGReaderPlugin_exports, class JPEGReadWriter JPEGReadWriter2Plugin_exports, class JPEGReadWriter2 LargeIntegers_exports, Large positive/negative number usage... MacMenubarPlugin_exports, class HostSystemMenus + subclasses Matrix2x3Plugin_exports, class MatrixTransform2x3 MIDIPlugin_exports, class MidiPrimTester,SimpleMIDIPort MiscPrimitivePlugin_exports, class BitMap, ByteArray, ByteString,ByteSymbol,SampledSound SecurityPlugin_exports, class SecurityManager SerialPlugin_exports, class SerialPort SocketPlugin_exports, class NetNameResolver, Socket,SocketAddress SoundCodecPrims_exports, class GSMCodec SoundGenerationPlugin_exports, class FMSound,LoopedSampledSound,PluckedSound,ReverbSound,SampledSound SoundPlugin_exports, class SoundPlayer,SoundRecorder UUIDPlugin_exports, class UUID ZipPlugin_exports, class DeflateStream,FastInflateStream,NSAbstractDeflateStream +subclasses,ZipEncoder, ZipWriteStream
These plugins are built in, but not used directly by Pharo 1.0-10496-rc1dev09.11.4.image
B3DAcceleratorPlugin_exports, Requires Ballon 3d http://www.squeaksource.com/Balloon3D.html Squeak3D_exports,
BMPReadWriterPlugin_exports, Oddly the smalltalk usage of this plugin was deleted by the author. GeniePlugin_exports, Requires http://www.squeaksource.com/genie.html Klatt_exports, Requires http://www.squeaksource.com/Speech.html RePlugin_exports, I think this is for the http://www.squeaksource.com/Regex.html StarSqueakPlugin_exports, Etoys SurfacePlugin_exports, Internal usage and exploited by various plugins like QuickTime for Sophie -- = = = = = ====================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http:// www.smalltalkconsulting.com = = = = = ======================================================================
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
does your spy support spy itself or detect that and does not spy dangerous guys. Stef On Dec 3, 2009, at 10:57 PM, Alexandre Bergel wrote:
hi John,
I have been working on a code instrumentation mechanism that use the "object as method" trick to get the runtime info. The "hello world" of the test coverage of Spy could be: 1- open a workspace, and doit: Gofer new squeaksource: 'Spy'; addPackage: 'ConfigurationOfSpy'; load. (Smalltalk at: #ConfigurationOfSpy) perform: #loadDefault.
2- doit: CoveragePackageSpy viewCoverageForPackagesMatching: 'Mondrian*'
3- after few seconds, a mondrian window will open. Pink inner boxes methods that are not covered by unit tests. The expression above (i)instruments all methods belonging to Mondrian, (ii) run all the test, (iii) open a simple mondrian visualization that tells whether a method is executed or not when running th test.
So far, it is good enough for small and controlled experiments. I would to do the coverage for the whole image, but I need some time for this. In 2 weeks I will be in holidays, I will therefore have time to work on this.
I also worked on a mechanism to extract type information from SUnit tests. Try: (TypingPackageSpy runTestAndGatherForPackagesMatching: 'Mondrian*') viewBasic
Cheers, Alexandre
On 3 Dec 2009, at 17:20, John M McIntosh wrote:
On 2009-12-03, at 7:33 AM, Alexandre Bergel wrote:
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
I used Spy on a number of projects, including Moose, Glamour, Mondrian, Seaside, ... But checking it on the whole image is worth trying.
Give me a couple of hours...
Thank you, allow me to give some clues about what internal plugins are in each macintosh VM (3.x, 4.x) series.
ADPCMCodecPlugin_exports, class ADPCMCodec AsynchFilePlugin_exports, class AsynchFile B2DPlugin_exports, class BallonEngine BitBltPlugin_exports, class BitBlt,WarpBlt DSAPrims_exports, class SHA1,SecureHashAlgorithm FFTPlugin_exports, class FFT FilePlugin_exports, class FileDirectory,StandardFileStream FloatArrayPlugin_exports, class BallonBuffer,FloatArray,MatrixTransform2x3 HostWindowPlugin_exports, class HostWindowProxy + subclasses InternetConfigPlugin_exports, class InternetConfiguration JoystickTabletPlugin_exports, class InputEventSensor JPEGReaderPlugin_exports, class JPEGReadWriter JPEGReadWriter2Plugin_exports, class JPEGReadWriter2 LargeIntegers_exports, Large positive/negative number usage... MacMenubarPlugin_exports, class HostSystemMenus + subclasses Matrix2x3Plugin_exports, class MatrixTransform2x3 MIDIPlugin_exports, class MidiPrimTester,SimpleMIDIPort MiscPrimitivePlugin_exports, class BitMap, ByteArray, ByteString,ByteSymbol,SampledSound SecurityPlugin_exports, class SecurityManager SerialPlugin_exports, class SerialPort SocketPlugin_exports, class NetNameResolver, Socket,SocketAddress SoundCodecPrims_exports, class GSMCodec SoundGenerationPlugin_exports, class FMSound,LoopedSampledSound,PluckedSound,ReverbSound,SampledSound SoundPlugin_exports, class SoundPlayer,SoundRecorder UUIDPlugin_exports, class UUID ZipPlugin_exports, class DeflateStream,FastInflateStream,NSAbstractDeflateStream +subclasses,ZipEncoder, ZipWriteStream
These plugins are built in, but not used directly by Pharo 1.0-10496-rc1dev09.11.4.image
B3DAcceleratorPlugin_exports, Requires Ballon 3d http://www.squeaksource.com/Balloon3D.html Squeak3D_exports,
BMPReadWriterPlugin_exports, Oddly the smalltalk usage of this plugin was deleted by the author. GeniePlugin_exports, Requires http://www.squeaksource.com/genie.html Klatt_exports, Requires http://www.squeaksource.com/Speech.html RePlugin_exports, I think this is for the http://www.squeaksource.com/Regex.html StarSqueakPlugin_exports, Etoys SurfacePlugin_exports, Internal usage and exploited by various plugins like QuickTime for Sophie -- = = = = = ====================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http:// www.smalltalkconsulting.com = = = = = ======================================================================
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
does your spy support spy itself or detect that and does not spy dangerous guys.
So far no. I would love to have Reflectivity for this... Cheers, Alexandre
Stef
On Dec 3, 2009, at 10:57 PM, Alexandre Bergel wrote:
hi John,
I have been working on a code instrumentation mechanism that use the "object as method" trick to get the runtime info. The "hello world" of the test coverage of Spy could be: 1- open a workspace, and doit: Gofer new squeaksource: 'Spy'; addPackage: 'ConfigurationOfSpy'; load. (Smalltalk at: #ConfigurationOfSpy) perform: #loadDefault.
2- doit: CoveragePackageSpy viewCoverageForPackagesMatching: 'Mondrian*'
3- after few seconds, a mondrian window will open. Pink inner boxes methods that are not covered by unit tests. The expression above (i)instruments all methods belonging to Mondrian, (ii) run all the test, (iii) open a simple mondrian visualization that tells whether a method is executed or not when running th test.
So far, it is good enough for small and controlled experiments. I would to do the coverage for the whole image, but I need some time for this. In 2 weeks I will be in holidays, I will therefore have time to work on this.
I also worked on a mechanism to extract type information from SUnit tests. Try: (TypingPackageSpy runTestAndGatherForPackagesMatching: 'Mondrian*') viewBasic
Cheers, Alexandre
On 3 Dec 2009, at 17:20, John M McIntosh wrote:
On 2009-12-03, at 7:33 AM, Alexandre Bergel wrote:
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests? I wondering how much of the image side code is actually used?
Part of the problem here in cross checking the list of internal plugins is that I need test cases to actually exploit the plugin code so that I can confirm it works, even to confirm it gets called.
this would be really interesting indeed. alex is spy working enough to see that?
I used Spy on a number of projects, including Moose, Glamour, Mondrian, Seaside, ... But checking it on the whole image is worth trying.
Give me a couple of hours...
Thank you, allow me to give some clues about what internal plugins are in each macintosh VM (3.x, 4.x) series.
ADPCMCodecPlugin_exports, class ADPCMCodec AsynchFilePlugin_exports, class AsynchFile B2DPlugin_exports, class BallonEngine BitBltPlugin_exports, class BitBlt,WarpBlt DSAPrims_exports, class SHA1,SecureHashAlgorithm FFTPlugin_exports, class FFT FilePlugin_exports, class FileDirectory,StandardFileStream FloatArrayPlugin_exports, class BallonBuffer,FloatArray,MatrixTransform2x3 HostWindowPlugin_exports, class HostWindowProxy + subclasses InternetConfigPlugin_exports, class InternetConfiguration JoystickTabletPlugin_exports, class InputEventSensor JPEGReaderPlugin_exports, class JPEGReadWriter JPEGReadWriter2Plugin_exports, class JPEGReadWriter2 LargeIntegers_exports, Large positive/negative number usage... MacMenubarPlugin_exports, class HostSystemMenus + subclasses Matrix2x3Plugin_exports, class MatrixTransform2x3 MIDIPlugin_exports, class MidiPrimTester,SimpleMIDIPort MiscPrimitivePlugin_exports, class BitMap, ByteArray, ByteString,ByteSymbol,SampledSound SecurityPlugin_exports, class SecurityManager SerialPlugin_exports, class SerialPort SocketPlugin_exports, class NetNameResolver, Socket,SocketAddress SoundCodecPrims_exports, class GSMCodec SoundGenerationPlugin_exports, class FMSound,LoopedSampledSound,PluckedSound,ReverbSound,SampledSound SoundPlugin_exports, class SoundPlayer,SoundRecorder UUIDPlugin_exports, class UUID ZipPlugin_exports, class DeflateStream,FastInflateStream,NSAbstractDeflateStream +subclasses,ZipEncoder, ZipWriteStream
These plugins are built in, but not used directly by Pharo 1.0-10496-rc1dev09.11.4.image
B3DAcceleratorPlugin_exports, Requires Ballon 3d http://www.squeaksource.com/Balloon3D.html Squeak3D_exports,
BMPReadWriterPlugin_exports, Oddly the smalltalk usage of this plugin was deleted by the author. GeniePlugin_exports, Requires http://www.squeaksource.com/genie.html Klatt_exports, Requires http://www.squeaksource.com/ Speech.html RePlugin_exports, I think this is for the http://www.squeaksource.com/Regex.html StarSqueakPlugin_exports, Etoys SurfacePlugin_exports, Internal usage and exploited by various plugins like QuickTime for Sophie -- = = = = = = = ==================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http:// www.smalltalkconsulting.com = = = = = = = ====================================================================
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
does your spy support spy itself or detect that and does not spy dangerous guys.
So far no. I would love to have Reflectivity for this...
Yes, will be available, uhm, at some point...
Let us know, I would love to give a try Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On Dec 3, 2009, at 7:58 AM, John McIntosh wrote:
Ok as I'm testing this new VM I"m wondering has anyone done a code coverage review of the Pharo image in relationship to the suite of available tests?
So Stefan Reichart (now at cmsbox.com) did his Master on test-smell detection. (see http://www.jot.fm/issues/issue_2007_10/paper12/ ) Part of this work was a test-coverage tool (this used multiple backends, methodwrappers, bytesurgeon and Persephone AST-interpratation using Philippe Marschall's work). We should really have a look at that again... (but fist we need to get the infrastructure in shape...) Having coverage analysis would help to clean up, too. I think a good part of the code in the image was executed last in 1997 or so... Marcus
participants (5)
-
Alexandre Bergel -
John M McIntosh -
John McIntosh
-
Marcus Denker -
Stéphane Ducasse