[Pharo-project] Proposal: coverage tests
Hi, what about to intrroduce tests that will test coverage of tests? Some improved version of the code below. Are there some tool for Smalltalk showing uncovered lines of code? There are such tools in Java world :-) One little problem is that this code do not work on Linux because of a VM error that cause test failures on CI server too. Cheers, Pavel | class test trace classTrace result | class := Time. test := TimeTest. trace := Dictionary new. thisContext runSimulated: [test suite run] contextAtEachStep: [ :current | | cls sel methods bytecodes | cls := current method methodClass. sel := current method selector. methods := trace at: cls ifAbsentPut: Dictionary new. bytecodes := methods at: sel ifAbsentPut: Set new. bytecodes add: current pc.] trace . result := String streamContents: [:s | classTrace := trace at: class ifAbsent: nil. classTrace ifNil: [ s nextPutAll: 'Class ', class name, ' not called'; cr. ] ifNotNil: [ s nextPutAll: class name; cr. class selectorsDo: [:sel | | m usedBytecodes allBytecodes | m := class >> sel. usedBytecodes := (classTrace at: sel ifAbsent: Set new) size. allBytecodes := 0. m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ].. s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll: (usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll: '%'; cr. ]]. ]. result..
Pavel, This is incredible ! This is why we love Smalltalk, in what other language could you write it like that ? I love it. Sven On 21 Feb 2012, at 16:03, Pavel Krivanek wrote:
Hi,
what about to intrroduce tests that will test coverage of tests? Some improved version of the code below. Are there some tool for Smalltalk showing uncovered lines of code? There are such tools in Java world :-) One little problem is that this code do not work on Linux because of a VM error that cause test failures on CI server too.
Cheers, Pavel
| class test trace classTrace result |
class := Time. test := TimeTest.
trace := Dictionary new.
thisContext runSimulated: [test suite run] contextAtEachStep: [ :current | | cls sel methods bytecodes | cls := current method methodClass. sel := current method selector. methods := trace at: cls ifAbsentPut: Dictionary new. bytecodes := methods at: sel ifAbsentPut: Set new. bytecodes add: current pc.]
trace .
result := String streamContents: [:s |
classTrace := trace at: class ifAbsent: nil. classTrace ifNil: [ s nextPutAll: 'Class ', class name, ' not called'; cr. ] ifNotNil: [ s nextPutAll: class name; cr. class selectorsDo: [:sel | | m usedBytecodes allBytecodes | m := class >> sel. usedBytecodes := (classTrace at: sel ifAbsent: Set new) size. allBytecodes := 0. m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ].. s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll: (usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll: '%'; cr. ]]. ]. result..
That is just AMAZINGLY COOL. I cannot believe we can do that in 5 lines of code. WOW. Why nobody wrote a better test coverage report wich such backend? Alex, does you stuff do something with that? cheers On Tue, Feb 21, 2012 at 4:03 PM, Pavel Krivanek <pavel.krivanek@gmail.com>wrote:
Hi,
what about to intrroduce tests that will test coverage of tests? Some improved version of the code below. Are there some tool for Smalltalk showing uncovered lines of code? There are such tools in Java world :-) One little problem is that this code do not work on Linux because of a VM error that cause test failures on CI server too.
Cheers, Pavel
| class test trace classTrace result |
class := Time. test := TimeTest.
trace := Dictionary new.
thisContext runSimulated: [test suite run] contextAtEachStep: [ :current | | cls sel methods bytecodes | cls := current method methodClass. sel := current method selector. methods := trace at: cls ifAbsentPut: Dictionary new. bytecodes := methods at: sel ifAbsentPut: Set new. bytecodes add: current pc.]
trace .
result := String streamContents: [:s |
classTrace := trace at: class ifAbsent: nil. classTrace ifNil: [ s nextPutAll: 'Class ', class name, ' not called'; cr. ] ifNotNil: [ s nextPutAll: class name; cr. class selectorsDo: [:sel | | m usedBytecodes allBytecodes | m := class >> sel. usedBytecodes := (classTrace at: sel ifAbsent: Set new) size. allBytecodes := 0. m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ].. s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll: (usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll: '%'; cr. ]]. ]. result..
-- Mariano http://marianopeck.wordpress.com
That is just AMAZINGLY COOL. I cannot believe we can do that in 5 lines of code. WOW. Why nobody wrote a better test coverage report wich such backend? Alex, does you stuff do something with that?
Hapao does not support statement coverage yet. We are currently working on it. We will announce it soon. Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
Well done, a real *masterpieche*! :-) Depreciated (marked as depreciated) classes could / should be moved either into a compatibility layer (sport) or, if only a few "apps" are using it, into the app itself, being slightly renamed for keeping up compatibility. This should work in a similar way by first identifying concerned apps, automatically moving - while renaming classes - into the app, exporting as new package. Does that make sense? How would u solve that? tnx in advance, Guido Stepken Am 21.02.2012 16:03 schrieb "Pavel Krivanek" <pavel.krivanek@gmail.com>:
Hi,
what about to intrroduce tests that will test coverage of tests? Some improved version of the code below. Are there some tool for Smalltalk showing uncovered lines of code? There are such tools in Java world :-) One little problem is that this code do not work on Linux because of a VM error that cause test failures on CI server too.
Cheers, Pavel
| class test trace classTrace result |
class := Time. test := TimeTest.
trace := Dictionary new.
thisContext runSimulated: [test suite run] contextAtEachStep: [ :current | | cls sel methods bytecodes | cls := current method methodClass. sel := current method selector. methods := trace at: cls ifAbsentPut: Dictionary new. bytecodes := methods at: sel ifAbsentPut: Set new. bytecodes add: current pc.]
trace .
result := String streamContents: [:s |
classTrace := trace at: class ifAbsent: nil. classTrace ifNil: [ s nextPutAll: 'Class ', class name, ' not called'; cr. ] ifNotNil: [ s nextPutAll: class name; cr. class selectorsDo: [:sel | | m usedBytecodes allBytecodes | m := class >> sel. usedBytecodes := (classTrace at: sel ifAbsent: Set new) size. allBytecodes := 0. m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ].. s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll: (usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll: '%'; cr. ]]. ]. result..
Hi all! I seem to recall a small discussion with Peter Hugosson-Miller at ESUG in Brest (walking to lunch :)) in which he told me a bit about the tests they used where he worked. IIRC a LOT of them were indeed "meta tests" that were testing properties of the source code itself (metrics, lints etc). Peter, any more ideas in this direction you can share? If coupled with "background execution" (or hey, in a forked image through OSProcess so we don't mess up the one we work in) and immediate feedback in browser tools this is "winner stuff". regards, Göran
This is the way how to get the the source code that belongs to uncovered bytecodes (expecting "trace" from the previous code) class := Time. selector := #print24:showSeconds:on:. m := class >> selector. usedBytecodes := ((trace at: class) at: selector ifAbsent: Set new). allBytecodes := Set new. m symbolicLinesDo: [:pc :lineForPC | allBytecodes add: pc ]. unusedBytecodes := allBytecodes copyWithoutAll: usedBytecodes. dm := m debuggerMap. source := (class sourceCodeAt: selector) asText. unusedBytecodes do: [:pc | range := dm rangeForPC: pc contextIsActiveContext: false. source makeBoldFrom: range first to: range last.]. source asMorph openInWindow. Cheers, -- Pavel On Tue, Feb 21, 2012 at 4:03 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
Hi,
what about to intrroduce tests that will test coverage of tests? Some improved version of the code below. Are there some tool for Smalltalk showing uncovered lines of code? There are such tools in Java world :-) One little problem is that this code do not work on Linux because of a VM error that cause test failures on CI server too.
Cheers, Â Pavel
| class test trace classTrace result  |
class := Time. test := TimeTest.
trace := Dictionary new.
thisContext runSimulated: [test suite run] Â contextAtEachStep: [ :current | Â Â Â Â | cls sel methods bytecodes | Â Â Â Â cls := current method methodClass. Â Â Â Â sel := current method selector. Â Â Â Â methods := trace at: cls ifAbsentPut: Dictionary new. Â Â Â Â bytecodes := methods at: sel ifAbsentPut: Set new. Â Â Â Â bytecodes add: current pc.]
trace .
result := String streamContents: [:s |
classTrace := trace at: class ifAbsent: nil. classTrace     ifNil: [         s nextPutAll: 'Class ', class name, ' not called'; cr. ]     ifNotNil: [         s nextPutAll: class name; cr.         class selectorsDo: [:sel |             | m usedBytecodes allBytecodes |             m := class >> sel.             usedBytecodes := (classTrace at: sel ifAbsent: Set new) size.             allBytecodes := 0.             m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ]..             s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll: (usedBytecodes  /  allBytecodes * 100) asFloat asString; nextPutAll: '%'; cr. ]]. ]. result..
participants (6)
-
Alexandre Bergel -
Guido Stepken -
Göran Krampe -
Mariano Martinez Peck -
Pavel Krivanek -
Sven Van Caekenberghe