[Pharo-project] Spell check test cases
Hi Lukas, I created some test cases for internal spell check, at http://www.squeaksource.com/RBBugFixes Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value. There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests. please let me know if these make sense. Thanks, ...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4288633.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Hi Stan,
Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked. Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9 - fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used: RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem? Lukas -- Lukas Renggli http://www.lukas-renggli.ch
Lukas Renggli wrote:
Hi Stan,
Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked.
Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9
- fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used:
RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments
Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem?
Lukas
-- Lukas Renggli http://www.lukas-renggli.ch
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Glad you like the tests. In my version of RBSpellChecker>>normalize, I also commented out the line: "or: [ input peek = $' ]" "this causes false positives eg 'OK' in class comments" OK returns an error on k, RB b etc. So that's what causes the two cases on a Linux box (fairly sure). I'm guessing you have letters k i b and t in the Mac dictionary, or it specifically doesn't check one letter words? As for the Mac, I didn't think of that. I'd say you have 'Smalltalk' in your Mac dictionary. I chose that as the error Pool Dictionary, to avoid polluting the system with a dictionary and risk it failing to clean up. So we could temporarily add a Pool Dictionary in setUp/tearDown, which was how I originally did it. BTW, it looks like RBPoolVariableNameSpellingRule should be RBPoolDictionaryNameSpellingRule, as that's what it looks at? Cheers, ...Stan PS just noticed that code critics browser doesn't highlight Method protocols, even though it does find the typos. Could be related, but I don't think so. -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4288993.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Glad you like the tests.
In my version of RBSpellChecker>>normalize, I also commented out the line: Â Â Â Â "or: [ input peek = $' ]" "this causes false positives eg 'OK' in class comments"
That change breaks word like don't Smalltalk's etc. I see that there are a few other cases where you don't want any $'. So I removed it for now.
OK returns an error on k, RB b etc.
So that's what causes the two cases on a Linux box (fairly sure). I'm guessing you have letters k i b and t in the Mac dictionary, or it specifically doesn't check one letter words?
RBSpellChecker>>#normalize: removes one letter words, so I don't think this can be the error. If you show the individual words that are checked against the dictionary on the transcript, you will see that they all look pretty ok.
BTW, it looks like RBPoolVariableNameSpellingRule should be RBPoolDictionaryNameSpellingRule, as that's what it looks at?
Ok, I renamed that. Probably this rule isn't really needed anyway ... Lukas -- Lukas Renggli http://www.lukas-renggli.ch
Lukas Renggli wrote:
Glad you like the tests.
In my version of RBSpellChecker>>normalize, I also commented out the line: Â Â Â Â "or: [ input peek = $' ]" "this causes false positives eg 'OK' in class comments"
That change breaks word like
don't Smalltalk's etc.
I see that there are a few other cases where you don't want any $'. So I removed it for now.
OK returns an error on k, RB b etc.
So that's what causes the two cases on a Linux box (fairly sure). I'm guessing you have letters k i b and t in the Mac dictionary, or it specifically doesn't check one letter words?
RBSpellChecker>>#normalize: removes one letter words, so I don't think this can be the error. If you show the individual words that are checked against the dictionary on the transcript, you will see that they all look pretty ok.
BTW, it looks like RBPoolVariableNameSpellingRule should be RBPoolDictionaryNameSpellingRule, as that's what it looks at?
Ok, I renamed that. Probably this rule isn't really needed anyway ...
Lukas
-- Lukas Renggli http://www.lukas-renggli.ch
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hi Lukas, I think you didn't pick up the other fix in RBBugFixes RBLiteralValuesSpellingRule>>initialize super initialize. matcher := RBParseTreeSearcher new. matcher matches: '`#literal' do: [ :node :answer | | nodeValue | nodeValue := node value. (nodeValue isString or: [ nodeValue isSymbol or: [ nodeValue isCollection ] ]) ifTrue: [ answer add: nodeValue ]. answer ] Currently it checks the node, not the node value. When I load this method, all tests run green.
RBSpellChecker>>#normalize: removes one letter words, so I don't think this can be the error.
The tests are on 'RB' 'OK' etc, in the quotes in class comments. Leading capitals are stripped off. If $' is passed through, the check gets made on K', and as it is 2 characters doesn't get disallowed. That's why the tests need the pass through of ' taken out at the moment. If instead, where you store the result: result add: output contents if you strip out trailing $' before storing, this would let you verify don't and smalltalk's as well, I think. Then you'd need a different test from the position in the stream for size = 1. I only hit this edge case because I had a class comment about namespace related prefixes- 'WA' 'RB' etc. So now 'OK' is not checked, under the 'strip out leading capitals' scenario. At some point it might be nice to check these as well - eg WARenderCanvas rather than WBRenderCanvas. YAGNI, NASA, other abbreviations. Code Critics browser still doesn't show Method Protocols in bold. Cheers, ...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4290734.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
I think you didn't pick up the other fix in RBBugFixes
RBLiteralValuesSpellingRule>>initialize     super initialize.     matcher := RBParseTreeSearcher new.     matcher matches: '`#literal'         do:             [ :node :answer |             | nodeValue |             nodeValue := node value.             (nodeValue isString or: [ nodeValue isSymbol or: [ nodeValue isCollection ] ]) ifTrue:                 [ answer add: nodeValue ].             answer ]
Thank you. Indeed, I did not see this change. I implemented it slightly different to ignore ByteArrays and to flatten out nested literal array collections.
The tests are on 'RB' 'OK' etc, in the quotes in class comments. Leading capitals are stripped off. If $' is passed through, the check gets made on K', and as it is 2 characters doesn't get disallowed. That's why the tests need the pass through of ' taken out at the moment. If instead, Â where you store the result: Â result add: output contents if you strip out trailing $' before storing, this would let you verify don't and smalltalk's as well, I think. Then you'd need a different test from the position in the stream for size = 1.
I only hit this edge case because I had a class comment about namespace related prefixes- 'WA' 'RB' etc.
So now 'OK' is not checked, under the 'strip out leading capitals' scenario. At some point it might be nice to check these as well - eg WARenderCanvas rather than WBRenderCanvas. YAGNI, NASA, other abbreviations.
Code Critics browser still doesn't show Method Protocols in bold.
In bold? They are highlighted here. Lukas -- Lukas Renggli http://www.lukas-renggli.ch
Lukas: if you then copy these tests in RB repository let me know so that I can included in Dev image. cheers mariano On Tue, Jan 12, 2010 at 11:57 AM, Lukas Renggli <renggli@gmail.com> wrote:
I think you didn't pick up the other fix in RBBugFixes
RBLiteralValuesSpellingRule>>initialize super initialize. matcher := RBParseTreeSearcher new. matcher matches: '`#literal' do: [ :node :answer | | nodeValue | nodeValue := node value. (nodeValue isString or: [ nodeValue isSymbol or: [ nodeValue isCollection ] ]) ifTrue: [ answer add: nodeValue ]. answer ]
Thank you. Indeed, I did not see this change. I implemented it slightly different to ignore ByteArrays and to flatten out nested literal array collections.
The tests are on 'RB' 'OK' etc, in the quotes in class comments. Leading capitals are stripped off. If $' is passed through, the check gets made on K', and as it is 2 characters doesn't get disallowed. That's why the tests need the pass through of ' taken out at the moment. If instead, where you store the result: result add: output contents if you strip out trailing $' before storing, this would let you verify don't and smalltalk's as well, I think. Then you'd need a different test from the position in the stream for size = 1.
I only hit this edge case because I had a class comment about namespace related prefixes- 'WA' 'RB' etc.
So now 'OK' is not checked, under the 'strip out leading capitals' scenario. At some point it might be nice to check these as well - eg WARenderCanvas rather than WBRenderCanvas. YAGNI, NASA, other abbreviations.
Code Critics browser still doesn't show Method Protocols in bold.
In bold? They are highlighted here.
Lukas
-- Lukas Renggli http://www.lukas-renggli.ch
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Lukas Renggli wrote:
Hi Stan,
Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked.
Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9
- fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used:
RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments
Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem?
Lukas
Hi, there are still $' present. Don't know if it's possible to strip them out earlier again. The following methods again strip leading and trailing '. They validate correctly, including 'don''t'. They are in Squeaksource RBBugFixes. So is a new version of the tests, that test spelling of 'don''t'. It also checks all the tests are run- one got lost in transcription last time. Also, the name of the Pool Dictionary test got reverted. ...Stan RBSpellChecker>>normalize: aString "Filter out non alphabetical characters, remove prefixes as commonly found in class names, split camel case expressions and filter out one character words." | result input output activeString | result := Set new. input := aString readStream. output := WriteStream on: (String new: 128). [ input atEnd ] whileFalse: [ [ input atEnd not and: [ input peek isLetter or: [ input peek = $' ] ] ] whileTrue: [ output nextPut: input next ]. output position = 0 ifTrue: [ input next ] ifFalse: [ | stream | stream := output contents readStream. [ stream atEnd ] whileFalse: [ output reset; nextPut: stream next. [ stream atEnd not and: [ stream peek isLowercase or: [ stream peek = $' ] ] ] whileTrue: [ output nextPut: stream next ]. "at this point we have allowed through $' to allow don't. We can also have leading or trailing $', if a comment contains eg 'WB' or 'NASA' " activeString := self copyWithoutLeadingTrailingSingleQuotes: output contents . activeString size > 1 ifTrue: [ result add: activeString ] ] ]. output reset ]. ^ result RBSpellChecker>>copyWithoutLeadingTrailingSingleQuotes: aByteString | startPosition endPosition | startPosition := 1. [ (aByteString at: startPosition) = $' ] whileTrue: [ startPosition := startPosition + 1. startPosition > aByteString size ifTrue: [ ^ '' ] ]. endPosition := aByteString size. [ (aByteString at: endPosition) = $' ] whileTrue: [ endPosition := endPosition - 1. endPosition < 1 ifTrue: [ ^ '' ] ]. ^ aByteString copyFrom: startPosition to: endPosition -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4292584.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Hi: I want to include this tests in next PharoDev as I did with the rest of the projects. The problem is known problem with MC and packages. If I load the packages from http://www.squeaksource.com/RBBugFixes It has the packages: Refactoring-Spelling-BugFixes Refactoring-Tests-Spelling The first one is interpreted as a part of the package Refactoring-Spelling and the second one as Refactoring-Tests Thus, those packages are "dirty" in compare to the repository: http://www.squeaksource.com/rb This is really problematic. A quick solution is to rename both packages to something like this: BugFixes-Refactoring-Spelling Refactoring-SpellingTests Any other has a better solution? Thanks Mariano On Tue, Jan 12, 2010 at 5:14 PM, Stan Shepherd <stan.shepherd414@gmail.com>wrote:
Lukas Renggli wrote:
Hi Stan,
Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked.
Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9
- fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used:
RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments
Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem?
Lukas
Hi, there are still $' present. Don't know if it's possible to strip them out earlier again.
The following methods again strip leading and trailing '. They validate correctly, including 'don''t'. They are in Squeaksource RBBugFixes. So is a new version of the tests, that test spelling of 'don''t'. It also checks all the tests are run- one got lost in transcription last time.
Also, the name of the Pool Dictionary test got reverted.
...Stan
RBSpellChecker>>normalize: aString "Filter out non alphabetical characters, remove prefixes as commonly found in class names, split camel case expressions and filter out one character words."
| result input output activeString | result := Set new. input := aString readStream. output := WriteStream on: (String new: 128). [ input atEnd ] whileFalse: [ [ input atEnd not and: [ input peek isLetter or: [ input peek = $' ] ] ] whileTrue: [ output nextPut: input next ]. output position = 0 ifTrue: [ input next ] ifFalse: [ | stream | stream := output contents readStream. [ stream atEnd ] whileFalse: [ output reset; nextPut: stream next. [ stream atEnd not and: [ stream peek isLowercase or: [ stream peek = $' ] ] ] whileTrue: [ output nextPut: stream next ].
"at this point we have allowed through $' to allow don't. We can also have leading or trailing $', if a comment contains eg 'WB' or 'NASA' " activeString := self copyWithoutLeadingTrailingSingleQuotes: output contents . activeString size > 1 ifTrue: [ result add: activeString ] ] ]. output reset ]. ^ result
RBSpellChecker>>copyWithoutLeadingTrailingSingleQuotes: aByteString | startPosition endPosition | startPosition := 1. [ (aByteString at: startPosition) = $' ] whileTrue: [ startPosition := startPosition + 1. startPosition > aByteString size ifTrue: [ ^ '' ] ]. endPosition := aByteString size. [ (aByteString at: endPosition) = $' ] whileTrue: [ endPosition := endPosition - 1. endPosition < 1 ifTrue: [ ^ '' ] ]. ^ aByteString copyFrom: startPosition to: endPosition
-- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4292584.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Mariano Martinez Peck wrote:
Hi: I want to include this tests in next PharoDev as I did with the rest of the projects.
The problem is known problem with MC and packages. If I load the packages from
http://www.squeaksource.com/RBBugFixes
It has the packages:
Refactoring-Spelling-BugFixes Refactoring-Tests-Spelling
The first one is interpreted as a part of the package Refactoring-Spelling and the second one as Refactoring-Tests Thus, those packages are "dirty" in compare to the repository: http://www.squeaksource.com/rb
This is really problematic. A quick solution is to rename both packages to something like this:
BugFixes-Refactoring-Spelling Refactoring-SpellingTests
Any other has a better solution?
Thanks
Mariano
On Tue, Jan 12, 2010 at 5:14 PM, Stan Shepherd <stan.shepherd414@gmail.com>wrote:
Lukas Renggli wrote:
Hi Stan,
Some are failing, as there are $' characters being passed through from the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked.
Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9
- fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used:
RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments
Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem?
Lukas
Hi, there are still $' present. Don't know if it's possible to strip them out earlier again.
The following methods again strip leading and trailing '. They validate correctly, including 'don''t'. They are in Squeaksource RBBugFixes. So is a new version of the tests, that test spelling of 'don''t'. It also checks all the tests are run- one got lost in transcription last time.
Also, the name of the Pool Dictionary test got reverted.
...Stan
RBSpellChecker>>normalize: aString "Filter out non alphabetical characters, remove prefixes as commonly found in class names, split camel case expressions and filter out one character words."
| result input output activeString | result := Set new. input := aString readStream. output := WriteStream on: (String new: 128). [ input atEnd ] whileFalse: [ [ input atEnd not and: [ input peek isLetter or: [ input peek = $' ] ] ] whileTrue: [ output nextPut: input next ]. output position = 0 ifTrue: [ input next ] ifFalse: [ | stream | stream := output contents readStream. [ stream atEnd ] whileFalse: [ output reset; nextPut: stream next. [ stream atEnd not and: [ stream peek isLowercase or: [ stream peek = $' ] ] ] whileTrue: [ output nextPut: stream next ].
"at this point we have allowed through $' to allow don't. We can also have leading or trailing $', if a comment contains eg 'WB' or 'NASA' " activeString := self copyWithoutLeadingTrailingSingleQuotes: output contents . activeString size > 1 ifTrue: [ result add: activeString ] ] ]. output reset ]. ^ result
RBSpellChecker>>copyWithoutLeadingTrailingSingleQuotes: aByteString | startPosition endPosition | startPosition := 1. [ (aByteString at: startPosition) = $' ] whileTrue: [ startPosition := startPosition + 1. startPosition > aByteString size ifTrue: [ ^ '' ] ]. endPosition := aByteString size. [ (aByteString at: endPosition) = $' ] whileTrue: [ endPosition := endPosition - 1. endPosition < 1 ifTrue: [ ^ '' ] ]. ^ aByteString copyFrom: startPosition to: endPosition
Hi Mariano, probably best to wait til the tests are in the main rb repository. At the moment they would fail, as there is still a discrepancy with what comes through from the parser. ...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4406315.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Thanks Stan for the warning. Ok, I will then wait yet until everything is in rb repository and tests are green. Cheers Mariano On Sun, Jan 17, 2010 at 12:31 AM, Stan Shepherd <stan.shepherd414@gmail.com>wrote:
Mariano Martinez Peck wrote:
Hi: I want to include this tests in next PharoDev as I did with the rest of the projects.
The problem is known problem with MC and packages. If I load the packages from
http://www.squeaksource.com/RBBugFixes
It has the packages:
Refactoring-Spelling-BugFixes Refactoring-Tests-Spelling
The first one is interpreted as a part of the package
Refactoring-Spelling
and the second one as Refactoring-Tests Thus, those packages are "dirty" in compare to the repository: http://www.squeaksource.com/rb
This is really problematic. A quick solution is to rename both packages to something like this:
BugFixes-Refactoring-Spelling Refactoring-SpellingTests
Any other has a better solution?
Thanks
Mariano
On Tue, Jan 12, 2010 at 5:14 PM, Stan Shepherd <stan.shepherd414@gmail.com>wrote:
Lukas Renggli wrote:
Hi Stan,
Some are failing, as there are $' characters being passed through
from
the parser. Also, the literals check sees to be checking the node, rather than the node value.
You are right, the code that extracted arguments and temps unnecessarily put the names into $' characters. This is totally wrong and also breaks the highlighting in the browser. Instead of changing the tokenizer (as you proposed) I fixed the places where the nodes were collected. Like this there should be no more $' characters in the strings that are checked.
Name: Refactoring-Spelling-lr.10 Author: lr Time: 12 January 2010, 12:16:16 am UUID: b40774a4-8371-42f4-883c-469d6fe1e020 Ancestors: Refactoring-Spelling-lr.9
- fixed the bug pointed out by Stan Shepherd that temps and arguments were not properly checked - this also fixes the issue of not highlighting typos in temps and arguments (which I looked into earlier today, but couldn't figure out the real cause)
There are some suggested changes in the same repository, that make the tests run green, and don't appear to affect the rest of Refactoring-Tests.
The tests are really cool. I would like to add them to the main repository, but two are broken depending on what checker is used:
RBMacSpellChecker: #testLiteralValues and #testPoolVariable RBInternalSpellChecker: #testLiteralValues and #testClassComments
Maybe that could be fixed by always using the internal one for the tests, but then I don't understand why the tests break. Does my fix not solve the whole problem?
Lukas
Hi, there are still $' present. Don't know if it's possible to strip them out earlier again.
The following methods again strip leading and trailing '. They validate correctly, including 'don''t'. They are in Squeaksource RBBugFixes. So is a new version of the tests, that test spelling of 'don''t'. It also checks all the tests are run- one got lost in transcription last time.
Also, the name of the Pool Dictionary test got reverted.
...Stan
RBSpellChecker>>normalize: aString "Filter out non alphabetical characters, remove prefixes as commonly found in class names, split camel case expressions and filter out one character words."
| result input output activeString | result := Set new. input := aString readStream. output := WriteStream on: (String new: 128). [ input atEnd ] whileFalse: [ [ input atEnd not and: [ input peek isLetter or: [ input peek = $' ] ] ] whileTrue: [ output nextPut: input next ]. output position = 0 ifTrue: [ input next ] ifFalse: [ | stream | stream := output contents readStream. [ stream atEnd ] whileFalse: [ output reset; nextPut: stream next. [ stream atEnd not and: [ stream peek isLowercase or: [ stream peek = $' ] ] ] whileTrue: [ output nextPut: stream next ].
"at this point we have allowed through $' to allow don't. We can also have leading or trailing $', if a comment contains eg 'WB' or 'NASA' " activeString := self copyWithoutLeadingTrailingSingleQuotes: output contents . activeString size > 1 ifTrue: [ result add: activeString ] ] ]. output reset ]. ^ result
RBSpellChecker>>copyWithoutLeadingTrailingSingleQuotes: aByteString | startPosition endPosition | startPosition := 1. [ (aByteString at: startPosition) = $' ] whileTrue: [ startPosition := startPosition + 1. startPosition > aByteString size ifTrue: [ ^ '' ] ]. endPosition := aByteString size. [ (aByteString at: endPosition) = $' ] whileTrue: [ endPosition := endPosition - 1. endPosition < 1 ifTrue: [ ^ '' ] ]. ^ aByteString copyFrom: startPosition to: endPosition
Hi Mariano, probably best to wait til the tests are in the main rb repository. At the moment they would fail, as there is still a discrepancy with what comes through from the parser.
...Stan
-- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4406315.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hi Lukas, did you get a chance to look at this? Refactoring-Spelling-Tests-StanShepherd.7.mcz in http://www.squeaksource.com/RBBugFixes/ is the present version of the tests. They will still need the tweaks in your parser or normalizer to run green. ...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4434232.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
I wonder if we shouldn't somehow trigger a different tokenizer depending on what we check, instead of using some magic? We have all this information. For example class names, variables, selectors, ... and comments, strings, ... could use a different tokenizer that knows about the specifics? Lukas 2010/1/21 Stan Shepherd <stan.shepherd414@gmail.com>:
Hi Lukas, did you get a chance to look at this?
Refactoring-Spelling-Tests-StanShepherd.7.mcz in http://www.squeaksource.com/RBBugFixes/ is the present version of the tests.
They will still need the tweaks in your parser or normalizer to run green.
...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4434232.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Lukas Renggli http://www.lukas-renggli.ch
Absolutely. Those hacks were to demonstrate that the tests were working, not as the mechanism to get there. Let me know if there's anything you need added/changed in the tests. ...Stan -- View this message in context: http://n2.nabble.com/Spell-check-test-cases-tp4288633p4439243.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
participants (3)
-
Lukas Renggli -
Mariano Martinez Peck -
Stan Shepherd