- Pharo-dev - lists.pharo.org

Software maintenance costs in the real world (Re: ftp repositories broken)

by David T. Lewis

For students of software engineering, I offer this as an example of one of the most common and costly types of error that you will encounter in real life. A small error may be introduced early in the development process, possibly slipping through the cracks through lack of review (as was the case in this example). The change is motivated by good intentions, but some aspect of the overall system design is overlooked. Once the mistake has been made, it becomes increasingly difficult to change as time goes by, and increasingly costly to support as new people need to understand the issue and work around its consequences. Over the life of the software, the accumulated costs of effort to understand and address the problem (typically budgeted separately under "software maintenance") far exceed the costs associated with the original change (typically budgeted as "development"). In addition to cost of maintenance, this kind of error can also defeat the intentions of the original developer. In the example we have here, the original change was motivated by an interest in making the system cleaner and easier to understand. But nearly a decade later, we see workarounds such as the #translatedPrimitives method below that are far worse than the original problem that someone was trying to solve. For the historical background on this particular exammple, see the earlier post by Henrik Sperre Johansen in this thread. Dave p.s. I am not a software engineer :-) On Tue, Apr 04, 2017 at 06:45:27PM +0200, Esteban Lorenzano wrote: > > > On 4 Apr 2017, at 18:01, Ben Coman <btc(a)openInWorld.com> wrote: > > > > > > > > On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov <dionisiydk(a)gmail.com <mailto:dionisiydk@gmail.com>> wrote: > > > > > > > > > 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano <estebanlm(a)gmail.com <mailto:estebanlm@gmail.com>>: > > >> > > >> no it doesn't: squeak does not have that primitive, that???s why it does not fails :) > > > > > > > > > I think you are wrong because I check it carefully in Squeak and I found it on github https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=???&q=primiti… <https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=%E2%9C%93&q=p…>. > > still is not there, because is not the same primitive :) > look, this is the way where it is generated: > > translatedPrimitives > "an assorted list of various primitives" > (ByteString compiledMethodAt: #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil: > [^#("Pharo uses findSubstringViaPrimitive:in:startingAt:matchTable:" > (Bitmap compress:toByteArray:) > (Bitmap decompress:fromByteArray:at:) > (Bitmap encodeBytesOf:in:at:) > (Bitmap encodeInt:in:at:) > (ByteString compare:with:collated:) > (ByteString translate:from:to:table:) > (ByteString findFirstInString:inSet:startingAt:) > (ByteString indexOfAscii:inString:startingAt:) > (String findSubstringViaPrimitive:in:startingAt:matchTable:) > (ByteArray hashBytes:startingWith:) > (SampledSound convert8bitSignedFrom:to16Bit:))]. > ^#( > (Bitmap compress:toByteArray:) > (Bitmap decompress:fromByteArray:at:) > (Bitmap encodeBytesOf:in:at:) > (Bitmap encodeInt:in:at:) > (ByteString compare:with:collated:) > (ByteString translate:from:to:table:) > (ByteString findFirstInString:inSet:startingAt:) > (ByteString indexOfAscii:inString:startingAt:) > (ByteString findSubstring:in:startingAt:matchTable:) > (ByteArray hashBytes:startingWith:) > (SampledSound convert8bitSignedFrom:to16Bit:) > ) > > means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: while squeak uses #findSubstring:in:startingAt:matchTable:. > Now, there is a bug there (because source is in String and not in ByteString), and I can fix that by asking correctly, but: > > 1) this means that sources for pharo and squeak will be different and pharo needs to be generated in pharo. This is contrary to what we are trying to do with the VM: we want all sources to be the same, and differences should come from compilation flags and/or the presence of certain plugins (this is because is a lot easier to debug the VM like that). > 2) I wonder??? why we are using this primitive an squeak is not? maybe we need to deprecate the use? > > > I believe Esteban's point Its not so much whether the VM has it > > (since they are essentially the same VM + extra C-libs for Pharo) > > but whether the Squeak Image is using it. > > > > Squeak does use primitiveFindSubstring, but a difference is attached to ByteString rather than String. > > That was added to Squeak 2015.05.01. Squeak String seems to have never used the primitive (since 2005). > > this is the important thing, I think??? so??? why can???t we use that instead the other one? > what???s the difference? > > cheers! > Esteban > > > > > > A diff shows some differences in implementation.. > > https://www.diffchecker.com/CmBJZjB6 <https://www.diffchecker.com/CmBJZjB6> > > which anyway wouldn't account for John's observations. > > > > cheers -ben >

April 8, 2017

Re: [Pharo-dev] SmallInteger #printString versus #printOn:

by Nicolas Cellier

Hmm forget about this, wrong thread... 2017-04-08 13:58 GMT+02:00 Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com>: > Ah, but now that I've put a > (selector == #unwindTo: and: [ self name = 'Context']) ifTrue:[self > halt: 'debug me']. > I can't reproduce the problem related to removal of Context methods!!! > > Instead, I've got the 'Merging Kernel-eem.1078' window with all the > MethodContext->Context renaming in conflict > I already reported that in another thread where I told that the upgrade > went "smoothly" for both my 32 & 64 bits images. > I don't know if it's related to package cache, or .mcd, or ??? > > It might also be that package ancestry is not correct du to overwriting of > some packages, otherwise we should not get a merge window, but well... > > Ah, but now I think I understand: it's just because I have pending changes > in Kernel due to the addition of halt: above (same for my images which > usually have some unpublished experiments). > We then trigger a merge rather than a load. And the merge sees conflict > probably because of broken ancestry (the GUID are not corresponding), but > once resolved manually, it processes smoothly without removing Context > methods... > > I can't instrument code that easily... This will have to wait for another > time slot :( > > 2017-04-08 13:41 GMT+02:00 Nicolas Cellier <nicolas.cellier.aka.nice@ > gmail.com>: > >> >> >> 2017-04-08 6:38 GMT+02:00 Ben Coman <btc(a)openinworld.com>: >> >>> I was going to slip this example into the welcome help, >>> >>> 1 printString [debugIt] >>> >>> and I'm confronted by SmallInteger>>printString >>> >>> printString >>> "Highly optimized version for base 10 >>> and that we know it is a SmallInteger." >>> | integer next result len | >>> self = 0 ifTrue: [^'0']. >>> self < 0 ifTrue: [^'-', self negated printString]. >>> len := self decimalDigitLength. >>> result := String new: len. >>> integer := self. >>> len to: 1 by: -1 do: [:i | >>> next := integer // 10. >>> result byteAt: i put: 48 + (integer - (next * 10)). >>> integer := next]. >>> ^result >>> >>> >>> and I gather from other discussions this should(?) belong in >>> SmallInteger>>printOn: >>> Or is there a large benefit to avoiding the #printStringLimitedTo: >>> machinery ? >>> >>> cheers -ben >>> >> >> It should be printOn: generally, except that printOn: requires creating a >> Stream, and requires passing thru Stream indirection for writing into the >> target String. >> >> As the comment tells, the sole reason for this implementation is "Highly >> optimized version". So consider that it's an exception to the rules. Maybe >> this kind of decision could be periodically re-examined in the light of >> performance measurements (I'm thinking of Sista)? >> > >

April 8, 2017

Re: [Pharo-dev] SmallInteger #printString versus #printOn:

by Nicolas Cellier

Ah, but now that I've put a (selector == #unwindTo: and: [ self name = 'Context']) ifTrue:[self halt: 'debug me']. I can't reproduce the problem related to removal of Context methods!!! Instead, I've got the 'Merging Kernel-eem.1078' window with all the MethodContext->Context renaming in conflict I already reported that in another thread where I told that the upgrade went "smoothly" for both my 32 & 64 bits images. I don't know if it's related to package cache, or .mcd, or ??? It might also be that package ancestry is not correct du to overwriting of some packages, otherwise we should not get a merge window, but well... Ah, but now I think I understand: it's just because I have pending changes in Kernel due to the addition of halt: above (same for my images which usually have some unpublished experiments). We then trigger a merge rather than a load. And the merge sees conflict probably because of broken ancestry (the GUID are not corresponding), but once resolved manually, it processes smoothly without removing Context methods... I can't instrument code that easily... This will have to wait for another time slot :( 2017-04-08 13:41 GMT+02:00 Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com>: > > > 2017-04-08 6:38 GMT+02:00 Ben Coman <btc(a)openinworld.com>: > >> I was going to slip this example into the welcome help, >> >> 1 printString [debugIt] >> >> and I'm confronted by SmallInteger>>printString >> >> printString >> "Highly optimized version for base 10 >> and that we know it is a SmallInteger." >> | integer next result len | >> self = 0 ifTrue: [^'0']. >> self < 0 ifTrue: [^'-', self negated printString]. >> len := self decimalDigitLength. >> result := String new: len. >> integer := self. >> len to: 1 by: -1 do: [:i | >> next := integer // 10. >> result byteAt: i put: 48 + (integer - (next * 10)). >> integer := next]. >> ^result >> >> >> and I gather from other discussions this should(?) belong in >> SmallInteger>>printOn: >> Or is there a large benefit to avoiding the #printStringLimitedTo: >> machinery ? >> >> cheers -ben >> > > It should be printOn: generally, except that printOn: requires creating a > Stream, and requires passing thru Stream indirection for writing into the > target String. > > As the comment tells, the sole reason for this implementation is "Highly > optimized version". So consider that it's an exception to the rules. Maybe > this kind of decision could be periodically re-examined in the light of > performance measurements (I'm thinking of Sista)? >

April 8, 2017

Re: [Pharo-dev] SmallInteger #printString versus #printOn:

by Nicolas Cellier

2017-04-08 6:38 GMT+02:00 Ben Coman <btc(a)openinworld.com>: > I was going to slip this example into the welcome help, > > 1 printString [debugIt] > > and I'm confronted by SmallInteger>>printString > > printString > "Highly optimized version for base 10 > and that we know it is a SmallInteger." > | integer next result len | > self = 0 ifTrue: [^'0']. > self < 0 ifTrue: [^'-', self negated printString]. > len := self decimalDigitLength. > result := String new: len. > integer := self. > len to: 1 by: -1 do: [:i | > next := integer // 10. > result byteAt: i put: 48 + (integer - (next * 10)). > integer := next]. > ^result > > > and I gather from other discussions this should(?) belong in > SmallInteger>>printOn: > Or is there a large benefit to avoiding the #printStringLimitedTo: > machinery ? > > cheers -ben > It should be printOn: generally, except that printOn: requires creating a Stream, and requires passing thru Stream indirection for writing into the target String. As the comment tells, the sole reason for this implementation is "Highly optimized version". So consider that it's an exception to the rules. Maybe this kind of decision could be periodically re-examined in the light of performance measurements (I'm thinking of Sista)?

April 8, 2017

Re: [Pharo-dev] stepping off the end of the debug stack

by Ben Coman

On Sat, Apr 8, 2017 at 12:32 PM, Ben Coman <btc(a)openinworld.com> wrote: > > I imagine a newcomer exploring the system could easily step off the end of > the debug stack like this... > > (1 printString) "debugIt" > then StepOver six times where the image hangs at "Processor > terminateActive" > > This could have a strong impact against first impressions. > What would it take to guard against this? > > cheers -ben > As an immediate stop-gap I'm adding the following comment to BlockClosure>>newProcess "IMPORTANT! Debug stepping this deep infrastructure may lock your Image" "If you are not sure what you are doing, close the debugger now." cheers -ben

April 8, 2017

SmallInteger #printString versus #printOn:

by Ben Coman

I was going to slip this example into the welcome help, 1 printString [debugIt] and I'm confronted by SmallInteger>>printString printString "Highly optimized version for base 10 and that we know it is a SmallInteger." | integer next result len | self = 0 ifTrue: [^'0']. self < 0 ifTrue: [^'-', self negated printString]. len := self decimalDigitLength. result := String new: len. integer := self. len to: 1 by: -1 do: [:i | next := integer // 10. result byteAt: i put: 48 + (integer - (next * 10)). integer := next]. ^result and I gather from other discussions this should(?) belong in SmallInteger>>printOn: Or is there a large benefit to avoiding the #printStringLimitedTo: machinery ? cheers -ben

April 8, 2017

stepping off the end of the debug stack

by Ben Coman

I imagine a newcomer exploring the system could easily step off the end of the debug stack like this... (1 printString) "debugIt" then StepOver six times where the image hangs at "Processor terminateActive" This could have a strong impact against first impressions. What would it take to guard against this? cheers -ben

April 8, 2017

books site description

by Ben Coman

Just a passing thought, at http://files.pharo.org/books/ it might be nice for the top descriptive text to indicate a reading order. Something like... Newcomers should start with Pharo By Example, then proceed to: - Spec UI Framework for desktop applications, - Enterprise Pharo for web development, - Agile Visualization for data processing. Deep Into Pharo and Numerical Methods are broadly applicable to all three domains. cheers -ben

April 8, 2017

Help currency review

by Ben Coman

At this late stage of Pharo 6 development, it might be good to do a pass at... World > Help > Help Browser content Epicea >Browsers - mentions two separate browser windows GT Debugger >> Actions - TODO GT Playground >> Overview - http://gt.moosetechnology.org redirects to http://gtoolkit.org/ and same for several GT entries Welcome > Getting Help says... - The "Pharo Users" mailing list: http://lists.pharo.org/mailman/listinfo/pharo-users_lists.pharo.org - The "Pharo Slack Team": http://slack4pharo.trentosur.com/ - The "Pharo IRC Channel": irc.freenode.net, #pharo channel - http://pharo.org/community So we should change from Slack to Discord. I can take care of this one but a couple questions... 1. I propose a Discord #Help channel to record here in the help browser. 2. I bumped into an interesting tip https://www.reddit.com/r/discordapp/comments/3yu1bh/server_vs_channel_invit… Thus I propose we have a #Welcome channel providing info like what the bot has been providing. Later it could be good to have a bot just in that channel, but not off someone's home server :) I would expect that channel to not need anyone lurking in it. Newcomers will jump immediately from there #Help or #General 3. How active is the IRC channel? Should we remove it from the help browser? The mention at http://pharo.org/community could be sufficient. 4. Should we backport this help info update to Pharo 5? cheers -ben

April 8, 2017

Enhanced GitHub code review

by Ben Coman

I bumped into this student dissertation. I only skimmed it quickly since its not a current interest of mine but I thought it might align with some members research interests. http://repository.tudelft.nl/islandora/object/uuid%3A52517b50-5da1-419a-b91… cheers -ben

April 8, 2017