December 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [ANN] Neo-Caching

by Jan Vrany

Hi, > By default, no concurrent access protection takes place, but optionally a semaphore for mutual exclusion can be used. This slows down access. > > cache useSemaphore. > There was enough "awesomes" already :-) so now some critics :-) Wouldn't it be better to rename #useSemaphore to #beThreadSafe or #beSynchronized. Also I would use recursion lock (monitor, if you like) rather than plain mutex. Best, Jan

Dec. 11, 2013

Re: [Pharo-dev] [ANN] Neo-Caching

by StÃ©phane Ducasse

On Dec 11, 2013, at 12:29 PM, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > > > On Tue, Dec 10, 2013 at 11:09 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > Sven thanks a lot > > yes! really cool! > > How can we resist to push this in Pharo 30 :) > > over my dead body :) > but in Pharo 4â¦ you are saying that you will let squeak be in advance compared to Pharo :) I cannot believe that :). Stef > > > Stef > > On Dec 10, 2013, at 4:54 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > Hi, > > > > Caching is an important technique to trade time (speed of execution) for space (memory consumption) in computation. Used correctly, caching can significantly improve an applicationâs performance. > > > > Given the key/value nature of a cache, a Dictionary is often the first data structure used for implementing it. This is however not a good idea since a simple Dictionary is unbounded by definition and can/will lead to explosive cache growth. > > > > Quick and dirty hacks to turn a Dictionary into a least-recently-used (LRU) and/or time-to-live (TTL) based limited cache are often incorrect and inefficient. > > > > The LRUCache implementation currently in Pharo is too simple and not efficient, being O(n) on its main operation. > > > > Pharo deserves and needs a good LRU/TTL cache implementation that can/should be used uniformly in various places of the system and in code built on top of it. That is why I implemented a proposal. > > > > Neo-Caching is an implementation of both a good LRU cache as well as a TTL extension of it. The caches are easy to use, yet have a number of interesting features. The implementation is properly O(1) on its main operations and is 2 to 3 times faster than LRUCache. The package also contains a double linked list implementation. The code comes with a full set of unit tests. There are class and method comments and the code is easy to read. > > > > The code (which has no dependencies) can be found here > > > > http://mc.stfx.eu/Neo > > http://www.smalltalkhub.com/mc/SvenVanCaekenberghe/Neo/main > > > > There is only one package to load. The code has been written in Pharo 3.0 but it should run on older versions as well. > > > > > > Code > > > > > > Letâs create a 16K ordered collection of keys that have some repetitions in them with a reasonable distribution: > > > > data := Array streamContents: [ :out | > > 1 to: 4096 do: [ :each | > > each primeFactorsOn: out. > > out nextPut: each ] ]. > > data := data collect: [ :each | each asWords ]. > > > > By using #asWords the keys have better hash properties. Now, we put all this in a cache: > > > > cache := NeoLRUCache new. > > cache maximumWeight: 512. > > cache factory: [ :key | key ]. > > data do: [ :each | cache at: each ]. > > cache. > > > > ==> a NeoLRUCache(#512 512/512 [ 1 ] [ :key | key ] 72%) > > > > We had a 72% hit ratio, which is good. The cache is of course full, 512/512 with 512 entries (not necessarily the same thing, see further). > > > > We can now benchmark this: > > > > [ data do: [ :each | cache at: each ] ] bench. > > > > ==> '35.8 per second.â > > > > And compare it to LRUCache: > > > > cache := LRUCache size: 512 factory: [ :key | key ]. > > > > [ data do: [ :each | cache at: each ] ] bench. > > > > ==> '12.6 per second.â > > > > > > Features > > > > > > Instead of just counting the number of entries, which is the default behaviour, the concept of weight is used to determine when a cache is full. For each cached value, a block or selector is used to compute its weight. When adding a new entry causes the weight to exceed a maximum, eviction of the least recently used item(s) takes place, until the weight is again below the maximum. > > > > cache > > computeWeight: #sizeInMemory; > > maximumWeight: 16*1024. > > > > Will keep the cache below 16Kb in real memory usage. This can be very useful when caching various sized images or MC packages, for example. > > > > By default, no concurrent access protection takes place, but optionally a semaphore for mutual exclusion can be used. This slows down access. > > > > cache useSemaphore. > > > > NeoTTLCache extends NeoLRUCache by maintaining a timestamp for each cached value. Upon cache hit, there is a check to see if this timestamp is not older than the allowed time to live. If so, the value is stale and will be recomputed. > > > > (cache := NeoTTLCache new) > > timeToLive: 10 minutes; > > factory: [ :key | ZnEasy get: key ]; > > maximumWeight: 32*1024; > > computeWeight: #contentLength. > > > > cache at: âhttp://zn.stfx.eu/zn/numbers.txt'. > > > > ==> a ZnResponse(200 OK text/plain;charset=utf-8 71B) > > > > cache > > > > ==> a NeoTTLCache(#1 71/32768 #contentLength [ :key | ZnEasy get: key ] 0% 0:00:10:00) > > > > Would be a simple HTTP cache, keeping resolved URLs in memory for 10 minutes maximum before refreshing them. It uses #contentLength on ZnResponse to compute the weight. You can see from the print string that there is now 1 entry, while the weight is 17 bytes out of 32768. > > > > > > These are the main points, please have a look at the code. Feedback is welcome. > > > > Regards, > > > > Sven > > > > > > >

Dec. 11, 2013

Re: [Pharo-dev] self item returns a string

by StÃ©phane Ducasse

Thanks a lot. I will check and resolve the bug. I wanted to continue working on unloading code so this is why we time bombed our investigation. Stef > > printing strings with quotes and doubling this quotes is done in > Finder>>computeWithMethodFinder: > > node formattedCode printOn: strm. > > replace it with > strm << node formattedCode. > > An awful big method btw. > > > nicolai >

Dec. 11, 2013

Re: [Pharo-dev] self item returns a string

by Nicolai Hess

2013/12/11 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > On Dec 11, 2013, at 10:31 AM, Benjamin < > Benjamin.VanRyseghem.Pharo(a)gmail.com> wrote: > > Sorry but I wrote part of this code 3 years ago, so I really do not > remember :( > > > No problem. > We tried during 30 min with pavel but failed. We should have a look again. > > Stef > > > Ben > > On 11 Dec 2013, at 09:51, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > wrote: > > Hi ben > > Im trying to fix the problem of the finder putting too many string around > the arguments. > if you type 'a' . 'b' . 'ab' and example > you will get '''a''' . '''b''' > > so I found that > > displayString > > self halt. > ^ self item asString > > self item returns a byteString and item is not redefined in FinderNode > > I tried to change it to > > displayString > > > ^ self item > > but self item is already '''a '''' > > the item on the node is already a '''a''''â¦. > > So I tried to check how the node were populated but I could not get > childrenItems to work. > > > > > printing strings with quotes and doubling this quotes is done in Finder>>computeWithMethodFinder: node formattedCode printOn: strm. replace it with strm << node formattedCode. An awful big method btw. nicolai

Dec. 11, 2013

Re: [Pharo-dev] [ANN] Neo-Caching

by Esteban Lorenzano

On Tue, Dec 10, 2013 at 11:09 PM, StÃ©phane Ducasse < stephane.ducasse(a)inria.fr> wrote: > Sven thanks a lot > yes! really cool! > How can we resist to push this in Pharo 30 :) > over my dead body :) but in Pharo 4... > > Stef > > On Dec 10, 2013, at 4:54 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > Hi, > > > > Caching is an important technique to trade time (speed of execution) for > space (memory consumption) in computation. Used correctly, caching can > significantly improve an applicationâs performance. > > > > Given the key/value nature of a cache, a Dictionary is often the first > data structure used for implementing it. This is however not a good idea > since a simple Dictionary is unbounded by definition and can/will lead to > explosive cache growth. > > > > Quick and dirty hacks to turn a Dictionary into a least-recently-used > (LRU) and/or time-to-live (TTL) based limited cache are often incorrect and > inefficient. > > > > The LRUCache implementation currently in Pharo is too simple and not > efficient, being O(n) on its main operation. > > > > Pharo deserves and needs a good LRU/TTL cache implementation that > can/should be used uniformly in various places of the system and in code > built on top of it. That is why I implemented a proposal. > > > > Neo-Caching is an implementation of both a good LRU cache as well as a > TTL extension of it. The caches are easy to use, yet have a number of > interesting features. The implementation is properly O(1) on its main > operations and is 2 to 3 times faster than LRUCache. The package also > contains a double linked list implementation. The code comes with a full > set of unit tests. There are class and method comments and the code is easy > to read. > > > > The code (which has no dependencies) can be found here > > > > http://mc.stfx.eu/Neo > > http://www.smalltalkhub.com/mc/SvenVanCaekenberghe/Neo/main > > > > There is only one package to load. The code has been written in Pharo > 3.0 but it should run on older versions as well. > > > > > > Code > > > > > > Letâs create a 16K ordered collection of keys that have some repetitions > in them with a reasonable distribution: > > > > data := Array streamContents: [ :out | > > 1 to: 4096 do: [ :each | > > each primeFactorsOn: out. > > out nextPut: each ] ]. > > data := data collect: [ :each | each asWords ]. > > > > By using #asWords the keys have better hash properties. Now, we put all > this in a cache: > > > > cache := NeoLRUCache new. > > cache maximumWeight: 512. > > cache factory: [ :key | key ]. > > data do: [ :each | cache at: each ]. > > cache. > > > > ==> a NeoLRUCache(#512 512/512 [ 1 ] [ :key | key ] 72%) > > > > We had a 72% hit ratio, which is good. The cache is of course full, > 512/512 with 512 entries (not necessarily the same thing, see further). > > > > We can now benchmark this: > > > > [ data do: [ :each | cache at: each ] ] bench. > > > > ==> '35.8 per second.â > > > > And compare it to LRUCache: > > > > cache := LRUCache size: 512 factory: [ :key | key ]. > > > > [ data do: [ :each | cache at: each ] ] bench. > > > > ==> '12.6 per second.â > > > > > > Features > > > > > > Instead of just counting the number of entries, which is the default > behaviour, the concept of weight is used to determine when a cache is full. > For each cached value, a block or selector is used to compute its weight. > When adding a new entry causes the weight to exceed a maximum, eviction of > the least recently used item(s) takes place, until the weight is again > below the maximum. > > > > cache > > computeWeight: #sizeInMemory; > > maximumWeight: 16*1024. > > > > Will keep the cache below 16Kb in real memory usage. This can be very > useful when caching various sized images or MC packages, for example. > > > > By default, no concurrent access protection takes place, but optionally > a semaphore for mutual exclusion can be used. This slows down access. > > > > cache useSemaphore. > > > > NeoTTLCache extends NeoLRUCache by maintaining a timestamp for each > cached value. Upon cache hit, there is a check to see if this timestamp is > not older than the allowed time to live. If so, the value is stale and will > be recomputed. > > > > (cache := NeoTTLCache new) > > timeToLive: 10 minutes; > > factory: [ :key | ZnEasy get: key ]; > > maximumWeight: 32*1024; > > computeWeight: #contentLength. > > > > cache at: âhttp://zn.stfx.eu/zn/numbers.txt'. > > > > ==> a ZnResponse(200 OK text/plain;charset=utf-8 71B) > > > > cache > > > > ==> a NeoTTLCache(#1 71/32768 #contentLength [ :key | ZnEasy get: key ] > 0% 0:00:10:00) > > > > Would be a simple HTTP cache, keeping resolved URLs in memory for 10 > minutes maximum before refreshing them. It uses #contentLength on > ZnResponse to compute the weight. You can see from the print string that > there is now 1 entry, while the weight is 17 bytes out of 32768. > > > > > > These are the main points, please have a look at the code. Feedback is > welcome. > > > > Regards, > > > > Sven > > > > > > >

Dec. 11, 2013

Re: [Pharo-dev] Nautilus extensions questions

by Camille Teruel

On 11 dÃ©c. 2013, at 11:31, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > Ben > > with pavel we are looking at Nautilus extensions because some of them should be moved to the extended packages. > Now I was wondering why > TClassDescription is extended with menu see below. > I imagine that this is because you do not have a classWrapper for the UI. Is it correct? > > Then I wonder if there is not a pattern where we should wrap elements and attach to the wrapper such behavior. I think there is a pattern too. Each time instances of a class are used by many parties, each party will potentially patch some custom information and behavior in the class. So in the end you have a big class with tons of responsibility (for example Class ClassDescription and Behavior) where you could have a small class with many views used by parties. With a proxy that has it's own behavior and state you can represent point of views on an object. You can for example have a UI view on a class. > codeRewritingClass: aBuilder > | target | > target := aBuilder model. > > (aBuilder item: #'Rewrite Code') > action: [ target refactor rewriteCode ]; > parent: #'Code Rewriting'; > order: 0. > > (aBuilder item: #'Search Code') > action: [ target refactor searchCode ]; > parent: #'Code Rewriting'; > order: 100. > > (aBuilder item: #'Type class') > action: [ target refactor typeClass ]; > parent: #'Code Rewriting'; > order: 200; > withSeparatorAfter. > > (aBuilder item: #'Category Regex') > action: [ target refactor categoryRegex ]; > parent: #'Code Rewriting'; > order: 300. > > (aBuilder item: #'Class Regex') > action: [ target refactor classRegex ]; > parent: #'Code Rewriting'; > order: 400. > > (aBuilder item: #'Protocol Regex') > action: [ target refactor protocolRegex ]; > parent: #'Code Rewriting'; > order: 500. > > (aBuilder item: #'Source Regex') > action: [ target refactor sourceRegex ]; > parent: #'Code Rewriting'; > order: 600; > withSeparatorAfter.

Dec. 11, 2013

[Pharo Trick: #0001] - Simple TODO list

by Torsten Bergmann

Hi, the mailinglist, stackoverflow and other Pharo related medias are typically used in the "here is a question to problem" followed by an "answer from community" style. But lets also think the other way around: if we already have a solution for a problem that others may have too or may have in the future we should share the answer/solution as well. I'm sure many of you have some tricks that make developing with Pharo much easier and others can profit from these tricks if we share them here on the list. Be it - a simple and useful script - a simple key combination other may not yet know of - a package one can use or how you use it - a link to a nice description on the web related on how to use Pharo - a custom tool that makes development easier - an algorithm or something that you think was solved very elegant - a way you solved a common problem easily in your app - ... Then just describe it in a few lines and post it. Also write in which Pharo version you tried it - this makes it easier for others to see in which version the trick/solution is known to work. The idea goes like this: if you have a trick to share then send a mail with subject "Pharo Trick: #xxxx] - Short Description" to the Pharo dev AND Pharo user list (see http://lists.pharo.org) Make sure to increase the trick counter xxxx. Using this special subject one can later easily get the tricks from the mailinglist archive. Also others can refer to the trick if it solves a common problem other may ask for on the list. Lets see if we are able to reach the goals of 100, 1000 or more Pharo tricks ;) Bye T. I will start with something easy but useful: ----------------------------------------------------------------------------------- [Pharo Trick: #0001] - Simple TODO ----------------------------------------------------------------------------------- Works in: Pharo3.0 Latest update: #30635 but should work in other versions too ----------------------------------------------------------------------------------- If you develop an app with Pharo you may not yet fully implement all features. Some kind of marker for future "ToDo's" is often needed. Using a pragma on a method is the easiest solution here: foo <todo: 'Add world later'> "That method is not yet fully implemented" Transcript show: 'Hello' Using a simple expression in the workspace you can browse all the TODOs for your project afterwards in a window: SystemNavigation default browseAllSendersOf: #todo: You can even query the system for the todo descriptions if you need the full list of things to work on: (PragmaCollector filter: [:prg | prg keyword = 'todo:']) reset; collect: [:each| each arguments first ] As the pragma keyword #todo: is very common and may be used by others too it may be wise to use a custom one like "myappplicationTodo:" or similar. -----------------------------------------------------------------------------------

Dec. 11, 2013

Re: [Pharo-dev] Package unloading and extensions

by StÃ©phane Ducasse

On Dec 11, 2013, at 11:45 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > This is strange so I report it. > When we unload Nautilus package we still get the methods in ThemeIcons with category > *Nautilus-contents and antoher one *Nautilus-icons > > so we thought that unload does not unload extensions whose name is composed (*Nautilus vs. *Nautilus-icons) > so we built a case with a package and extensions (on instance and class side of different classes). > But unload was unloading them perfectly: *Box, *Box-green, *Box-foo > > the only difference is that the second experience we did we used the UI MC browser while else we used > some script ie > > (MCWorkingCopy forPackage: (MCPackage named: 'Nautilus')) unload. > > So we are puzzled. We unloaded Nautilus using the MC Browser and we got the same result the methods in ThemeIcons *Nautilus-icons and *Nautilus-contents where not unloaded. > > Stef and Pavel

Dec. 11, 2013

Package unloading and extensions

by StÃ©phane Ducasse

This is strange so I report it. When we unload Nautilus package we still get the methods in ThemeIcons with category *Nautilus-contents and antoher one *Nautilus-icons so we thought that unload does not unload extensions whose name is composed (*Nautilus vs. *Nautilus-icons) so we built a case with a package and extensions (on instance and class side of different classes). But unload was unloading them perfectly: *Box, *Box-green, *Box-foo the only difference is that the second experience we did we used the UI MC browser while else we used some script ie (MCWorkingCopy forPackage: (MCPackage named: 'Nautilus')) unload. So we are puzzled. Stef and Pavel

Dec. 11, 2013

Re: [Pharo-dev] Nautilus extensions questions

by Benjamin

Ask Sebastien since it seems he is the author. I can not help more Ben On 11 Dec 2013, at 11:31, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > Ben > > with pavel we are looking at Nautilus extensions because some of them should be moved to the extended packages. > Now I was wondering why > TClassDescription is extended with menu see below. > I imagine that this is because you do not have a classWrapper for the UI. Is it correct? > > Then I wonder if there is not a pattern where we should wrap elements and attach to the wrapper such behavior. > > > > codeRewritingClass: aBuilder > | target | > target := aBuilder model. > > (aBuilder item: #'Rewrite Code') > action: [ target refactor rewriteCode ]; > parent: #'Code Rewriting'; > order: 0. > > (aBuilder item: #'Search Code') > action: [ target refactor searchCode ]; > parent: #'Code Rewriting'; > order: 100. > > (aBuilder item: #'Type class') > action: [ target refactor typeClass ]; > parent: #'Code Rewriting'; > order: 200; > withSeparatorAfter. > > (aBuilder item: #'Category Regex') > action: [ target refactor categoryRegex ]; > parent: #'Code Rewriting'; > order: 300. > > (aBuilder item: #'Class Regex') > action: [ target refactor classRegex ]; > parent: #'Code Rewriting'; > order: 400. > > (aBuilder item: #'Protocol Regex') > action: [ target refactor protocolRegex ]; > parent: #'Code Rewriting'; > order: 500. > > (aBuilder item: #'Source Regex') > action: [ target refactor sourceRegex ]; > parent: #'Code Rewriting'; > order: 600; > withSeparatorAfter.

Dec. 11, 2013