November 2014 - Pharo-dev - lists.pharo.org

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/40352 Home: https://github.com/pharo-project/pharo-core

Nov. 4, 2014

[pharo-project/pharo-core] 641774: 40352

by GitHub

Branch: refs/heads/4.0 Home: https://github.com/pharo-project/pharo-core Commit: 641774a4f95a77aa2eaf780895e12781d6021491 https://github.com/pharo-project/pharo-core/commit/641774a4f95a77aa2eaf7808… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2014-11-04 (Tue, 04 Nov 2014) Changed paths: M HelpSystem-Core.package/ClassAPIHelpBuilder.class/instance/building/build.st M HelpSystem-Core.package/ClassAPIHelpBuilder.class/instance/private building/buildMethodTopicsOn_for_.st M HelpSystem-Core.package/HelpBrowser.class/instance/ui/initWindow.st M HelpSystem-Core.package/WikiStyleHelpBuilder.class/instance/building/build.st M HelpSystem-Core.package/WikiStyleHelpBuilder.class/instance/private accessing/helpContentFromString_.st M HelpSystem-Core.package/WikiStyleHelpBuilder.class/instance/private extraction/extractHelpContentFromString_.st R Polymorph-Widgets.package/DropListItem.class/README.md R Polymorph-Widgets.package/DropListItem.class/class/instance creation/named_do_.st R Polymorph-Widgets.package/DropListItem.class/class/instance creation/on_do_.st R Polymorph-Widgets.package/DropListItem.class/definition.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/=.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/actionHolder.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/icon.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/icon_.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/label.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/label_.st R Polymorph-Widgets.package/DropListItem.class/instance/accessing/model.st R Polymorph-Widgets.package/DropListItem.class/instance/execution/value.st R Polymorph-Widgets.package/DropListItem.class/instance/initialization/initialize.st R Polymorph-Widgets.package/DropListItem.class/instance/private/model_.st R Polymorph-Widgets.package/DropListItem.class/instance/protocol/action_.st R Polymorph-Widgets.package/DropListItem.class/instance/protocol/display_.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - scripts/script352.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - updates/update40352.st M ScriptLoader40.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A Spec-Core.package/DropListItem.class/README.md A Spec-Core.package/DropListItem.class/class/instance creation/named_do_.st A Spec-Core.package/DropListItem.class/class/instance creation/on_do_.st A Spec-Core.package/DropListItem.class/definition.st A Spec-Core.package/DropListItem.class/instance/accessing/=.st A Spec-Core.package/DropListItem.class/instance/accessing/actionHolder.st A Spec-Core.package/DropListItem.class/instance/accessing/icon.st A Spec-Core.package/DropListItem.class/instance/accessing/icon_.st A Spec-Core.package/DropListItem.class/instance/accessing/label.st A Spec-Core.package/DropListItem.class/instance/accessing/label_.st A Spec-Core.package/DropListItem.class/instance/accessing/model.st A Spec-Core.package/DropListItem.class/instance/execution/value.st A Spec-Core.package/DropListItem.class/instance/initialization/initialize.st A Spec-Core.package/DropListItem.class/instance/private/model_.st A Spec-Core.package/DropListItem.class/instance/protocol/action_.st A Spec-Core.package/DropListItem.class/instance/protocol/display_.st M Versionner-Core-Model.package/MBConfigurationRoot.class/instance/updating/register.st Log Message: ----------- 40352 14384 Move DropListItem to Spec-Core-Widgets-Support https://pharo.fogbugz.com/f/cases/14384 14381 Replace Announcer>>#on:send:to:s senders in Versionner-Core-Model https://pharo.fogbugz.com/f/cases/14381 14386 Small critic pass over HelpSystem-Core https://pharo.fogbugz.com/f/cases/14386 http://files.pharo.org/image/40/40352.zip

Nov. 4, 2014

Re: [Pharo-dev] Dealing with very large files

by Alexandre Bergel

Ok! Thanks Nicolai, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > On Nov 4, 2014, at 10:19 AM, Nicolai Hess <nicolaihess(a)web.de> wrote: > > 2014-11-04 13:59 GMT+01:00 Alexandre Bergel <alexandre.bergel(a)me.com <mailto:alexandre.bergel@me.com>>: > Hi! > > A question just popped up: Can Pharo handle very large files? I mean, can I open and navigate within a 1 Gb file? > Someone has experience to share? > > Cheers, > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu <http://www.bergel.eu/> > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > 1GB works larger than 2GB does not work on linux yet: > 14250 <https://pharo.fogbugz.com/default.asp?14250> > Enable large file support on linux > > >

Nov. 4, 2014

Re: [Pharo-dev] Dealing with very large files

by Alexandre Bergel

> Is it enough to open a stream and traverse the stream? Yes. Pharo works well in that setting? Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > On Nov 4, 2014, at 10:24 AM, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Hi Alex, > > What is the scenario exacty? What do you mean by navigate? Is it enough to open a stream and traverse the stream? > > Doru > > On Tue, Nov 4, 2014 at 2:19 PM, Nicolai Hess <nicolaihess(a)web.de <mailto:nicolaihess@web.de>> wrote: > 2014-11-04 13:59 GMT+01:00 Alexandre Bergel <alexandre.bergel(a)me.com <mailto:alexandre.bergel@me.com>>: > Hi! > > A question just popped up: Can Pharo handle very large files? I mean, can I open and navigate within a 1 Gb file? > Someone has experience to share? > > Cheers, > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu <http://www.bergel.eu/> > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > 1GB works larger than 2GB does not work on linux yet: > 14250 <https://pharo.fogbugz.com/default.asp?14250> > Enable large file support on linux > > > > > > > -- > www.tudorgirba.com <http://www.tudorgirba.com/> > > "Every thing has its own flow"

Nov. 4, 2014

Re: [Pharo-dev] LRUCache for high latency stuff

by Sven Van Caekenberghe

Hi Thierry, I made a prototype implementation, would this be helpful ? LRUCache>>#at: key put: value "Populate the receiver by putting value under key. This operation is neither considered a hit or a miss. If key is already present, replace it. Return value" ^ self critical: [ (keyIndex associationAt: key ifAbsent: [ nil ]) ifNil: [ | newAssociation link | newAssociation := self newAssociationKey: key value: value. link := lruList addLast: newAssociation. keyIndex at: key put: link ] ifNotNil: [ :existingAssociation | | link | link := existingAssociation value. weight remove: link value value. link value value: value. self promote: link ]. self addWeight: value. value ] (#newAssociationKey:value: is new, you can replace it with key->value) Then you can do: testPopulate | cache | cache := self newCache. cache factory: [ self fail ]. cache at: #foo put: -1. cache at: #bar put: 200. cache at: #foo put: 100. self assert: cache size equals: 2. self assert: cache totalWeight equals: 2. self assert: cache hits isZero. self assert: cache misses isZero. self assert: (cache at: #foo) equals: 100. self assert: (cache at: #bar) equals: 200. cache validateInvariantWith: self For Norbert I did a little change that allows you to subclass either LRUCache or TTLCache and customise the Associations being used as well as the stale testing. He wanted to have entry specific TTLs, not just one value for the whole cache. Maybe that could also help you. The other question about HTTP client behaviour: I don't know how many requests you could run parallel, 10s maybe 100s. I have it somewhere using a custom limited resource pool, but when the pool is empty (all workers active), clients wait. In my use case, it was all synchronous. If you like the #at:put: I could move it into the image. Sven > On 03 Nov 2014, at 23:14, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Hi Sven, > > 2014-11-03 22:52 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Hi Thierry, > > > On 31 Oct 2014, at 14:10, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > > > Hi all, > > > > I tried to use LRUCache for some application of mine with a two stages, over-the-internet data requests, and I couldn't find a way of using it. > > > > My use case was: > > - retrieve some data with high latency (over 10 seconds) > > - in a interactive application, where I need to be able to continue while the data is being loaded. > > > > So I designed it in two stages: > > - Have a cache > > - if the data isn't in the cache > > -- trigger a load with a fork > > -- put some temporary data in the cache (something which says loading) > > -- continue > > - When the data load is terminated > > -- replace the cached temporary data with the final version. > > > > The thing is that I couldn't do the last one with the LRUCache (there is not at:put:, only a at:ifAbsentPut: []). Did I miss something? > > > > Thierry > > Relevant question and a rather special use case I would say. > > The API of AbstractCache and its subclasses is minimal by design. It certainly does not contain everything that Dictionary has. > > A cache delivers values based on keys. The ifAbsent: block is similar to the factory: block, it implements getting the value. Either you hit the cache or you miss it and it gets loaded. Entries drop off either because there are too many in some metric (LRUCache) or because they expire (TTLCache). > > Now, #at:put: is like a replace, is it a hit or a miss ? Or neither ? > > It would be a replace, and neither a hit nor a miss. > > > You could implement it with a #removeKey: and and #at:ifAbsent: - modulo some concurrent access problems. > > I do those operations under a mutex, otherwise the system becomes incoherent :) > > > Maybe we should add it (I guess it could be useful to fill a cold cache as well) - but I am not totally convinced. > > Now, your use case is a bit odd and dangerous. You spawn a thread/process for each miss, there could be a large number of them - do you want that ? Will the value always/still be needed when it arrives ? > > A non needed value (to late value) will be discarded almost immediately, so this is not an issue. The number of threads may be high, yes, but it stays apparently within reasonable limits (how many Zn requests one can do simultaneously from Pharo? Is there any throttling?). > > > One way to implement your use case would be to encapsulate your special behaviour (the delayed loading) in the value itself (since you have the two cases/states exposed), no ? > > Yes. This is more or less what I implemented, but without merging them in a single object. > > This makes using the LRUCache as is, but is no better than the previous approach about the number of threads in flight (those could be solved by a sharedQueue and a few loading processes feeding themselves out of the shared queue). > > Now I know how to use the LRUCache, but I think my use case is in a way too simple for its features (no real TTL, LRU only works if the cache size is over a certain threshold and that threshold isn't static). > > Thanks, > > Thierry > > Sven

Nov. 4, 2014

Re: [Pharo-dev] Dealing with very large files

by Tudor Girba

Hi Alex, What is the scenario exacty? What do you mean by navigate? Is it enough to open a stream and traverse the stream? Doru On Tue, Nov 4, 2014 at 2:19 PM, Nicolai Hess <nicolaihess(a)web.de> wrote: > 2014-11-04 13:59 GMT+01:00 Alexandre Bergel <alexandre.bergel(a)me.com>: > >> Hi! >> >> A question just popped up: Can Pharo handle very large files? I mean, can >> I open and navigate within a 1 Gb file? >> Someone has experience to share? >> >> Cheers, >> Alexandre >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> >> > 1GB works larger than 2GB does not work on linux yet: > 14250 <https://pharo.fogbugz.com/default.asp?14250> > Enable large file support on linux > > > > -- www.tudorgirba.com "Every thing has its own flow"

Nov. 4, 2014

Re: [Pharo-dev] Dealing with very large files

by Nicolai Hess

2014-11-04 13:59 GMT+01:00 Alexandre Bergel <alexandre.bergel(a)me.com>: > Hi! > > A question just popped up: Can Pharo handle very large files? I mean, can > I open and navigate within a 1 Gb file? > Someone has experience to share? > > Cheers, > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > 1GB works larger than 2GB does not work on linux yet: 14250 <https://pharo.fogbugz.com/default.asp?14250> Enable large file support on linux

Nov. 4, 2014

Dealing with very large files

by Alexandre Bergel

Hi! A question just popped up: Can Pharo handle very large files? I mean, can I open and navigate within a 1 Gb file? Someone has experience to share? Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Nov. 4, 2014

Re: [Pharo-dev] encodings and regex clash?

by stepharo

Thanks sven I will check on my file. Because may be my html files are funky. On 2/11/14 21:26, Sven Van Caekenberghe wrote: > StÃ©ph, > > For me all this works and gives the same result: > > '<head> > -BASE- > <title>Dr. Stéphane Ducasse</title> > </head>' copyWithRegex: '-BASE-' matchesReplacedWith: '<p>Foo</p>'. > > 'foo.html' asFileReference writeStreamDo: [ :out | out << '<head> > -BASE- > <title>Dr. Stéphane Ducasse</title> > </head>' ]. > > 'foo.html' asFileReference contents copyWithRegex: '-BASE-' matchesReplacedWith: '<p>Foo</p>'. > > Sven > >> On 02 Nov 2014, at 16:15, Max Leske <maxleske(a)gmail.com> wrote: >> >> >>> On 02.11.2014, at 14:59, stepharo <stepharo(a)free.fr> wrote: >>> >>> >>> On 2/11/14 12:37, Max Leske wrote: >>>>> On 01.11.2014, at 22:07, stepharo <stepharo(a)free.fr> wrote: >>>>> >>>>> Hi guys >>>>> >>>>> I'm doing a small variable expansion to relocate my webpage. I want to expand -base- into <base>http://sdmeta.gforge.inria.fr</base> >>>>> Now it works when I test with some strings inside the image >>>>> >>>>> >>>>> trying >>>>> "self new trying" >>>>> >>>>> ^ self sample >>>>> copyWithRegex: '-BASE-' >>>>> matchesReplacedWith: self baseString. >>>>> >>>>> sample >>>>> >>>>> >>>>> ^ '<?xml version="1.0" encoding="ISO-8859-1"?> >>>>> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> >>>>> <link rel="stylesheet" href="stef.css"/> >>>>> <head> >>>>> -BASE- >>>>> <title>Dr. Stéphane Ducasse</title> >>>>> </head>' >>>>> >>>>> baseString >>>>> ^ '<base>http://sdmeta.gforge.inria.fr </base>' >>>>> >>>>> self new trying results >>>>> >>>>> '<?xml version="1.0" encoding="ISO-8859-1"?> >>>>> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> >>>>> <link rel="stylesheet" href="stef.css"/> >>>>> <head> >>>>> <base>http://sdmeta.gforge.inria.fr </base> >>>>> <title>Dr. Stéphane Ducasse</title> >>>>> </head>' >>>>> >>>>> and this is good. >>>>> >>>>> Now when I work with a stream it does. >>>>> >>>>> >>>>> >>>>> contentsOf: aFileSystemFile >>>>> "self new contentsOf: (FileSystem workingDirectory / 'Books.html') " >>>>> contents := aFileSystemFile readStream contents >>>>> >>>>> tryingContents >>>>> "self new tryings" >>>>> >>>>> ^ self sample >>>>> copyWithRegex: '-BASE-' >>>>> matchesReplacedWith: contents. >>>> Whatâs the âwrongâ output you get here? >>> In fact I get the stream contents but the regex does not match. >>> I suspect a problem with the encodings of the file. Do you have an idea? Stef >> Noâ¦ I donât see how encoding could be a problem. Can you create a minimal working (failing) test file and post it? >> > > >

Nov. 4, 2014

Re: [Pharo-dev] encodings and regex clash?

by stepharo

Thanks max I will do it. >> In fact I get the stream contents but the regex does not match. >> I suspect a problem with the encodings of the file. Do you have an >> idea? Stef > > Noâ¦ I donât see how encoding could be a problem. Can you create a > minimal working (failing) test file and post it? >

Nov. 4, 2014