December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Fuel materialization on spur

by Robert Withers

Please consider Avro. robert On 12/16/2015 08:58 AM, H. Hirzel wrote: > If you want to move data to Java then you probably go for JSON or a > particuar XML format. > > On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >> No, it is a Smalltalk format. >> >> On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >>> >>>> It probably should be noted here as well that >>>> >>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>>> >>> Just to know does java implementation exists? >>> -- . .. .. ^,^ best, robert

Dec. 16, 2015

Re: [Pharo-dev] [Vm-dev] Re: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support))

by Eliot Miranda

Hi Hannes, > On Dec 16, 2015, at 2:22 AM, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: > > >> On 12/16/15, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> Hi Todd, >> >>> On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: >>> >>> Hi Eliot, >>> >>> On Dec 15, 2015, at 13:46, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>> Just so you know, I will dig my heels in as deeply as I am able to >>> prevent >>> the use of C++ libraries in the VM. It destroys the simulator, which is >>> the most important thing we have for VM development productivity. As far >>> as I'm concerned any use of external libraries to implement core >>> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >>> are >>> built upon. >>> >>> >>> OK, I defer to you because you certainly know more about the VM internals >>> and what does and doesn't work well than anyone else. >>> >>> So I guess I would like to know your recommendation for 1) how best to >>> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >>> get to worry about endian). >> >> Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it >> should start as an 8-bit-per-Unicode-character string. Attempts to store >> Character values that won't fit cause the String to become a String whose >> element size is large enough to accommodate the character. > > This is the case: see tests here http://wiki.squeak.org/squeak/6316 > > In Spur, >> become: is cheap so this growth pays only for the reallocation and copying >> of the at a, not for an expensive heap scan necessary to do the become:. > > Could you elaborate on this please? Well, there are two presentations online, one at ESUG 2014, and one at Splash 2015, a paper at ISMM 2015, and a blog post if you want a full account, but... Spur supports lazy become via forwarders. When an object is becommed, it is turned into a forwarder to the object it becomes, and when a pair of objects are becommed each is copied and the two originals turned into forwarders to the opposite copy. Immediately after the forwarding the receiver in each stack frame in the stack zone is scanned to follow forwarders so that no read barrier is needed when accessing inst vars. Forwarders have a unique class index so any message send to a forwarder will fail. The check for sends to forwarders needs to be done only just before a full lookup. A forwarder looks different to other objects (they have a unique format field in their object header) so any primitive that encounters a forwarder in its operands will fail (since primitives validate their arguments). So in primitive failure the VM checks for forwarders amongst the operands and if any are found, they are followed and the primitive is retried. There are read barriers in some operations, hence I call the scheme a partial read barrier. The main cost is the stack zone scan which must be performed for pointer objects (since only these can have inst vars). This takes very little time on current machinery. Becoming bit objects like strings is faster because no scan is needed. >>> Bearing in mind that both representations are variable length and so >>> while >>> accessing the n'th byte/word is O(1), accessing the n'th character is >>> necessarily O(n) unless you know you have no surrogates in your string. >> >> Right, so UTF-8 and UTF-16 are not convenient representations and to be >> provided only for interchange. > > +1 > > Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. > There are converters for UTF-8 and UTF-16. > >>> Also...since NSString has been mentioned...it is worth noting that >>> NSString is built atop CFString (source code here: >>> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >>> does a fair job of optimizing memory by using bytes where it can and >>> shorts >>> where it cannot. It is also worth noting that characterAt: actually does >>> the wrong thing, since it assumes characters are no bigger than FFFF >>> rather >>> than 10FFFF. >> >> Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one >> has become: it is very easy to manage. Now with Spur not only do we have >> become:, we have a fairly fast become:. >> >> Does this make sense? >> >> >>> Also...I'll just toss in this very nice article on unicode and how >>> NSString deals with it. >>> https://www.objc.io/issues/9-strings/unicode/ >>> >>> -Todd Blanchard >> >> _,,,^..^,,,_ >> best, Eliot

Dec. 16, 2015

Error when using Git

by Alexandre Bergel

Hi! I get the following error: Any idea whatâs wrong? Cheers, Alexandre â _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 16, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50505 Home: https://github.com/pharo-project/pharo-core

Dec. 16, 2015

[pharo-project/pharo-core] 48fb68: 50505

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 48fb68df9024ddf7132306e92cf692f1ec9727d1 https://github.com/pharo-project/pharo-core/commit/48fb68df9024ddf7132306e9… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-16 (Wed, 16 Dec 2015) Changed paths: M Kernel-Tests.package/BehaviorTest.class/instance/tests/testCompactAndUncompactClass.st M Kernel-Tests.package/BehaviorTest.class/instance/tests/testUncompactClassNotAllowed.st M Kernel-Tests.package/SizeInMemoryTest.class/instance/tests/testSizeInMemoryCompactClasses.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50504.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50505.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50504.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50505.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A Traits.package/TBehavior.class/instance/testing/isEphemeronClass.st A Traits.package/TBehavior.class/instance/testing/isImmediateClass.st Log Message: ----------- 50505 17254 Object>>#clone method is not categorised https://pharo.fogbugz.com/f/cases/17254 http://files.pharo.org/image/50/50505.zip

Dec. 16, 2015

Re: [Pharo-dev] Do we have the new primitive?? [WAS] Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

On Tue, Feb 28, 2012 at 7:52 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Mon, Feb 27, 2012 at 3:16 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Sat, 25 Feb 2012, Mariano Martinez Peck wrote: >> >> >>> All I can say is that I am impressed by the numbers it is >>> really much >>> faster. >>> I still don't understand why I send this email with a subject >>> say >>> IdentitySet because what I really need is a fast/large >>> IdentityDictionary >>> :( Anyway, there's a place where we can use this >>> LargeIdentitySet in Fuel >>> I think). >>> >>> So Levente, you say this is not possible to adapt this for >>> dictionary? can >>> we contact Eliot to provide such a primitive? >>> >>> >>> As promised, I uploaded my LargeIdentityDictionary implementation to >>> http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st . >>> The numbers will be a bit worse compared to LargeIdentitySet, because of >>> the lack of the primitive, but it's still 2-3x faster than other solutions >>> (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm >>> about to propose this primitive with other improvements on the vm-dev >>> list. >>> >> >> My proposals are still on the way. :) >> >> >> >>> >>> Hi Eliot/Levente. What is the status of this? Do we have already the new >>> primitive? If true, how can we adapt LargeIdentitySet to use such new >>> primitive? >>> >> >> AFAIK the new primitive is not implemented yet. Adding the primitive to >> the interpreter VM is very easy, but it seems to be a lot more complicated >> (to me) to add it to Cog, because the receiver can be a MethodContext which >> needs special handling. >> I'll rewrite both LargeIdentitySet and LargeIdentityDictionary when the >> primitive is ready. >> > > Thanks Levente. So we should wait Eliot. > I will ping again in a couple of weeks/months ;) > > I am reviving a very old thread. Levente, as you know, we are using a variation of your LargeIdentitySet and LargeIdentityDictionary for Fuel. Considering Spur's new #identityHash, had you have the chance to try/think if these large collection classes is still worth? What it seems clear is that with Spur we could simply replace: ProtoObject >> largeIdentityHash <primitive: 75> With: ProtoObject >> largeIdentityHash ^ self basicIdentityHash Right? But I still wonder about the collection classes itself. Thoughts? > > >> >> >> Levente >> >> >> >>> Thanks! >>> >>> >>> >>> >>> >>> >>> >>> >>> Levente >>> >>> >>> thanks >>> >>> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi < >>> leves(a)elte.hu> wrote: >>> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>> >>> How about my numbers? :) >>> >>> "Preallocate objects, so we won't count gc time." >>> n := 1000000. >>> objects := Array new: n streamContents: [ :stream | >>> n timesRepeat: [ stream nextPut: Object new ] ]. >>> >>> set := IdentitySet new: n. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "4949" >>> >>> set := LargeIdentitySet new. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "331" >>> >>> set := (PluggableSet new: n) >>> hashBlock: [ :object | object identityHash * 4096 + object >>> class >>> identityHash * 64 ]; "Change this to #basicIdentityHash in >>> Pharo" >>> equalBlock: [ :a :b | a == b ]; >>> yourself. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "5511" >>> >>> >>> I also have a LargeIdentityDictionary, which is relatively >>> fast, but not >>> as fast as LargeIdentitySet, because (for some unknown >>> reason) we don't >>> have a primitive that could support it. If we had a primitive >>> like >>> primitive 132 which would return the index of the element if >>> found or 0 if >>> not, then we could have a really fast LargeIdentityDictionary. >>> >>> >>> Levente >>> >>> Hehe yes, if writing a version fully exploiting the limited range, >>> that's >>> probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/ >>> LargeIdentitySet.st> >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls >>> to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as >>> I >>> used above in 871 ms. (181 with preallocation). >>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>> Henry >>> >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> >>> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Denis Kudriashov

2015-12-16 14:58 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: > If you want to move data to Java then you probably go for JSON or a > particuar XML format. > Of course. Or any other format you can find :) SIXX is XML. And maybe somebody implement it in different languages. I just interesting about it.

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by H. Hirzel

If you want to move data to Java then you probably go for JSON or a particuar XML format. On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: > No, it is a Smalltalk format. > > On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >> >>> It probably should be noted here as well that >>> >>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>> >> >> Just to know does java implementation exists? >> >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by H. Hirzel

No, it is a Smalltalk format. On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: > >> It probably should be noted here as well that >> >> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >> > > Just to know does java implementation exists? >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Denis Kudriashov

2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: > It probably should be noted here as well that > > http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ > Just to know does java implementation exists?

Dec. 16, 2015