- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Fastest utf-8 encoder contest

by Sven Van Caekenberghe

Hi Igor, It is very good that someone tries to optimize this, it is a weak point currently, performance wise. I am not a low-level guy, so I can't really help with the assembler code. But utf-8 encoding/decoding is an area that interests me. I can point to the 2 currently available utf-8 encoders/decoders that I know of, as this might be instructive for your quest (provided you did not already know of them): UTF8TextConverter ZnUTF8Encoder In Seaside, there is some optimization code in GRPharoUtf8CodecStream>>#encodeFast: that avoids encoding for runs of Latin1 characters. HTH, Sven On 13 Jun 2012, at 04:44, Igor Stasenko wrote: > Hi, hardcore hackers. > please take a look at the code and tell if it can be improved. > > The AsmJit snippet below transforms an unicode integer value > to 1..4-byte sequence of utf-8 > > then the outer piece of code (which is not yet written) will > accumulate the results of this snippet > to do a memory-aligned (4byte) writes.. > like that, if 4 unicode characters can be encoded into 4 utf-8 bytes > (which mostly the case for latin-1 char range), then there will be > 4 memory reads (to read four 32-bit unicode values) but only single > memory write (to write four 8-bit utf-8 encoded values). > > The idea is to make utf-8 encoding speed close to memory copying speed :) > > convertUnicode: asm > " > input: > - EAX 32-bit unicode value to convert > > output: > - EAX - utf-8 encoded character (in little-endian byte order) max 4 bytes > - EDX - number of encoded bytes > " > | moreThanOne moreThanTwo moreThanThree end | > > moreThanOne := asm uniqueLabelName: 'moreThanOne'. > moreThanTwo := asm uniqueLabelName: 'moreThanTwo'. > moreThanThree := asm uniqueLabelName: 'moreThanThree'. > end := asm uniqueLabelName: 'end'. > > asm > cmp: asm EAX with: 16r7F; > jg: moreThanOne; > "one byte" > mov: 1 to: asm EDX; > jmp: end; > label: moreThanOne; > cmp: asm EAX with: 16r7FF; > jg: moreThanTwo; > "two bytes 80 .. 7FF" > " AH AL " > "00000aaa aabbbbbb" > > "110aaaaa 10bbbbbb > AL AH (little endian order) > " > shr: asm EAX with: 2; > shl: asm AL with: 2; > or: asm AX with: 2r1100000010000000; > xchg: asm AL with: asm AH; > mov: 2 to: asm EDX; > jmp: end; > label: moreThanTwo; > cmp: asm EAX with: 16rFFFF; > jg: moreThanThree; > "three bytes 800 ... FFFF" > " AH AL " > "aaaabbbb bbcccccc" > " => 1110aaaa 10bbbbbb 10cccccc" > shl: asm EAX with: 4; > shr: asm AX with: 2; > shr: asm AL with: 2; > " EAX = ...aaaa xxbbbbbb xxcccccc " > or: asm EAX with: 2r111000001000000010000000; "16rE08080" > shl: asm EAX with: 8; > bswap: asm EAX; > mov: 3 to: asm EDX; > jmp: end; > "four bytes 1000 ... 10FFFF" > " AH AL " > "000aaabb bbbbcccc ccdddddd" > "=> 11110aaa 10bbbbbb 10cccccc 10dddddd" > mov: asm EAX to: asm EDX; > shl: asm EAX with: 4; > shr: asm AX with: 2; > shr: asm AL with: 2; > " EAX = 0000000a aabbbbbb xxcccccc xxdddddd " > and: asm EAX with: 16r3F3F3F; > bswap: asm EAX; > shr: asm EDX with: 18; "6*3" > or: asm DL with: 16r11110000; > mov: asm DL to: asm AL; > mov: 4 to: asm EDX; > label: end > > > -- > Best regards, > Igor Stasenko. >

June 13, 2012

Re: [Pharo-project] Problem with FileHandle >> open [WAS] Re: FileReference should throw error when deleting unexisting files?

by Mariano Martinez Peck

On Tue, Jun 12, 2012 at 11:55 PM, Camillo Bruni <camillobruni(a)gmail.com>wrote: > > > Update to the last image 20129 there are 4 failing tests and I come to > this conclusion. > > ok but these are whitebox tests which have to be adapted... (apparently my > changes got lost there) > > FileSystem-Tests-Core-CamilloBruni.9 > > in PharoInbox fixes this :) (by changing the tests) > > > thanks, they are good now. http://code.google.com/p/pharo/issues/detail?id=6059 -- Mariano http://marianopeck.wordpress.com

June 13, 2012

[Pharo-project] StandardFileStream size limit?

by GÃ¶ran Krampe

Hey all! I have just added large file (chunkwise upload and download) support to Kom/Networking/Iliad in Pharo 1.4 and while testing it I tried uploading a 2.9Gb file. I haven't looked at the exact error message, and I haven't tried writing a trivial test - but the "half written" file on disk is almost EXACTLY 2Gb (2147483647) so I am wondering - do we still have a 2Gb file size limit? This is "one click 1.4" on Ubuntu 64 with ext4. regards, GÃ¶ran

June 13, 2012

Re: [Pharo-project] DBXTalk & seaside / success in Pier one-click / my 1.4 woes follow-up

by Nick Ager

and have a look at this video: https://vimeo.com/37032840 On 13 June 2012 04:16, Yanni Chiu <yanni(a)rogers.com> wrote: > On 12/06/12 7:52 PM, Cameron Sanders wrote: > >> >> My test is the most trivial test of the Person tutorial for Magritte, >> grab my partial version from >> MCHttpRepository >> location: 'http://www.debrispublishing.**com/camshared<http://www.debrispublishing.com/camshared> >> ' >> user: '' >> password: '' >> >> and package is 'Cam'. >> > > Your code worked in my Magritte2 image. But, it looks like you built > yourself a Magritte3 image. If so, you must "port" your example from > Magritte2 to Magritte3. I'm not sure where the doc's are, but web search > turned up: http://www.slideshare.net/**nickager/magritte3<http://www.slideshare.net/nickager/magritte3> > > You need to change the #descriptionXXX stuff to use pragmas, if I remember > correctly. Again, I've not ported my code to Magritte3, but I had a brief > look earlier to understand what issues I would have. > > >

June 13, 2012

Re: [Pharo-project] DBXTalk & seaside / success in Pier one-click / my 1.4 woes follow-up

by Yanni Chiu

On 12/06/12 7:52 PM, Cameron Sanders wrote: > > My test is the most trivial test of the Person tutorial for Magritte, > grab my partial version from > MCHttpRepository > location: 'http://www.debrispublishing.com/camshared' > user: '' > password: '' > > and package is 'Cam'. Your code worked in my Magritte2 image. But, it looks like you built yourself a Magritte3 image. If so, you must "port" your example from Magritte2 to Magritte3. I'm not sure where the doc's are, but web search turned up: http://www.slideshare.net/nickager/magritte3 You need to change the #descriptionXXX stuff to use pragmas, if I remember correctly. Again, I've not ported my code to Magritte3, but I had a brief look earlier to understand what issues I would have.

June 13, 2012

[Pharo-project] Fastest utf-8 encoder contest

by Igor Stasenko

Hi, hardcore hackers. please take a look at the code and tell if it can be improved. The AsmJit snippet below transforms an unicode integer value to 1..4-byte sequence of utf-8 then the outer piece of code (which is not yet written) will accumulate the results of this snippet to do a memory-aligned (4byte) writes.. like that, if 4 unicode characters can be encoded into 4 utf-8 bytes (which mostly the case for latin-1 char range), then there will be 4 memory reads (to read four 32-bit unicode values) but only single memory write (to write four 8-bit utf-8 encoded values). The idea is to make utf-8 encoding speed close to memory copying speed :) convertUnicode: asm " input: - EAX 32-bit unicode value to convert output: - EAX - utf-8 encoded character (in little-endian byte order) max 4 bytes - EDX - number of encoded bytes " | moreThanOne moreThanTwo moreThanThree end | moreThanOne := asm uniqueLabelName: 'moreThanOne'. moreThanTwo := asm uniqueLabelName: 'moreThanTwo'. moreThanThree := asm uniqueLabelName: 'moreThanThree'. end := asm uniqueLabelName: 'end'. asm cmp: asm EAX with: 16r7F; jg: moreThanOne; "one byte" mov: 1 to: asm EDX; jmp: end; label: moreThanOne; cmp: asm EAX with: 16r7FF; jg: moreThanTwo; "two bytes 80 .. 7FF" " AH AL " "00000aaa aabbbbbb" "110aaaaa 10bbbbbb AL AH (little endian order) " shr: asm EAX with: 2; shl: asm AL with: 2; or: asm AX with: 2r1100000010000000; xchg: asm AL with: asm AH; mov: 2 to: asm EDX; jmp: end; label: moreThanTwo; cmp: asm EAX with: 16rFFFF; jg: moreThanThree; "three bytes 800 ... FFFF" " AH AL " "aaaabbbb bbcccccc" " => 1110aaaa 10bbbbbb 10cccccc" shl: asm EAX with: 4; shr: asm AX with: 2; shr: asm AL with: 2; " EAX = ...aaaa xxbbbbbb xxcccccc " or: asm EAX with: 2r111000001000000010000000; "16rE08080" shl: asm EAX with: 8; bswap: asm EAX; mov: 3 to: asm EDX; jmp: end; "four bytes 1000 ... 10FFFF" " AH AL " "000aaabb bbbbcccc ccdddddd" "=> 11110aaa 10bbbbbb 10cccccc 10dddddd" mov: asm EAX to: asm EDX; shl: asm EAX with: 4; shr: asm AX with: 2; shr: asm AL with: 2; " EAX = 0000000a aabbbbbb xxcccccc xxdddddd " and: asm EAX with: 16r3F3F3F; bswap: asm EAX; shr: asm EDX with: 18; "6*3" or: asm DL with: 16r11110000; mov: asm DL to: asm AL; mov: 4 to: asm EDX; label: end -- Best regards, Igor Stasenko.

June 13, 2012

Re: [Pharo-project] DBXTalk & seaside / success in Pier one-click / my 1.4 woes follow-up

by Cameron Sanders

Nick & Yanni, Starting with a clean Pharo 1.4-one-click, running on a Mac, and loading as listed below (see botton of page), does not deliver the desired results. A trivial Magritte described object, rendered (via #call:) "...asComponent addValidatedForm; yourself)" displays only the buttons, and no content editors. It silently fails. My test is the most trivial test of the Person tutorial for Magritte, grab my partial version from MCHttpRepository location: 'http://www.debrispublishing.com/camshared' user: '' password: '' and package is 'Cam'. See the class method of the Manager to register the app; then open a browser on localhost:8080/cam and when one adds a person, there is nothing but buttons. Perhaps the pattern of use is deprecated? CamPersonManager>>add | p | p := self call: (CamPerson new asComponent addValidatedForm ; yourself). p ifNotNil: [ self persons add: p] This same trivial sample code does work in one Pier image I was using. But I was trying really hard to move to a 1.4 image, for the numerous small improvements and fixes; and as Yanni suggested, to be positioned for future support. *If anyone has seaside/Magritte working in 1.4 would you please share with me your exact loading scripts (in proper sequence), or a snapshot of the minimal image (without any extras, of course)?* In the meantime I will try a 1.3 build ... again. I have put a lot of hours in trying to get a 1.4 build that works -- and a few failed 1.3 builds. Everyone else seems to do this in stride. But apparently, I am doing something wrong... dumb... idiotic... and my co-workers are starting to doubt me & my choice of platforms. Any help is most appreciated. Thanks in advance, Cam PS: what I loaded and how -- using Pier config to get seaside & magritte combos that work together (with regard to #description, etc.). Gofer new squeaksource: 'MetacelloRepository'; package: 'ConfigurationOfPier3'; load. (Smalltalk at: #ConfigurationOfPier3) load. Gofer it squeaksource: 'MetacelloRepository'; package: 'ConfigurationOfPier3AddOns'; load. (Smalltalk at: #ConfigurationOfPier3AddOns) load. "Error about ImageSegment not being available -- used by #serialize: which is used by #snapshot -- but not an issue for me right now, so Proceed" Gofer it squeaksource: 'MetacelloRepository'; package: 'ConfigurationOfFuel'; load. ((Smalltalk at: #ConfigurationOfFuel) project version: '1.8') load.

June 12, 2012

Re: [Pharo-project] DBXTalk & seaside / success in Pier one-click / my 1.4 woes follow-up

by Yanni Chiu

On 12/06/12 4:36 PM, Mariano Martinez Peck wrote: > > In we DO NOT load SIXX, or other serializers (but yes our Benchmark > package, progressUpate and examples) Maybe it was versions prior to 1.7 that included SIXX, but the 1 or 2 times I tried to use the ConfigurationOf it was pulling in a lot more than I wanted. Since there was only a single monticello package for the core (and one for the tests), I've been loading just that one package since then.

June 12, 2012

Re: [Pharo-project] Pharo 2.0 Jenkins build failing

by Sean P. DeNigris

StÃ©phane Ducasse wrote > > What is the problem? > Are you changes loading? > Then can we load something after :) > The problem is actually just before the new progress bar gets installed, in that same update (#20120). A related change to Collections-Abstract (Collection>>do:displayingProgress:every:) is causing a DNU when headless. I think I have a solution. I emailed you and esteban... -- View this message in context: http://forum.world.st/Pharo-2-0-Jenkins-build-failing-tp4634306p4634561.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

June 12, 2012

Re: [Pharo-project] Problem with FileHandle >> open [WAS] Re: FileReference should throw error when deleting unexisting files?

by Camillo Bruni

> Update to the last image 20129 there are 4 failing tests and I come to this conclusion. ok but these are whitebox tests which have to be adapted... (apparently my changes got lost there) FileSystem-Tests-Core-CamilloBruni.9 in PharoInbox fixes this :) (by changing the tests)

June 12, 2012