- Pharo-dev - lists.pharo.org

[Pharo-project] Fastest utf-8 encoder contest
by Igor Stasenko June 21, 2012

June 21, 2012

Hi, hardcore hackers. please take a look at the code and tell if it can be improved. The AsmJit snippet below transforms an unicode integer value to 1..4-byte sequence of utf-8 then the outer piece of code (which is not yet written) will accumulate the results of this snippet to do a memory-aligned (4byte) writes.. like that, if 4 unicode characters can be encoded into 4 utf-8 bytes (which mostly the case for latin-1 char range), then there will be 4 memory reads (to read four 32-bit unicode values) but only single memory write (to write four 8-bit utf-8 encoded values). The idea is to make utf-8 encoding speed close to memory copying speed :) convertUnicode: asm " input: - EAX 32-bit unicode value to convert output: - EAX - utf-8 encoded character (in little-endian byte order) max 4 bytes - EDX - number of encoded bytes " | moreThanOne moreThanTwo moreThanThree end | moreThanOne := asm uniqueLabelName: 'moreThanOne'. moreThanTwo := asm uniqueLabelName: 'moreThanTwo'. moreThanThree := asm uniqueLabelName: 'moreThanThree'. end := asm uniqueLabelName: 'end'. asm cmp: asm EAX with: 16r7F; jg: moreThanOne; "one byte" mov: 1 to: asm EDX; jmp: end; label: moreThanOne; cmp: asm EAX with: 16r7FF; jg: moreThanTwo; "two bytes 80 .. 7FF" " AH AL " "00000aaa aabbbbbb" "110aaaaa 10bbbbbb AL AH (little endian order) " shr: asm EAX with: 2; shl: asm AL with: 2; or: asm AX with: 2r1100000010000000; xchg: asm AL with: asm AH; mov: 2 to: asm EDX; jmp: end; label: moreThanTwo; cmp: asm EAX with: 16rFFFF; jg: moreThanThree; "three bytes 800 ... FFFF" " AH AL " "aaaabbbb bbcccccc" " => 1110aaaa 10bbbbbb 10cccccc" shl: asm EAX with: 4; shr: asm AX with: 2; shr: asm AL with: 2; " EAX = ...aaaa xxbbbbbb xxcccccc " or: asm EAX with: 2r111000001000000010000000; "16rE08080" shl: asm EAX with: 8; bswap: asm EAX; mov: 3 to: asm EDX; jmp: end; "four bytes 1000 ... 10FFFF" " AH AL " "000aaabb bbbbcccc ccdddddd" "=> 11110aaa 10bbbbbb 10cccccc 10dddddd" mov: asm EAX to: asm EDX; shl: asm EAX with: 4; shr: asm AX with: 2; shr: asm AL with: 2; " EAX = 0000000a aabbbbbb xxcccccc xxdddddd " and: asm EAX with: 16r3F3F3F; bswap: asm EAX; shr: asm EDX with: 18; "6*3" or: asm DL with: 16r11110000; mov: asm DL to: asm AL; mov: 4 to: asm EDX; label: end -- Best regards, Igor Stasenko.

5 13

[Pharo-project] SimplifiedChineseEnvironment>>leadingChar
by Hilaire Fernandes June 21, 2012

June 21, 2012

Hello, I am curious to know what is the right fix to this issue when the locale is chinese: http://code.google.com/p/pharo/issues/detail?id=6117 Thanks Hilaire -- Dr. Geo -- http://www.drgeo.eu

1 0

[Pharo-project] [update 2.0] #20150
by Esteban Lorenzano June 21, 2012

June 21, 2012

20150 ----- Issue 6113: jenkins pharo tests for windows http://code.google.com/p/pharo/issues/detail?id=6113 (not working yet... but close)

1 0

[Pharo-project] Tests Gueard report for 20144
by Pavel Krivanek June 21, 2012

June 21, 2012

Tests Gueard report for 20144 http://code.google.com/p/pharo/wiki/TestsGuard Because the Pharo job and the test job were broken for some time, this report collects several updates. We have no platform dependent test failure again. A lot of thests is now green but we have some new failures. Fixed tests: * CompressionTests.Archive.ZipArchiveTest.testAddNonExistentFile * CompressionTests.Archive.ZipArchiveTest.testDate29Feb2000 * CompressionTests.Archive.ZipArchiveTest.testPackDate * CompressionTests.Archive.ZipArchiveTest.testPackTime * CompressionTests.Archive.ZipArchiveTest.testSetLastModification * CompressionTests.Archive.ZipArchiveTest.testUnpackDate * CompressionTests.Archive.ZipArchiveTest.testUnpackTime * FileSystem.Tests.Core.FileReferenceTest.testRename * FileSystem.Tests.Disk.DiskFileSystemTest.testCreateDirectoryNoParent * FileSystem.Tests.Disk.DiskFileSystemTest.testEnsureDirectory * FileSystem.Tests.Disk.DiskFileSystemTest.testEnsureDirectoryCreatesParent * FileSystem.Tests.Disk.DiskFileSystemTest.testFileNames * FileSystem.Tests.Disk.DiskFileSystemTest.testReadStreamDoIfAbsent * FileSystem.Tests.Disk.DiskFileSystemTest.testReadStreamIfAbsent * FileSystem.Tests.Disk.DiskFileSystemTest.testWriteStreamDoIfPresent * FileSystem.Tests.Disk.DiskFileSystemTest.testWriteStreamIfPresent * FileSystem.Tests.Disk.FileHandleTest.testEnsureClosed * Graphics.Tests.Files.PNGReadWriterTest.test1BitColors * Graphics.Tests.Files.PNGReadWriterTest.test2BitColors * Graphics.Tests.Files.PNGReadWriterTest.test4BitColors * Graphics.Tests.Files.PNGReadWriterTest.test8BitColors * Multilingual.Tests.TextConversion.MultiByteFileStreamTest.testBasicChunk * Multilingual.Tests.TextConversion.MultiByteFileStreamTest.testBinaryUpTo * Tests.System.CommandLine.BasicCodeLoaderTest.testInstallSourceFilesFail * CompressionTests.Archive.ZipArchiveTest.testCreateWithRelativeNames * CompressionTests.Archive.ZipArchiveTest.testFilePermissions * CompressionTests.Archive.ZipArchiveTest.testZip * FileSystem.Tests.Disk.DiskFileSystemTest.testRootIsNotAFile * Gofer.Tests.GoferApiTest.testDirectoryRepository * Gofer.Tests.GoferApiTest.testSubDirectoryRepository New failures: * Tests.Monticello.MCDirectoryRepositoryTest.testIncludesName * Tests.Monticello.MCDirectoryRepositoryTest.testStoreAndLoad * Tests.Monticello.MCWorkingCopyTest.testAncestorMerge * Tests.Monticello.MCWorkingCopyTest.testBackport * Tests.Monticello.MCWorkingCopyTest.testDoubleRepeatedMerge * Tests.Monticello.MCWorkingCopyTest.testMergeIntoImageWithNoChanges * Tests.Monticello.MCWorkingCopyTest.testMergeIntoUnmodifiedImage * Tests.Monticello.MCWorkingCopyTest.testNaming * Tests.Monticello.MCWorkingCopyTest.testOptimizedLoad * Tests.Monticello.MCWorkingCopyTest.testRedundantMerge * Tests.Monticello.MCWorkingCopyTest.testRepeatedMerge * Tests.Monticello.MCWorkingCopyTest.testRepositoryFallback * Tests.Monticello.MCWorkingCopyTest.testSelectiveBackport * Tests.Monticello.MCWorkingCopyTest.testSimpleMerge * Tests.Monticello.MCWorkingCopyTest.testSnapshotAndLoad * Tests.Release.ReleaseTest.testObsoleteClasses -- Pavel

4 7

[Pharo-project] [update 2.0] #20149
by Esteban Lorenzano June 21, 2012

June 21, 2012

20149 ----- Issue 6097: Bad source and changes path for one click on Windows (Jenkins?) http://code.google.com/p/pharo/issues/detail?id=6097 Issue 6096: WindowsResolver sends "self default" but no method #default is implemented http://code.google.com/p/pharo/issues/detail?id=6096 (windows should be working now)

1 0

[Pharo-project] [update 2.0] #20145
by Esteban Lorenzano June 21, 2012

June 21, 2012

20145 ----- Issue 5945: Nice packaging for Text stuff http://code.google.com/p/pharo/issues/detail?can=2&q=5945

4 13

[Pharo-project] [update 2.0] #20148
by Esteban Lorenzano June 21, 2012

June 21, 2012

20148 ----- Issue 6097: Bad source and changes path for one click on Windows (Jenkins?) http://code.google.com/p/pharo/issues/detail?id=6097

1 0

[Pharo-project] [update 2.0] #20147
by Esteban Lorenzano June 21, 2012

June 21, 2012

20147 ----- Issue 6098: Cleaning obsolete repositories in Gofer http://code.google.com/p/pharo/issues/detail?id=6098

1 0

[Pharo-project] [update 2.0] #20146
by Esteban Lorenzano June 21, 2012

June 21, 2012

20146 ----- Issue 6097: Bad source and changes path for one click on Windows (Jenkins?) http://code.google.com/p/pharo/issues/detail?id=6097 (but still not working)

1 0

[Pharo-project] Current Jenkins Pharo 2.0 images broken on Windows
by Torsten Bergmann June 21, 2012

June 21, 2012

Hi, please note that for Windows Users the latest Jenkins images for 2.0 are currently unusable due to the latest FS changes. I found the cause last night and described the details in http://code.google.com/p/pharo/issues/detail?id=6097 Here is the red test that should be made green: testWindowsAbsolutePathName self assert: (AbsolutePath from: 'c:\test') isAbsolute by fixing Path>>from:delimiter. Currently this method only takes Unix root "/" into account and returns a relative path instead of an absolute one on Windows. This causes a lot of other FS-related stuff to fail. It has to be fixed by someone on a non-Windows platform since I can not be fixed in the broken image itself on a Win system. There is also a bug in WindowsStore>>pathFromString:, all details in #6097. Would be nice if this could be fixed quickly - maybe Camillo is already looking... Bye T. -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

2 1