- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Error parsing XML File

by HernÃ¡n Morales Durand

Hi Alex, Try with the attached XML file in your Pharo directory XMLDOMParser parseDocumentFromFileNamed: (FileDirectory default fullNameFor: 'likelySubtags.xml') however, it seems to be fixed in XML-Parser-JAAyer.72 Cheers, HernÃ¡n 2010/3/27 Alexandre Bergel <alexandre(a)bergel.eu>: > Can you send me the file please? > > Alexandre > > > On 27 Mar 2010, at 12:47, Fabrizio Perin wrote: > >> Hi Alex, >> thanks for your effort. Actually the problem was related exactly to the >> readStream, my method to import the XML files uses the class FileStream, >> instead now it use StandardReadStream and everything works fine. All Tests >> where green in my image too (including your new test) but the error still >> raises trying to import from a file. So i investigate in the direction of >> the readStream from a file and i found the solution. I'm still not sure >> which is the problem using FileStream instead StandardFileStream. >> >> Thanks a lot, >> >> Fabrizio >> >> 2010/3/26 Alexandre Bergel <alexandre(a)bergel.eu> >> Hi Fabrizo, >> >> I think you're in the right place to talk about that. >> >> I haven't been able to reproduce your error. >> I added a test: >> >> XMLParserTest>>testNonUTF8Characters >> >> Â Â Â self shouldnt: [XMLDOMParser parseDocumentFrom: >> Â Â Â Â Â Â Â '<foo>Bean BLABLABLA EidgenÃ¶ssisches Institut fÃ¼r >> BLABLALBLA</foo>' readStream] raise: Error. >> >> It goes green in my image. Do you have a different way to get the >> readStream from the String? >> >> Cheers, >> Alexandre >> >> >> On 26 Mar 2010, at 12:14, Fabrizio Perin wrote: >> >> Hi, >> I was parsing an XML File with the last version of XML Parser >> (XML-Parser-JAAyer.68) and i get an error related to a not UTF-8 character >> that the parser found into the document. The XML document contains some >> german character: >> >> <![CDATA[SES: Bean BLABLABLA EidgenÃ¶ssisches Institut fÃ¼r BLABLALBLA]]> >> >> Actually i'm not sure if the error is which is in the UTF8TextConverter or >> something is wrong in the invokation from the parser. Anyway i parse several >> time the same document with older versions of the XML-Parser >> (XML-Parser-JAAyer.57) and it always works well. I'm not sure if the mailing >> list of Pharo is the right place to report this problem in the case i'm i'm >> sorry. >> >> Here the trace from the log: >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel Â http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

March 28, 2010

Re: [Pharo-project] Help system question / monticello

by StÃ©phane Ducasse

On Mar 28, 2010, at 7:00 PM, Michael Roberts wrote: > ... i have been looking at the situation. It is tricky. Ideally the > help would be near the code it is the help for. That way any API > described in help could be updated as the code is updated. I want to, > for example, commit some network help. > > However, as pointed out, the core help classes are not in the core > image so the help can't be loaded and therefore can't go in the core > monticello package. I did not check the help code but it should be just some text tagged so it could put somewhere. And I alwyays thought as in drdoc that we should a class per pakage or group of package where we can attach metadata. > This could be solved by having the few classes required to define help > in the core image. Or, we could look at extending the help system to > use pragmas to annotate the necessary methods without needing to > subclass CustomHelp. > > The other thing, is that in the HelpSystem project, there is a naming > convention already in use e.g Metacello-Help. So in principle I would > upload a new package Monticello-Help. > > However, Monticello is already defined in the core image as a package, > so if I create a package Monticello-Help I dirty the core package. > > So what should I do? I was thinking of uploading MonticelloHelp in the > short term to the help system project. Or I can just add the help to > the PharoProject-Help. > > Perhaps we should think about the help system a bit more deeply, in > case we want a convention we can follow for the long-term and scales. I suggest that we give a try to see how it flies. I'm not that having to be a subclass is good. In DrDoc I duplicated code just to make sure that we could load the package in absence of DrDoc. > > thanks, > Mike > > On Sun, Mar 28, 2010 at 5:01 PM, StÃ©phane Ducasse > <stephane.ducasse(a)inria.fr> wrote: >> yes >> >> On Mar 28, 2010, at 5:43 PM, laurent laffont wrote: >> >>> >>> 2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> >>> >>> >>> On Fri, Mar 26, 2010 at 5:37 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >>> Excellent idea. >>> I do not know where is the story about the help system. I would like something extremely simple. >>> >>> Published in the pharo inbox and I will merge it. >>> >>> >>> This is not as easy as that. If I understood correctly, he said HelpSystem classes. That is, extensions, to Torsten's HelpSystem. >>> Which is in http://www.squeaksource.com/HelpSystem >>> >>> That package is not in PharoCore, neither in PharoDev images for the moment (It can be added for PharoDev 1.1 if people want). >>> So...you will commit those classes....but if there is no HelpSystem, it has no sense. >>> >>> >>> Is there a sense in having a sort of Staging repository for things waiting for inclusion in Pharo ? Like the Linux kernel development process. If a package in Staging has a minimum of activity, developers around it, tests passes, then it goes into next Pharo major release. >>> >>> So you can put ConfigurationOfHelpSystem in Staging. Michael can add its package too. >>> >>> I also like the 2 weeks merge window of Linux where new modules are added. Then the merge window closes and begin the test / fix / debug process with several rc released. So new major versions are produced quite often and it's easy to upgrade from one major version to another one. >>> >>> Laurent Laffont >>> >>> >>> Cheers >>> >>> Mariano >>> >>> >>> >>> On Mar 25, 2010, at 7:40 PM, Michael Roberts wrote: >>> >>>> Hi, based on Lukas' recent blog post about Monticello merging, I >>>> created some help system classes for this content. What is the best >>>> way to integrate this? We have our own version of Monticello in the >>>> Pharo repository. Which package, would we add it to? Or do we want to >>>> add it upstream somewhere? >>>> >>>> Also, has anyone thought or spiked some simple formatting? that would >>>> be really nice. >>>> >>>> thanks, >>>> Mike >>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

March 28, 2010

Re: [Pharo-project] squeakToUTF-8 and related?

by StÃ©phane Ducasse

I do not think that we should only focus on the web. Smalltalk is much better than that. Now we should go step by step. I think that pharo is just at the beginning. STef

March 28, 2010

Re: [Pharo-project] Help system question / monticello

by Mariano Martinez Peck

On Sun, Mar 28, 2010 at 2:00 PM, Michael Roberts <mike(a)mjr104.co.uk> wrote: > ... i have been looking at the situation. It is tricky. Ideally the > help would be near the code it is the help for. That way any API > described in help could be updated as the code is updated. I want to, > for example, commit some network help. > > However, as pointed out, the core help classes are not in the core > image so the help can't be loaded and therefore can't go in the core > monticello package. > That's why I said it was not as easy as it seemed to be :) > > This could be solved by having the few classes required to define help > in the core image. Or, we could look at extending the help system to > use pragmas to annotate the necessary methods without needing to > subclass CustomHelp. > > I like very much this apporoach. It would like to have: Metacello-Core Metacello-Documentation and put in Documentation methods with pragmas. Or even those methods can be directly in Monticello-Core Then, when someone install HelpSystem (probably, not in the core but in the Dev), it search the pragmas and makes the HelpSystem with that. Similar to the new menu registration scheme with the difference that the one who looks the pragmas and do something, is not in the core but in a separate package. > The other thing, is that in the HelpSystem project, there is a naming > convention already in use e.g Metacello-Help. So in principle I would > upload a new package Monticello-Help. > Yes, it would be cool to define a convention. Help or Documentation ? which seems to be more abstract for you ? > > However, Monticello is already defined in the core image as a package, > so if I create a package Monticello-Help I dirty the core package. > > Ufffff yes...we always have the same problem with this.... > So what should I do? I was thinking of uploading MonticelloHelp in the > short term to the help system project. Or I can just add the help to > the PharoProject-Help. > > Perhaps we should think about the help system a bit more deeply, in > case we want a convention we can follow for the long-term and scales. > > thanks, > Mike > > On Sun, Mar 28, 2010 at 5:01 PM, StÃ©phane Ducasse > <stephane.ducasse(a)inria.fr> wrote: > > yes > > > > On Mar 28, 2010, at 5:43 PM, laurent laffont wrote: > > > >> > >> 2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> > >> > >> > >> On Fri, Mar 26, 2010 at 5:37 AM, StÃ©phane Ducasse < > stephane.ducasse(a)inria.fr> wrote: > >> Excellent idea. > >> I do not know where is the story about the help system. I would like > something extremely simple. > >> > >> Published in the pharo inbox and I will merge it. > >> > >> > >> This is not as easy as that. If I understood correctly, he said > HelpSystem classes. That is, extensions, to Torsten's HelpSystem. > >> Which is in http://www.squeaksource.com/HelpSystem > >> > >> That package is not in PharoCore, neither in PharoDev images for the > moment (It can be added for PharoDev 1.1 if people want). > >> So...you will commit those classes....but if there is no HelpSystem, it > has no sense. > >> > >> > >> Is there a sense in having a sort of Staging repository for things > waiting for inclusion in Pharo ? Like the Linux kernel development process. > If a package in Staging has a minimum of activity, developers around it, > tests passes, then it goes into next Pharo major release. > >> > >> So you can put ConfigurationOfHelpSystem in Staging. Michael can add its > package too. > >> > >> I also like the 2 weeks merge window of Linux where new modules are > added. Then the merge window closes and begin the test / fix / debug process > with several rc released. So new major versions are produced quite often and > it's easy to upgrade from one major version to another one. > >> > >> Laurent Laffont > >> > >> > >> Cheers > >> > >> Mariano > >> > >> > >> > >> On Mar 25, 2010, at 7:40 PM, Michael Roberts wrote: > >> > >> > Hi, based on Lukas' recent blog post about Monticello merging, I > >> > created some help system classes for this content. What is the best > >> > way to integrate this? We have our own version of Monticello in the > >> > Pharo repository. Which package, would we add it to? Or do we want to > >> > add it upstream somewhere? > >> > > >> > Also, has anyone thought or spiked some simple formatting? that would > >> > be really nice. > >> > > >> > thanks, > >> > Mike > >> > > >> > _______________________________________________ > >> > Pharo-project mailing list > >> > Pharo-project(a)lists.gforge.inria.fr > >> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > >> > >> > >> _______________________________________________ > >> Pharo-project mailing list > >> Pharo-project(a)lists.gforge.inria.fr > >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > >> > >> > >> _______________________________________________ > >> Pharo-project mailing list > >> Pharo-project(a)lists.gforge.inria.fr > >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > >> > >> _______________________________________________ > >> Pharo-project mailing list > >> Pharo-project(a)lists.gforge.inria.fr > >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010

Re: [Pharo-project] Squeak VM+FT2Plugin, all Pharo 1.0 rc3 tests green

by laurent laffont

Thank you ! Here the result: Squeak VM 3.11.3 -r 2135 + FT2Plugin + --CFLAGS="-O3 -fomit-frame-pointer -mfpmath=sse -march=native -s -mtune=generic" ========================================== 339072847 bytecodes/sec; 9560580 sends/sec 338848444 bytecodes/sec; 9552573 sends/sec 338177014 bytecodes/sec; 9473233 sends/sec 337953795 bytecodes/sec; 9496896 sends/sec 338848444 bytecodes/sec; 9496896 sends/sec so the best performance I have so far. Laurent Laffont On Sun, Mar 28, 2010 at 6:10 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 28 Mar 2010, laurent laffont wrote: > > Thanks for the link. When I clean my build directory and run configure >> with >> only one option (CFLAGS="-02") then it's OK. >> >> With several CFLAGS options it's strange, and now I understand why >> >> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-January/003740.html >> > > There's a full fix here: > http://lists.squeakfoundation.org/pipermail/vm-dev/2010-January/003742.html > > > >> I'm starting to think that git can be better for squeak-vm than svn, so we >> can propose branches with patches included.... >> >> > Probably yes. > > > Levente > > > Thanks >> >> Laurent Laffont >> >> >> On Sun, Mar 28, 2010 at 5:50 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> Your optimization flags are probably ignored by cmake: see this thread >>> >>> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-January/003736.html >>> >>> You could also give these flags a try on x86 cpus: -O3 >>> -fomit-frame-pointer >>> -mfpmath=sse -march=native >>> >>> >>> Levente >>> >>> >>> On Sun, 28 Mar 2010, laurent laffont wrote: >>> >>> More benchmarks. >>> >>>> >>>> Updated procedure with gcc optimizations activated: >>>> wget >>>> >>>> http://lolgzs.free.fr/pharo/squeak-vm/Squeak-3.11.3.2135-pharo-src.tar.gz >>>> tar -xvzf Squeak-3.11.3.2135-pharo-src.tar.gz >>>> cd Squeak-3.11.3.2135-pharo-src/ >>>> mkdir build && cd build >>>> ../unix/cmake/configure --CFLAGS="-O2" >>>> make >>>> sudo make install >>>> >>>> >>>> Squeak VM 3.11.3 -r 2135 + FT2Plugin >>>> ========================================== >>>> 334421946 bytecodes/sec; 8026581 sends/sec >>>> 334640522 bytecodes/sec; 7449884 sends/sec >>>> 334421946 bytecodes/sec; 7449884 sends/sec >>>> 334421946 bytecodes/sec; 7675486 sends/sec >>>> 327156549 bytecodes/sec; 7598782 sends/sec >>>> >>>> Squeak VM 3.11.3 -r 2135 + FT2Plugin + CFLAGS=-O2 >>>> ========================================== >>>> 330109606 bytecodes/sec; 9183392 sends/sec >>>> 331177231 bytecodes/sec; 9190792 sends/sec >>>> 330535829 bytecodes/sec; 9198204 sends/sec >>>> 329896907 bytecodes/sec; 9220511 sends/sec >>>> 330963154 bytecodes/sec; 8973857 sends/sec >>>> >>>> Squeak VM 3.11.3 -r 2135 + FT2Plugin + CFLAGS=-O3 >>>> ========================================== >>>> 296811594 bytecodes/sec; 10138464 sends/sec >>>> 296467863 bytecodes/sec; 10129460 sends/sec >>>> 296639629 bytecodes/sec; 10138464 sends/sec >>>> 296124927 bytecodes/sec; 7920675 sends/sec >>>> 296296296 bytecodes/sec; 7926179 sends/sec >>>> >>>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification >>>> ========================================== >>>> 306586826 bytecodes/sec; 7218843 sends/sec >>>> 308248043 bytecodes/sec; 7209717 sends/sec >>>> 308805790 bytecodes/sec; 7223415 sends/sec >>>> 309552599 bytecodes/sec; 7218843 sends/sec >>>> 309927360 bytecodes/sec; 7269453 sends/sec >>>> >>>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification + CFLAGS=-O2 >>>> ========================================== >>>> 293073840 bytecodes/sec; 8807546 sends/sec >>>> 292070735 bytecodes/sec; 8841684 sends/sec >>>> 281628162 bytecodes/sec; 8896858 sends/sec >>>> 294422081 bytecodes/sec; 8903804 sends/sec >>>> >>>> Laurent Laffont >>>> >>>> >>>> 2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> >>>> >>>> >>>> >>>>> 2010/3/27 laurent laffont <laurent.laffont(a)gmail.com> >>>>> >>>>> OK, I've tried the Smalltalk one... >>>>> >>>>> >>>>>> With rev 2151 gnuification (awk) is done as I have gnu-interp.c >>>>>> and gnu-interp.c.log in my build directory. >>>>>> >>>>>> Benchmark gives me: >>>>>> >>>>>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification >>>>>> ========================================== >>>>>> 312385600 bytecodes/sec; 7320778 sends/sec >>>>>> 310868245 bytecodes/sec; 8271046 sends/sec >>>>>> 310679611 bytecodes/sec; 8271046 sends/sec >>>>>> 310868245 bytecodes/sec; 7232576 sends/sec >>>>>> 310491206 bytecodes/sec; 8271046 sends/sec >>>>>> >>>>>> compared to >>>>>> >>>>>> Squeak VM 3.11.3 -r 2135 + FT2Plugin >>>>>> ========================================== >>>>>> 334421946 bytecodes/sec; 8026581 sends/sec >>>>>> 334640522 bytecodes/sec; 7449884 sends/sec >>>>>> 334421946 bytecodes/sec; 7449884 sends/sec >>>>>> 334421946 bytecodes/sec; 7675486 sends/sec >>>>>> 327156549 bytecodes/sec; 7598782 sends/sec >>>>>> >>>>>> so it seems that on my machine rev 2135 without gnuification is a >>>>>> little >>>>>> better. I have GCC 4.4.3, Linux 32bits. >>>>>> >>>>>> >>>>>> I don't understand....it is slower with the gnuification ? >>>>>> >>>>> >>>>> weird.... >>>>> >>>>> >>>>> >>>>> >>>>> Laurent Laffont >>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Mar 27, 2010 at 8:55 PM, Levente Uzonyi <leves(a)elte.hu> >>>>>> wrote: >>>>>> >>>>>> On Sat, 27 Mar 2010, laurent laffont wrote: >>>>>> >>>>>>> >>>>>>> I've tried gnuification on rev 2151. cmake/configure OK. But while >>>>>>> >>>>>>> building: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> /home/lol/sandbox/squeakvm/Squeak-3.11.3.2151-pharo-src/build/gnu-interp.c:25104:2: >>>>>>>> erreur: #error GNUIFICATION FAILED ( 0 ) >>>>>>>> >>>>>>>> >>>>>>>> Which gnuification method did you try? >>>>>>> There are two different versions. The origial version is an awk >>>>>>> script >>>>>>> which is run when you're building the vm with make. This is executed >>>>>>> by >>>>>>> default if you have awk installed. >>>>>>> There's another version written in smalltalk (linked by Mariano into >>>>>>> this >>>>>>> thread) which is not used by VMMAker at the moment, you probably >>>>>>> don't >>>>>>> need >>>>>>> that. If you applied the latter one, you have to make sure that the >>>>>>> first >>>>>>> one is not executed. You can't gnuifying the sources twice. >>>>>>> Since both do the same, the best you can do now is to ignore the >>>>>>> smalltalk version. If VMMaker will adopt Gnuifier.st we can remove >>>>>>> the >>>>>>> awk >>>>>>> dependency from the build process, until then you shouldn't apply it >>>>>>> if >>>>>>> your >>>>>>> platform has awk installed. >>>>>>> >>>>>>> >>>>>>> Levente >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Laurent Laffont >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Mar 27, 2010 at 6:47 PM, laurent laffont >>>>>>>> <laurent.laffont(a)gmail.com>wrote: >>>>>>>> >>>>>>>> Does it mean latest SVN source is already "Gnuified" ? With latest >>>>>>>> >>>>>>>> rev. >>>>>>>>> >>>>>>>>>> I >>>>>>>>>> >>>>>>>>>> have a little drop in benchmarks. However I'd rather put a stable >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> release >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> for Pharo 1.0 than latest trunk. What do you think ? >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The so-called "gnuification" is a transformation of the interp.c >>>>>>>>>> file. >>>>>>>>>> So >>>>>>>>>> this is done after generating the sources with VMMaker. The cmake >>>>>>>>>> configuration that comes with revision 2135 does not have this >>>>>>>>>> step >>>>>>>>>> enabled, >>>>>>>>>> and the more recent version that I used, revision 2151, has it. >>>>>>>>>> The >>>>>>>>>> magic is >>>>>>>>>> in the awk script in unix/cmake/gnuify.awk which produes >>>>>>>>>> gnu-interp.c. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK thanks. I will take a look. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Laurent Laffont >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>>> Adrian >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Laurent Laffont >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Adrian >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Pharo-project mailing list >>>>>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> >>>>>>>>>>>> Pharo-project mailing list >>>>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>>>> >>>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Pharo-project mailing list >>>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>>> >>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>> >>>>>>> Pharo-project mailing list >>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Pharo-project mailing list >>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Pharo-project mailing list >>>>> Pharo-project(a)lists.gforge.inria.fr >>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>> >>>>> >>>>> >>>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> >> > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010

Re: [Pharo-project] squeakToUTF-8 and related?

by Nicolas Cellier

2010/3/28 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: > >> You should ask Sophie team, their knowledge certainly is far more >> advanced than mine. > > The problem is that most of them disappeared after the java rewrite announce. > >> String should be a SequenceableCollection of Character. >> Internally, for space/speed reasons they rather store a code >> representing the value of a Character. >> >> In a simple model, this value would be the Unicode encoding... >> In squeak, only lowest 22 bits of a Character value are used to encode >> the character (#charCode). >> Bits of rank 23 to 30 encode a so called #leadingChar. >> I guess we stopped at bit #30 just to be sure to handle SmallInteger values. >> Don't count on me to explain leadingChar, I can't... > > :) > I read the comment of the class and some code and got lost > >> For leadingChar ~~ 0, i'm not event sure of correct charCode interpretation... >> >> For value < 256, the interpretation of the charCode is not exactly unicode... >> It's more CP1252 (with assigned values to codes from 128 to 159). >> >> Once upon a time, it used to be Mac Roman encoding instead... >> Let's forget the past (but you could so some remnants in old code). >> >> ------------------------------ >> >> When marshalling/unmarshalling strings to/from outside world we >> could/should use ByteArray... >> Unwisely, we don't. >> Instead, we reuse a String as storage for these codes. >> As a result, you see all these squeakToUtf8, utf8ToSqueak etc... >> That means that the contents of the String cannot be interpreted >> outside of its context... Very very bad IMHO. >> Under this point of view, the String has no more a self-contained >> meaning, but is just a blob of codes (on 8 or 32 bits). >> Fortunately, we mostly use these forms for temporary storage, but >> even, I don't like it. >> >> There are other alternatives like defining subclasses of String that >> encapsulate their encodings and know how to be well behaved Strings, >> not just context dependent blobs. >> For example, you could as well define an UT8String. >> VW went on this kind of path long time ago (not sure for utf8 though). >> >> Well, I'm not sure whether I succeeded in explaining something at all >> or just added confusion... > > don;t worry. > for the seaside book I started to read unicode standard and history now it would be good to > know what to do and do it :) > Ask Seaside folks, they certainly have some ideas. >> Anyway, Unicode is not simple, because it attempts to represent >> several centuries of typesetting conventions of different cultures... >> So don't expect the code to be as simple as in the ASCII times. >> It forces you to ask what is a character at all? Several glyphs exist >> for the same character (upper and lower case for a latin example), >> some characters can be decomposed as a base character and a >> diacritical mark, etc... > > Yes I read that. >> >> Character rendering is even worse, with kerning, ligatures, anti >> aliasing, hinting, etc... >> Designing a font of good quality is a lot of work, especially if you >> have to support unicode ! >> If it's getting too complex and we don't get the task force to handle >> it, we'd better hook OS primitives to measure/render. >> I guess it is far beyond you original question, but that will arise >> soon, because without good fonts and good rendering, Unicode support >> is kind of void. > > Yes this is all the question of a community not moving during 10 years (not only squeak) > and the world making progress and more important getting more and more complex. > So may be relying on external libraries will be more and more important (which I do not like). We have the Cuis alternative for simplicity. Not sure we should waste time competing in areas where we can't win. Seaside just take advantages of web standards and browsers, and it's the main commercial Smalltalk niche these days, isn't it ? Nicolas >> Nicolas >> >> 2010/3/28 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >>> Hi >>> >>> I'm trying to remember the situation with the internal representation of string in pharo/squeak >>> to revise http://book.seaside.st/book/in-action/serving-files/character-encodings/sea… >>> >>> I saw that in pharo we have this NonASCIIMap. I do not remember what have been done in pharo. >>> Argh memory leaks.... Nicolas do you remember the situation? >>> >>> In this context what is the squeakToUTF8 related behavior? >>> is squeak still using latin-1 or in the midst of changing? >>> >>> Stef >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010

Re: [Pharo-project] Squeak VM+FT2Plugin, all Pharo 1.0 rc3 tests green

by laurent laffont

2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> > Hi Laurent. I have been trying to compile the VM directly from your > sources, but I found some problems. > > The first one was that I haven't installed cmake. Thus, I have to do a sudo > apt-get install cmake. After that, it could do the configure. > > Then, the second error is that when trying to configure, there are some > libraries or headers not found. I attach the output of the configure > command. > In debian-based distro headers are in separate packages, the ones that ends with -dev. (flamewar inside:) that's why I've switched to ArchLinux, far more simple). So when a header is not found, like asoundlib.h, the easy way is to go to http://packages.ubuntu.com/ , go down the page to "Search the contents of packages" and fill the search field with the name of file. You will see that it's libasound2-dev package. In debian (I don't really know ubuntu), you have apt-file to search for a package's file: apt-file search asoundlib.h And a really great command is "auto-apt run" which installs automagically all needed packages for a given command. You can try: auto-apt run ../unix/cmake/configure when it detects a missing header, it will search the corresponding package and prompt you to install it. > > In addition, is the output of the configure written somewhere like > config.log or similar ? > It seems no. You can do ../unix/cmake/configure > config.log > > If you notice the file, there are several things not found, for example: > > !! XDisplayControlPlugin disabled > !! vm-display-Quartz disabled > !! vm-sound-MacOSX disabled > -- vm-sound-Sun: sys/audioio.h not found > -- vm-sound-Sun: sun/audioio.h not found > -- vm-sound-Sun: /usr/include/stropts.h > !! vm-sound-Sun disabled > -- Looking for sys/select.h > -- Looking for sys/select.h - found > !! vm-display-X11 disabled > -- vm-sound-NAS: audio/audio.h not found > -- vm-sound-NAS: libaudio not found > !! vm-sound-NAS disabled > -- vm-display-fbdev: /usr/include/linux/fb.h > -- vm-sound-pulse: libpulse-simple not found > !! vm-sound-pulse disabled > -- vm-sound-ALSA: alsa/asoundlib.h not found > -- vm-sound-ALSA: libasound not found > !! vm-sound-ALSA disabled > -- vm-sound-OSS: alsa/asoundlib.h not found > -- vm-sound-OSS: libasound not found > !! vm-sound-OSS disabled > > > See that it cannot find the vm-sound-OSS. > libasound-dev package http://packages.ubuntu.com/search?searchon=contents&keywords=libasound.a&mo… > > Then, I did a make and a make install. But when trying to run squeak, I > have this error: > > ubuntu@ubuntu-desktop:~/Pharo/vm/Squeak-3.11.3.2135-pharo-src/build$ > squeak > + exec padsp /usr/local/lib/squeak/3.11.13-2135/squeakvm -pathenc UTF-8 > -encoding UTF-8 -plugins /usr/local/lib/squeak/3.11.13-2135 -vm-sound-oss > could not find module vm-sound-OSS > Aborted > > > Of course it cannto find it..as I already know that from the configure. > So..here I have 2 questions: > > 1) how can I install the things I am missing ? > Now you know :) > > 2) why the "squeak" executable has "hardcoded" the parameters "-pathenc > UTF-8 -encoding UTF-8 -plugins /usr/local/lib/squeak/3.11.13-2135 > -vm-sound-oss" > is that correct ? do I ALWAYS want all those parameters ? And if I don't > want them, how can I execute a "clean" squeak ? > > May be you can remove sound related plugin... take a look at ../unix/cmake/configure --help ... --without-<plugin> do not build the named plugin .... You can also generate your own src using the Pharo-Core with VMMaker image I've made http://lolgzs.free.fr/pharo/squeak-vm/PharoCore-1.0-VMMaker-1.2.tar.gz VMMaker is already configured. Just remove the sound plugin from external plugins, adapt paths to your system and click "Entire". > > Finally, in another thread, Eliot told me how to be able to debug SqueakVM > with Gdb and threads. He told me to do: > > ../unix/cmake/configure CFLAGS="-g3 -pthread" LIBS=-lpthread > > but configure says: > > unknown option: CFLAGS=-g3 -pthread > unknown option: LIBS=-lpthread > you forgot the -- ../unix/cmake/configure --CFLAGS="-g3-pthread" Laurent Laffont > > Do you know how can I do this ? should I send this email to vm mailing > list ? > > Thanks a lot in advance! > > Mariano > > > > On Sun, Mar 28, 2010 at 12:50 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> Your optimization flags are probably ignored by cmake: see this thread >> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-January/003736.html >> >> You could also give these flags a try on x86 cpus: -O3 >> -fomit-frame-pointer -mfpmath=sse -march=native >> >> >> Levente >> >> >> On Sun, 28 Mar 2010, laurent laffont wrote: >> >> More benchmarks. >>> >>> Updated procedure with gcc optimizations activated: >>> wget >>> http://lolgzs.free.fr/pharo/squeak-vm/Squeak-3.11.3.2135-pharo-src.tar.gz >>> tar -xvzf Squeak-3.11.3.2135-pharo-src.tar.gz >>> cd Squeak-3.11.3.2135-pharo-src/ >>> mkdir build && cd build >>> ../unix/cmake/configure --CFLAGS="-O2" >>> make >>> sudo make install >>> >>> >>> Squeak VM 3.11.3 -r 2135 + FT2Plugin >>> ========================================== >>> 334421946 bytecodes/sec; 8026581 sends/sec >>> 334640522 bytecodes/sec; 7449884 sends/sec >>> 334421946 bytecodes/sec; 7449884 sends/sec >>> 334421946 bytecodes/sec; 7675486 sends/sec >>> 327156549 bytecodes/sec; 7598782 sends/sec >>> >>> Squeak VM 3.11.3 -r 2135 + FT2Plugin + CFLAGS=-O2 >>> ========================================== >>> 330109606 bytecodes/sec; 9183392 sends/sec >>> 331177231 bytecodes/sec; 9190792 sends/sec >>> 330535829 bytecodes/sec; 9198204 sends/sec >>> 329896907 bytecodes/sec; 9220511 sends/sec >>> 330963154 bytecodes/sec; 8973857 sends/sec >>> >>> Squeak VM 3.11.3 -r 2135 + FT2Plugin + CFLAGS=-O3 >>> ========================================== >>> 296811594 bytecodes/sec; 10138464 sends/sec >>> 296467863 bytecodes/sec; 10129460 sends/sec >>> 296639629 bytecodes/sec; 10138464 sends/sec >>> 296124927 bytecodes/sec; 7920675 sends/sec >>> 296296296 bytecodes/sec; 7926179 sends/sec >>> >>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification >>> ========================================== >>> 306586826 bytecodes/sec; 7218843 sends/sec >>> 308248043 bytecodes/sec; 7209717 sends/sec >>> 308805790 bytecodes/sec; 7223415 sends/sec >>> 309552599 bytecodes/sec; 7218843 sends/sec >>> 309927360 bytecodes/sec; 7269453 sends/sec >>> >>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification + CFLAGS=-O2 >>> ========================================== >>> 293073840 bytecodes/sec; 8807546 sends/sec >>> 292070735 bytecodes/sec; 8841684 sends/sec >>> 281628162 bytecodes/sec; 8896858 sends/sec >>> 294422081 bytecodes/sec; 8903804 sends/sec >>> >>> Laurent Laffont >>> >>> >>> 2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> >>> >>> >>>> >>>> 2010/3/27 laurent laffont <laurent.laffont(a)gmail.com> >>>> >>>> OK, I've tried the Smalltalk one... >>>> >>>>> >>>>> With rev 2151 gnuification (awk) is done as I have gnu-interp.c >>>>> and gnu-interp.c.log in my build directory. >>>>> >>>>> Benchmark gives me: >>>>> >>>>> Squeak VM 3.11.3 -r 2151 + FT2Plugin + Gnuification >>>>> ========================================== >>>>> 312385600 bytecodes/sec; 7320778 sends/sec >>>>> 310868245 bytecodes/sec; 8271046 sends/sec >>>>> 310679611 bytecodes/sec; 8271046 sends/sec >>>>> 310868245 bytecodes/sec; 7232576 sends/sec >>>>> 310491206 bytecodes/sec; 8271046 sends/sec >>>>> >>>>> compared to >>>>> >>>>> Squeak VM 3.11.3 -r 2135 + FT2Plugin >>>>> ========================================== >>>>> 334421946 bytecodes/sec; 8026581 sends/sec >>>>> 334640522 bytecodes/sec; 7449884 sends/sec >>>>> 334421946 bytecodes/sec; 7449884 sends/sec >>>>> 334421946 bytecodes/sec; 7675486 sends/sec >>>>> 327156549 bytecodes/sec; 7598782 sends/sec >>>>> >>>>> so it seems that on my machine rev 2135 without gnuification is a >>>>> little >>>>> better. I have GCC 4.4.3, Linux 32bits. >>>>> >>>>> >>>>> I don't understand....it is slower with the gnuification ? >>>> >>>> weird.... >>>> >>>> >>>> >>>> >>>> Laurent Laffont >>>>> >>>>> >>>>> >>>>> On Sat, Mar 27, 2010 at 8:55 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >>>>> >>>>> On Sat, 27 Mar 2010, laurent laffont wrote: >>>>>> >>>>>> I've tried gnuification on rev 2151. cmake/configure OK. But while >>>>>> >>>>>>> building: >>>>>>> >>>>>>> >>>>>>> >>>>>>> /home/lol/sandbox/squeakvm/Squeak-3.11.3.2151-pharo-src/build/gnu-interp.c:25104:2: >>>>>>> erreur: #error GNUIFICATION FAILED ( 0 ) >>>>>>> >>>>>>> >>>>>> Which gnuification method did you try? >>>>>> There are two different versions. The origial version is an awk script >>>>>> which is run when you're building the vm with make. This is executed >>>>>> by >>>>>> default if you have awk installed. >>>>>> There's another version written in smalltalk (linked by Mariano into >>>>>> this >>>>>> thread) which is not used by VMMAker at the moment, you probably don't >>>>>> need >>>>>> that. If you applied the latter one, you have to make sure that the >>>>>> first >>>>>> one is not executed. You can't gnuifying the sources twice. >>>>>> Since both do the same, the best you can do now is to ignore the >>>>>> smalltalk version. If VMMaker will adopt Gnuifier.st we can remove the >>>>>> awk >>>>>> dependency from the build process, until then you shouldn't apply it >>>>>> if your >>>>>> platform has awk installed. >>>>>> >>>>>> >>>>>> Levente >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Laurent Laffont >>>>>>> >>>>>>> >>>>>>> On Sat, Mar 27, 2010 at 6:47 PM, laurent laffont >>>>>>> <laurent.laffont(a)gmail.com>wrote: >>>>>>> >>>>>>> Does it mean latest SVN source is already "Gnuified" ? With latest >>>>>>> >>>>>>>> rev. >>>>>>>>> I >>>>>>>>> >>>>>>>>> have a little drop in benchmarks. However I'd rather put a stable >>>>>>>>>> >>>>>>>>>> release >>>>>>>>> >>>>>>>>> for Pharo 1.0 than latest trunk. What do you think ? >>>>>>>>>> >>>>>>>>>> >>>>>>>>> The so-called "gnuification" is a transformation of the interp.c >>>>>>>>> file. >>>>>>>>> So >>>>>>>>> this is done after generating the sources with VMMaker. The cmake >>>>>>>>> configuration that comes with revision 2135 does not have this step >>>>>>>>> enabled, >>>>>>>>> and the more recent version that I used, revision 2151, has it. The >>>>>>>>> magic is >>>>>>>>> in the awk script in unix/cmake/gnuify.awk which produes >>>>>>>>> gnu-interp.c. >>>>>>>>> >>>>>>>>> >>>>>>>>> OK thanks. I will take a look. >>>>>>>> >>>>>>>> Laurent Laffont >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>>> Adrian >>>>>>>>> >>>>>>>>> >>>>>>>>> Laurent Laffont >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Adrian >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Pharo-project mailing list >>>>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>>>> >>>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> >>>>>>>>>> Pharo-project mailing list >>>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>>> >>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Pharo-project mailing list >>>>>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>> Pharo-project mailing list >>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Pharo-project mailing list >>>>> Pharo-project(a)lists.gforge.inria.fr >>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>> >>>> >>> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010

Re: [Pharo-project] squeakToUTF-8 and related?

by StÃ©phane Ducasse

> You should ask Sophie team, their knowledge certainly is far more > advanced than mine. The problem is that most of them disappeared after the java rewrite announce. > String should be a SequenceableCollection of Character. > Internally, for space/speed reasons they rather store a code > representing the value of a Character. > > In a simple model, this value would be the Unicode encoding... > In squeak, only lowest 22 bits of a Character value are used to encode > the character (#charCode). > Bits of rank 23 to 30 encode a so called #leadingChar. > I guess we stopped at bit #30 just to be sure to handle SmallInteger values. > Don't count on me to explain leadingChar, I can't... :) I read the comment of the class and some code and got lost > For leadingChar ~~ 0, i'm not event sure of correct charCode interpretation... > > For value < 256, the interpretation of the charCode is not exactly unicode... > It's more CP1252 (with assigned values to codes from 128 to 159). > > Once upon a time, it used to be Mac Roman encoding instead... > Let's forget the past (but you could so some remnants in old code). > > ------------------------------ > > When marshalling/unmarshalling strings to/from outside world we > could/should use ByteArray... > Unwisely, we don't. > Instead, we reuse a String as storage for these codes. > As a result, you see all these squeakToUtf8, utf8ToSqueak etc... > That means that the contents of the String cannot be interpreted > outside of its context... Very very bad IMHO. > Under this point of view, the String has no more a self-contained > meaning, but is just a blob of codes (on 8 or 32 bits). > Fortunately, we mostly use these forms for temporary storage, but > even, I don't like it. > > There are other alternatives like defining subclasses of String that > encapsulate their encodings and know how to be well behaved Strings, > not just context dependent blobs. > For example, you could as well define an UT8String. > VW went on this kind of path long time ago (not sure for utf8 though). > > Well, I'm not sure whether I succeeded in explaining something at all > or just added confusion... don;t worry. for the seaside book I started to read unicode standard and history now it would be good to know what to do and do it :) > Anyway, Unicode is not simple, because it attempts to represent > several centuries of typesetting conventions of different cultures... > So don't expect the code to be as simple as in the ASCII times. > It forces you to ask what is a character at all? Several glyphs exist > for the same character (upper and lower case for a latin example), > some characters can be decomposed as a base character and a > diacritical mark, etc... Yes I read that. > > Character rendering is even worse, with kerning, ligatures, anti > aliasing, hinting, etc... > Designing a font of good quality is a lot of work, especially if you > have to support unicode ! > If it's getting too complex and we don't get the task force to handle > it, we'd better hook OS primitives to measure/render. > I guess it is far beyond you original question, but that will arise > soon, because without good fonts and good rendering, Unicode support > is kind of void. Yes this is all the question of a community not moving during 10 years (not only squeak) and the world making progress and more important getting more and more complex. So may be relying on external libraries will be more and more important (which I do not like). > Nicolas > > 2010/3/28 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >> Hi >> >> I'm trying to remember the situation with the internal representation of string in pharo/squeak >> to revise http://book.seaside.st/book/in-action/serving-files/character-encodings/sea… >> >> I saw that in pharo we have this NonASCIIMap. I do not remember what have been done in pharo. >> Argh memory leaks.... Nicolas do you remember the situation? >> >> In this context what is the squeakToUTF8 related behavior? >> is squeak still using latin-1 or in the midst of changing? >> >> Stef >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

March 28, 2010

Re: [Pharo-project] Help system question / monticello

by Michael Roberts

... i have been looking at the situation. It is tricky. Ideally the help would be near the code it is the help for. That way any API described in help could be updated as the code is updated. I want to, for example, commit some network help. However, as pointed out, the core help classes are not in the core image so the help can't be loaded and therefore can't go in the core monticello package. This could be solved by having the few classes required to define help in the core image. Or, we could look at extending the help system to use pragmas to annotate the necessary methods without needing to subclass CustomHelp. The other thing, is that in the HelpSystem project, there is a naming convention already in use e.g Metacello-Help. So in principle I would upload a new package Monticello-Help. However, Monticello is already defined in the core image as a package, so if I create a package Monticello-Help I dirty the core package. So what should I do? I was thinking of uploading MonticelloHelp in the short term to the help system project. Or I can just add the help to the PharoProject-Help. Perhaps we should think about the help system a bit more deeply, in case we want a convention we can follow for the long-term and scales. thanks, Mike On Sun, Mar 28, 2010 at 5:01 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > yes > > On Mar 28, 2010, at 5:43 PM, laurent laffont wrote: > >> >> 2010/3/28 Mariano Martinez Peck <marianopeck(a)gmail.com> >> >> >> On Fri, Mar 26, 2010 at 5:37 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >> Excellent idea. >> I do not know where is the story about the help system. I would like something extremely simple. >> >> Published in the pharo inbox and I will merge it. >> >> >> This is not as easy as that. If I understood correctly, he said HelpSystem classes. That is, extensions, to Torsten's HelpSystem. >> Which is in http://www.squeaksource.com/HelpSystem >> >> That package is not in PharoCore, neither in PharoDev images for the moment (It can be added for PharoDev 1.1 if people want). >> So...you will commit those classes....but if there is no HelpSystem, it has no sense. >> >> >> Is there a sense in having a sort of Staging repository for things waiting for inclusion in Pharo ? Like the Linux kernel development process. If a package in Staging has a minimum of activity, developers around it, tests passes, then it goes into next Pharo major release. >> >> So you can put ConfigurationOfHelpSystem in Staging. Michael can add its package too. >> >> I also like the 2 weeks merge window of Linux where new modules are added. Then the merge window closes and begin the test / fix / debug process with several rc released. So new major versions are produced quite often and it's easy to upgrade from one major version to another one. >> >> Laurent Laffont >> >> >> Cheers >> >> Mariano >> >> >> >> On Mar 25, 2010, at 7:40 PM, Michael Roberts wrote: >> >> > Hi, based on Lukas' recent blog post about Monticello merging, I >> > created some help system classes for this content. Â What is the best >> > way to integrate this? Â We have our own version of Monticello in the >> > Pharo repository. Which package, would we add it to? Â Or do we want to >> > add it upstream somewhere? >> > >> > Also, has anyone thought or spiked some simple formatting? that would >> > be really nice. >> > >> > thanks, >> > Mike >> > >> > _______________________________________________ >> > Pharo-project mailing list >> > Pharo-project(a)lists.gforge.inria.fr >> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010

Re: [Pharo-project] squeakToUTF-8 and related?

by Nicolas Cellier

You should ask Sophie team, their knowledge certainly is far more advanced than mine. String should be a SequenceableCollection of Character. Internally, for space/speed reasons they rather store a code representing the value of a Character. In a simple model, this value would be the Unicode encoding... In squeak, only lowest 22 bits of a Character value are used to encode the character (#charCode). Bits of rank 23 to 30 encode a so called #leadingChar. I guess we stopped at bit #30 just to be sure to handle SmallInteger values. Don't count on me to explain leadingChar, I can't... For leadingChar ~~ 0, i'm not event sure of correct charCode interpretation... For value < 256, the interpretation of the charCode is not exactly unicode... It's more CP1252 (with assigned values to codes from 128 to 159). Once upon a time, it used to be Mac Roman encoding instead... Let's forget the past (but you could so some remnants in old code). ------------------------------ When marshalling/unmarshalling strings to/from outside world we could/should use ByteArray... Unwisely, we don't. Instead, we reuse a String as storage for these codes. As a result, you see all these squeakToUtf8, utf8ToSqueak etc... That means that the contents of the String cannot be interpreted outside of its context... Very very bad IMHO. Under this point of view, the String has no more a self-contained meaning, but is just a blob of codes (on 8 or 32 bits). Fortunately, we mostly use these forms for temporary storage, but even, I don't like it. There are other alternatives like defining subclasses of String that encapsulate their encodings and know how to be well behaved Strings, not just context dependent blobs. For example, you could as well define an UT8String. VW went on this kind of path long time ago (not sure for utf8 though). Well, I'm not sure whether I succeeded in explaining something at all or just added confusion... Anyway, Unicode is not simple, because it attempts to represent several centuries of typesetting conventions of different cultures... So don't expect the code to be as simple as in the ASCII times. It forces you to ask what is a character at all? Several glyphs exist for the same character (upper and lower case for a latin example), some characters can be decomposed as a base character and a diacritical mark, etc... Character rendering is even worse, with kerning, ligatures, anti aliasing, hinting, etc... Designing a font of good quality is a lot of work, especially if you have to support unicode ! If it's getting too complex and we don't get the task force to handle it, we'd better hook OS primitives to measure/render. I guess it is far beyond you original question, but that will arise soon, because without good fonts and good rendering, Unicode support is kind of void. Nicolas 2010/3/28 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: > Hi > > I'm trying to remember the situation with the internal representation of string in pharo/squeak > to revise http://book.seaside.st/book/in-action/serving-files/character-encodings/sea… > > I saw that in pharo we have this NonASCIIMap. I do not remember what have been done in pharo. > Argh memory leaks.... Nicolas do you remember the situation? > > In this context what is the squeakToUTF8 related behavior? > is squeak still using latin-1 or in the midst of changing? > > Stef > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 28, 2010