October 2020 - Pharo-dev - lists.pharo.org

Re: [ANN] Libgit 1.0.0 in Pharo 9

by tesonep＠gmail.com

Hi Martin, thanks for testing it. Can you tell me the version / flavour of Debian are you using? Maybe we need to ship PCRE with the VM. I am not fun of that, but we need to do it until we can have a proper packaging for each Linux distribution (Esteban is working on having a nice OBS configuration for the VM extending what Holger has previously done). It takes time and a lot of testing, but the idea is to fix (or at least control better) the dependency nightmare between different Linux distributions. Cheers, Pablo On Tue, Oct 13, 2020 at 7:49 AM Martin McClure <martin(a)hand2mouse.com> wrote: > > Thanks, this is great news. > > Are there instructions on how to make P9 actually use the new version of > libgit2? I see that the stable Linux VM for 9.0 does include > "libgit2.1.0.0.so" but Pharo is still loading libgit2.so.0.25.1. > > ...this may be because Pharo's libgit2.1.0.0.so has not-found > dependencies on libpcre.so.3 and libpcreposix.so.3. > > I find Debian's versioning of libpcre somewhat confusing. The package > "libpcre3" is the *old* libraries, and new stuff is supposed to use > "pcre2." 2 is apparently newer than 3. At any rate PCRE library naming > is rather distro-specific, and although I have the right packages > installed, the libraries have different filenames. > > But libgit2 is not supposed to have PCRE as a dependency. The source > code for PCRE is included in the source for libgit2. Seems like it would > be possible (and nice!) to compile the libgit2.1.0.0.so included with > the VM to include the PCRE code internally, rather than depend on an > external library. > > Thanks, > -Martin > > On 10/12/20 2:26 AM, tesonep(a)gmail.com wrote: > > Hi, a later announcement (this have been done some months ago... but I > > never send the mail). > > > > We have upgraded in Pharo 9 to use the latest stable version of Libgit. > > It required to have some improvements in Libgit to handle the updated > > version and also to support the previous version. > > > > From the point of view of new features, it does not add new things. > > However, this version fixes a lot of existing problems in Libgit. > > > > We can be sure this was a successful deployment as we don't have > > problems with it. We are really happy that this has been done > > transparently and keeping the working version with different > > configurations of images and VMs. As we should support new and old > > images, on the same VM. And also running new images in old VMs > > > > Thanks. > > > -- Pablo Tesone. tesonep(a)gmail.com

Oct. 13, 2020

Re: [ANN] Libgit 1.0.0 in Pharo 9

by Martin McClure

Thanks, this is great news. Are there instructions on how to make P9 actually use the new version of libgit2? I see that the stable Linux VM for 9.0 does include "libgit2.1.0.0.so" but Pharo is still loading libgit2.so.0.25.1. ...this may be because Pharo's libgit2.1.0.0.so has not-found dependencies on libpcre.so.3 and libpcreposix.so.3. I find Debian's versioning of libpcre somewhat confusing. The package "libpcre3" is the *old* libraries, and new stuff is supposed to use "pcre2." 2 is apparently newer than 3. At any rate PCRE library naming is rather distro-specific, and although I have the right packages installed, the libraries have different filenames. But libgit2 is not supposed to have PCRE as a dependency. The source code for PCRE is included in the source for libgit2. Seems like it would be possible (and nice!) to compile the libgit2.1.0.0.so included with the VM to include the PCRE code internally, rather than depend on an external library. Thanks, -Martin On 10/12/20 2:26 AM, tesonep(a)gmail.com wrote: > Hi, a later announcement (this have been done some months ago... but I > never send the mail). > > We have upgraded in Pharo 9 to use the latest stable version of Libgit. > It required to have some improvements in Libgit to handle the updated > version and also to support the previous version. > > From the point of view of new features, it does not add new things. > However, this version fixes a lot of existing problems in Libgit. > > We can be sure this was a successful deployment as we don't have > problems with it. We are really happy that this has been done > transparently and keeping the working version with different > configurations of images and VMs. As we should support new and old > images, on the same VM. And also running new images in old VMs > > Thanks. >

Oct. 13, 2020

Re: P3Converter and DateAndTime>>#readFrom: performance

by Sven Van Caekenberghe

Hi Esteban, Thanks for this detailed feedback, this is most interesting. I don't have enough time to look at this in detail, I will do so later, but here is some initial feedback. Note that PostreSQL supports many different date/time formats and that the user can set preferences per connection, hence some flexibility in parsing is probably needed, and that is one reason why the parsing is implemented the way it is. Check section 8.5.2. from https://www.postgresql.org/docs/13/datatype-datetime.html Note that all of the following give the same result: Date readFrom: 'March 2 2020' readStream. Date readFrom: '3/2/2020' readStream. Date readFrom: '2020-03-02' readStream. Date parsing is probably slow because it has to deal with all these formats. Of course, parsing a fixed well-known format is always faster. The timezones are tricky too, as preferences can be set per connection, so we have to be careful. I will come back to this. Sven > On 12 Oct 2020, at 15:04, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi, > > I recently stumbled into some performance issue I had, so I profiled > it and got it fixed, but by doing that I also noticed a huge time > spent when reading DateAndTimes and Dates (which also uses the former) > and the culprit seems to be `Date class>>#readFrom:`. > > Digging into this, I noticed that this was called from > `DateAndTime>>#readFrom:`, but in particular 90% of the time spent was > when instantiating the Date part of it, because it does unnecessary > lookup of the index of Month when input stream only have a month > index. > > What I did then was to change the default behavior of DateAndTime > class>>#readFrom: > > readFrom: aStream defaultOffset: defaultOffset > "Parse and return a new DateAndTime instance from stream, > as a Date, an optional Time and an optional TimeZone offset. > The time defaults to midnight, the timezone to defaultOffset" > "self readFrom: '2013-03-04T23:47:52.876+01:00' readStream" > > "From this" > date := Date readFrom: aStream. > > "To this" > date := Date readFrom: (aStream next: 10) pattern: 'yyyy-mm-dd'. > > And also in P3Converter to specify the pattern directly. > > P3Converter>>#convertDateFrom: bytes length: length description: description > ^Date readFrom: (self asciiStreamFor: bytes length: length) pattern: > 'yyyy-mm-dd' > > With these two changes I got a 8x-10x speedup when reading > Date/DateAndTime from the database. > > Then I dig a little further and in P3Converter added a little check to > avoid translating the offset of a DateAndTime when it is not needed. > > P3Converter>>#convertDateAndTimeWithoutTimezoneFrom: bytes length: > length description: description > "TIMESTAMP WITHOUT TIME ZONE (TIMESTAMP) is stored internally in > Postgres the way it was inserted, its representation remains constant, > with no offset added. We use the timezone of the connection to do the > necessary shifting. This assumes that the timezones used during > insert/update and query are the same." > > | timestamp offset | > timestamp := DateAndTime readFrom: (self asciiStreamFor: bytes > length: length) defaultOffset: Duration zero. > offset := self timezone offsetForTimestamp: timestamp. > ^offset isZero "<-- added this check to avoid translation" > ifTrue: [ timestamp ] > ifFalse: [ timestamp translateTo: offset ] > > And the speedup went even higher. But I don't know whether this last > part is correct. > > Of course this seems as a quick and dirty fix, but I don't know > whether DateAndTime>>readFrom: should read any format or just ISO 8601 > formats nor whether the Date/DateAndTime coming from PostgreSQL are > always formatted that way. I ran my app's small test suite and haven't > had any issues, neither during development with these changes applied. > > I think there is a significant slowdown involved in the instantiation > of Dates that should be fixed, whether this fix is definitive or not > is why I'm asking here. > > Regards! > > Esteban A. Maringolo

Oct. 12, 2020

P3Converter and DateAndTime>>#readFrom: performance

by Esteban Maringolo

Hi, I recently stumbled into some performance issue I had, so I profiled it and got it fixed, but by doing that I also noticed a huge time spent when reading DateAndTimes and Dates (which also uses the former) and the culprit seems to be `Date class>>#readFrom:`. Digging into this, I noticed that this was called from `DateAndTime>>#readFrom:`, but in particular 90% of the time spent was when instantiating the Date part of it, because it does unnecessary lookup of the index of Month when input stream only have a month index. What I did then was to change the default behavior of DateAndTime class>>#readFrom: readFrom: aStream defaultOffset: defaultOffset "Parse and return a new DateAndTime instance from stream, as a Date, an optional Time and an optional TimeZone offset. The time defaults to midnight, the timezone to defaultOffset" "self readFrom: '2013-03-04T23:47:52.876+01:00' readStream" "From this" date := Date readFrom: aStream. "To this" date := Date readFrom: (aStream next: 10) pattern: 'yyyy-mm-dd'. And also in P3Converter to specify the pattern directly. P3Converter>>#convertDateFrom: bytes length: length description: description ^Date readFrom: (self asciiStreamFor: bytes length: length) pattern: 'yyyy-mm-dd' With these two changes I got a 8x-10x speedup when reading Date/DateAndTime from the database. Then I dig a little further and in P3Converter added a little check to avoid translating the offset of a DateAndTime when it is not needed. P3Converter>>#convertDateAndTimeWithoutTimezoneFrom: bytes length: length description: description "TIMESTAMP WITHOUT TIME ZONE (TIMESTAMP) is stored internally in Postgres the way it was inserted, its representation remains constant, with no offset added. We use the timezone of the connection to do the necessary shifting. This assumes that the timezones used during insert/update and query are the same." | timestamp offset | timestamp := DateAndTime readFrom: (self asciiStreamFor: bytes length: length) defaultOffset: Duration zero. offset := self timezone offsetForTimestamp: timestamp. ^offset isZero "<-- added this check to avoid translation" ifTrue: [ timestamp ] ifFalse: [ timestamp translateTo: offset ] And the speedup went even higher. But I don't know whether this last part is correct. Of course this seems as a quick and dirty fix, but I don't know whether DateAndTime>>readFrom: should read any format or just ISO 8601 formats nor whether the Date/DateAndTime coming from PostgreSQL are always formatted that way. I ran my app's small test suite and haven't had any issues, neither during development with these changes applied. I think there is a significant slowdown involved in the instantiation of Dates that should be fixed, whether this fix is definitive or not is why I'm asking here. Regards! Esteban A. Maringolo

Oct. 12, 2020

[ANN] Libgit 1.0.0 in Pharo 9

by tesonep＠gmail.com

Hi, a later announcement (this have been done some months ago... but I never send the mail). We have upgraded in Pharo 9 to use the latest stable version of Libgit. It required to have some improvements in Libgit to handle the updated version and also to support the previous version. >From the point of view of new features, it does not add new things. However, this version fixes a lot of existing problems in Libgit. We can be sure this was a successful deployment as we don't have problems with it. We are really happy that this has been done transparently and keeping the working version with different configurations of images and VMs. As we should support new and old images, on the same VM. And also running new images in old VMs Thanks. -- Pablo Tesone. tesonep(a)gmail.com

Oct. 12, 2020

Re: [Pharo-users] [ANN] Pharo Consortium New Gold Member: InfOil

by StÃ©phane Ducasse

us too :) S. > On 30 Sep 2020, at 16:39, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > As a former employee of the company I'm really happy to see this > finally happening! > > Best regards! > > Esteban A. Maringolo > > On Wed, Sep 30, 2020 at 11:13 AM Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> >> The Pharo Consortium is very happy to announce that InfOil has joined the Consortium as a Gold Member. >> >> InfOil is an Argentinian company focused on software development for the Oil and Gas industry with clients >> in Latin America, Cuba, and Canada. >> >> About >> - InfOil: http://www.infoil.com.ar >> - Pharo Consortium: http://consortium.pharo.org >> >> The goal of the Pharo Consortium is to allow companies and institutions to support the ongoing development >> and future of Pharo. >> >> Individuals can support Pharo via the Pharo Association: >> >> - http://association.pharo.org -------------------------------------------- StÃ©phane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, BÃ¢t.A, Park Plaza Villeneuve d'Ascq 59650 France

Oct. 11, 2020

This week (41/2020) on the Issue Tracker

by Marcus Denker

This week (41/2020) on the Issue Tracker ================================ Hi, We had the idea to try to do a summary of all the things that got merged during the week. I tried for a while an automatic tool (a Probot bot), but that only works for some weeks and then stops (I think they have some rate limiting problem for the bot). And it was not really usefulâ¦ as it just listed all PRs and Issues. This one puts a focus on *merged* PRs to Pharo itself only and sorts them by category with a little bit of context. Parser/Syntax Highlighting ======================= Improvement to make the prior huge improvement on parsing of syntactically wrong code much more usable: Several fixes to the parser and syntax feedback #7411 https://github.com/pharo-project/pharo/pull/7411 First class Variables / Compiler cleanup ==================== Lots of work has been done to unify all meta objects describing Variables into one hierarchy (see Variable and subclasses). This in turn enables many cleanups and simplification on the level of the name analysis of the Compiler and the API that the debugger uses to read Variables. Nearly finished, but some smaller steps remain: Variable-scope-addTests #7435 (all Variable objects should know the scope where they are from) https://github.com/pharo-project/pharo/pull/7435 Name Analysis: Lookup Slots In Class #7403 https://github.com/pharo-project/pharo/pull/7403 reduce-sends-doSemanticAnalysis #7402 https://github.com/pharo-project/pharo/pull/7402 7405-Debug-It-is-broken #7406 (this was a bad side effect of a refactoring needed to be able to move the call of #doSemanticAnalysis into the Compiler) https://github.com/pharo-project/pharo/pull/7406 Fluid Class Definions ====================== This is ongoing work to have a better class definiton for what is now called "slot enabeled class defintion" (the one where one can use First class Variables and where the Layout class is shown explicitly) 7444-Prepare-fluid-class-integration-part-4 #7445 https://github.com/pharo-project/pharo/pull/7445 7437-prepare-fluid-class-integration-part-3 #7438 https://github.com/pharo-project/pharo/pull/7438 7412-preparing-the-integration-of-fluid-class-part-2 #7434 https://github.com/pharo-project/pharo/pull/7434 7431-preparing-fluid-class-Part-1 #7433 https://github.com/pharo-project/pharo/pull/7433 Debugger ======= Debugger-should-show-Workspace-bindings #7408 A try to simplify how the workspace binding is found when debugging. Leads to many problems, will be reverted. https://github.com/pharo-project/pharo/pull/7408 7362-Package-Debugger-Filters-is-not-used-and-can-be-removed #7430 https://github.com/pharo-project/pharo/pull/7430 Added a closeAllDebugger method to trait TDebugger class, that users â¦ #7328 (just a first step, not yet solved) https://github.com/pharo-project/pharo/pull/7328 Exposing signalContext in Exception #7417 https://github.com/pharo-project/pharo/pull/7417 ReadOnly Literals ================= The compiler and the VM has support to make literals read-only. As the ability to change literals without the code reflecting it has lead to ugly bugs in the past, turning this on looks like a good idea: SourceFileBufferedReadWriteStreamTest-readOnlyLiteral #7418 https://github.com/pharo-project/pharo/pull/7418 betterName-isSharedLiteral #7432 https://github.com/pharo-project/pharo/pull/7432 Enable-ReadOnly-Literals #7393 https://github.com/pharo-project/pharo/pull/7393 Bug Fixes ========== Background failures log in CI #7337 https://github.com/pharo-project/pharo/pull/7337 5956-Wrong-format-of-DoubleWordArray #7404 Ongoing work. All code is merged, but now we need to still see that we can create classes like that with the bootrap https://github.com/pharo-project/pharo/pull/7404 Fix_for_PluggableSliderMorph #7415 https://github.com/pharo-project/pharo/pull/7415 7414-division-of-number-by-array-fails-with-doesNotUnderstand-isZero #7425 https://github.com/pharo-project/pharo/pull/7425 Cleaning integer division #7436 https://github.com/pharo-project/pharo/pull/7436 Comments ======== 7042-Better-comment-for-CompiledBlock-and-FullBlock https://github.com/pharo-project/pharo/pull/7447 Cleanups ======== 7367-pragmas-not-needed-textEditorShiftedMenu-fileListContentMenu #7426 https://github.com/pharo-project/pharo/pull/7426 Fixed #7420. Added transformation rules to some non-transforming deprecations that were identified as automatable in the Deprewriter paper #7427 https://github.com/pharo-project/pharo/pull/7427 7364-RBSelectorNode-has-unclassified-methods #7424 https://github.com/pharo-project/pharo/pull/7424 7416-Fileout-a-protocol-raises-a-DNU #7441 https://github.com/pharo-project/pharo/pull/7441 7365-DictionaryValueHold-removeKey-calls-a-non-existing-method #7407 https://github.com/pharo-project/pharo/pull/7407 For a complete list of all issues and PRs, see: https://github.com/pharo-project/pharo/pulse

Oct. 9, 2020

Re: Super nice parser improvements soon available

by Tim Mackinnon

That is awesome work - I love you guys for continuing to improve this stuff! Itâs inspiring... Tim > On 3 Oct 2020, at 10:24, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > > ï»¿ > >> In the last week Iâve worked on doing several enhances to the parser >> >> https://github.com/pharo-project/pharo/pull/7411 >> >> Besides several refactorings and cleanups, major changes include: >> >> - we are able to correctly detect unfinished statements (missing dots) >> - we are able to correctly detect unreachable statements (like a statement after a return statement) >> >> And we still continue parsing. This makes that the faulty parser used for syntax highlighting and other much more precise, and several of my touches also fix some of the exceptions found while typing (that were due to wrong behaviours on the parser). >> >> Also I did a small iteration on top of the âsyntax feedbackâ so itâs less noisy. We can remove it at any time (thatâs just removing a configuration). But Iâd like more feedback before :) >> >> To see the improvements >> >> Pharo 8 >> >> <PastedGraphic-3.png> >> >> Pharo9 previous to these changes >> >> <PastedGraphic-1.png> >> >> Pharo9 with these changes >> >> <PastedGraphic-2.png> >> >> >> Now, Iâll not deny this is still perfectible, as everything, but we can do another iteration on top :) >> >> Guille >> _______________________________________________ >> Lse-consortium mailing list >> Lse-consortium(a)lists.gforge.inria.fr >> https://lists.gforge.inria.fr/mailman/listinfo/lse-consortium > > -------------------------------------------- > StÃ©phane Ducasse > http://stephane.ducasse.free.fr / http://www.pharo.org > 03 59 35 87 52 > Assistant: Aurore Dalle > FAX 03 59 57 78 50 > TEL 03 59 35 86 16 > S. Ducasse - Inria > 40, avenue Halley, > Parc Scientifique de la Haute Borne, BÃ¢t.A, Park Plaza > Villeneuve d'Ascq 59650 > France >

Oct. 3, 2020

Super nice parser improvements soon available

by StÃ©phane Ducasse

> In the last week Iâve worked on doing several enhances to the parser > > https://github.com/pharo-project/pharo/pull/7411 <https://github.com/pharo-project/pharo/pull/7411> > > Besides several refactorings and cleanups, major changes include: > > - we are able to correctly detect unfinished statements (missing dots) > - we are able to correctly detect unreachable statements (like a statement after a return statement) > > And we still continue parsing. This makes that the faulty parser used for syntax highlighting and other much more precise, and several of my touches also fix some of the exceptions found while typing (that were due to wrong behaviours on the parser). > > Also I did a small iteration on top of the âsyntax feedbackâ so itâs less noisy. We can remove it at any time (thatâs just removing a configuration). But Iâd like more feedback before :) > > To see the improvements > > Pharo 8 > > > > Pharo9 previous to these changes > > > > Pharo9 with these changes > > > > > Now, Iâll not deny this is still perfectible, as everything, but we can do another iteration on top :) > > Guille > _______________________________________________ > Lse-consortium mailing list > Lse-consortium(a)lists.gforge.inria.fr > https://lists.gforge.inria.fr/mailman/listinfo/lse-consortium -------------------------------------------- StÃ©phane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, BÃ¢t.A, Park Plaza Villeneuve d'Ascq 59650 France

Oct. 3, 2020

Re: beta testers for the new version of the Pharo mooc

by tbrunz

Hi Stef, Do you have information on the schedule yet? (Times of day) I'm asking people, and they want to know how much time/how to schedule this. Thanks, -Ted -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Oct. 3, 2020