July 2017 - Pharo-dev - lists.pharo.org

Discussing the roadmap

by Stephane Ducasse

We would like to share this list with you and get your feedback and inputs. It will be presented and discussed again at ESUG. Stef on the behalf of the engineering team of the Pharo consortium. # Pharo 7 (and 8) potential roadmap This document contains a list of actions that should be done during Pharo 7 and 8. It is not a definitive list. It is the result of a discussion within the engineer team and RModers. It first lists actions that should be performed at the image level then lists the actions for the VM. ## Image As a general principle, we will try to remove something when we add a new features or version of a component. ### New tools - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo. For the moment it supports Git. This tool will become central to the development of projects in Pharo. The key and first enhancements will be: - cherry picking - multiple directories support, subtrees - better new development process support - Calypso: Calypso is a new tool suite for editing and navigating code. It is modular and can easily be extended. - integration, set as the default browser - Hermes (binary loader): Hermes is a code loader that does not require the compiler to be loaded. It will be used to speed up the bootstrap. - Beacon: Beacon is an announcement-based logger. It should be introduced but in addition the left over of previous logging should be cleaned and removed for example many of the transcript show: should be removed. - Inspector refreshing: the inspector should refresh its values by default. - Cargo: Cargo is a new package manager. It supports the expression of dependencies between packages. We are currently validating that it can support conditional loading (platform) and building new tooling to express and validate data. - Check dependencies when committing. Pharo comes with a dependency analyser tools. Such tools should be used before commiting to warn the user when a new dependency is introduced in a package. ### Cleaning bloat - Removing of Nautilus: Once Calypso will be integrated and exhibit similar fetaures than Nautilus. Nautilus should be removed. - Remove old text editor: there is only one or two widgets still using the old text editor and the rubric text editor. - Remove TxText: TxText was an attempt to get a new generation text editor. Now it has been rewritten with a new design in Bloc so we should remove it since Bloc editor is better and actively maintained. - Remove Komitter: Iceberg already supports cherrypicking on commit therefore Komitter can be safely removed from Pharo. - Remove system categorizer: The old system categorizer is not used anymore and should be remove. - Old compiler removal: The old compiler should now get unloaded from Pharo. - The old compiler should be moved to an external package and make sure that it can be reloaded. - In addition the encoders should be separated. (@@ more here@@) - Old inspector cleanup: we should remove the eyeInspector framework but we should introduce a minimal fallback inspector. This inspector should work without Spec or any frameworks. - Clean behavior protocol. The number of methods in the core Behavior, ClassDescription and Class requires some analysis and cleaning. - Kernel modularization: the effort on modularising the packages should be continued. ### Language infrastructure The following points are more related to the infrastructure of manipulating and loading class definitions. There are the basis for the future module system and cleaning some often hidden parts of the system. - New class definition: The class definition is not scaling anymore due to the large number of options (traits, slot, tage, package, immediate, ephemeron). A fluid-based message API should be designed. - Support for Undefined classes: When loading old code or code whose superclass has not yet being loaded, the system has inconsistent behavior. Depending on the tools, it may not load the code, raise a warning or decide that the superclass is Object without any other notice and losing the name of the original superclass. We are working on a new mechanism to support a unique way to handle such case. To be presented at IWST/ESUG - Class definition parser: Class definition is parser in different place of the system and in addition the output is the direct creation of a class object instead of an object representing the class definition that can be further manipulated. We are working on class definition parser. It produces a separate AST. It will help the building of tools. - Better update infrastructure. Pablo Tesone has been working on a better update mechanism, better modular class builder. - Ring2: Ring is a metamodel to manipulate code that is not actively running in the current system. It is useful to perform analysis (such as browsing, navigating off-line or remote definitions) before actually loading the code. Ring got redesigned by Pavel Krivanek to support the bootstrap of Pharo 60. The results is Ring2. - Opal is the Pharo Compiler. It should be enhanced to support handle better the warning and a general enhancement pass is needed. ### System enhancements - Commandline enhancements. RMOD is currently improving the command-line infrastructure and making sure that the system can work in read-only mode. - Cleaning streams: the idea is to make sure that the system does not use the old streams. The idea is to start using the fileStreams and make sure that the Stream package can be substituated in the future by other streams with the same API. Therefore hardcoded class names should be substituted with factory (readSream writeStream). From that perspective we do not think that it is wise to introduce Xtreams. We should analyse the API of the current implementation. - New theme from the beginning. It is really important that each version of Pharo is identified open the default opening. Pharo should come up with two default themes: one light and one dark. - Better themes/palettes support - Better and nicer Tabs. Tabs design should be revisited. - OSWindow world rendering: the effort to remove the logic of the windowing from the VM should be continued. OSWindow should be used instead. - Freetype: The current freetype plugin is the source of many bugs and problem. Thibault Raffaillac used FFI to do direct call to openGL (@@to verify@@). - Refactoring 2: Refactoring 2 is an improved version of the refactorings developed by Gustavos Santos and they should used to replace the existing one. ## VM - 64-bits by default: Mac and Linux 64 bits VMs have been working for over a year and since June 2017 a Windows 64 bits VM has been working. The plan is to stabilize each part of Pharo that is not working in 64 bits as well as in 32 bits and make the 64 bits Pharo images and VM the default download for Pharo. - Headless VM: Ronie Salgado has built a headless VM on his VM branch, we need to merge and check everything works fine. With the headless VM, Pharo can be run in true headless mode (not with a hidden window) and it is required for future VM features (embeddable VM, ...). - Embeddable VM: The VM should be able to be embedded in external applications, the application accessing objects through well-defined APIs. - Idle VM: The goal is to avoid the active waiting loop consuming several per cent of cpu time when the Pharo image is not doing anything. - Android VM: integration of the Android VM build and tests in the integration setup. - Threaded FFI: all FFI calls should be non-blocking (reviving prototype of Eliot) - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers - Better support for large heaps (GC tuning API, incremental GC). Pharo 64 bit is now able to manage large heap. However better performance can be proposed by offering better settings for the different GC zone. - Integrate various fixes to support better high resolution display. In Retina mac display Pharo looks blurry. The fix to support Retina should be integrated in the VM. ### Sista-related Sista is an optimizing JIT for Pharo. It is the result of multiple years of developement by Clement Bera and Eliot Miranda. An optimizing JIT is manipulating code (folding constant, rewriting it) before compiling it to assembly code. - New Block Closure implementation by default: Allows one to implement in the Opal compiler the copying and clean blocks optimisations, reduce massively the complexity of some parts of the JIT (debugging API to map machine code pc to bytecode pc for example) and is required for the Sista support. Some work remains in debugging/IDE support. - New Bytecode set in production: Eases bytecode decoding (simplifying the code base of the bytecode to source code decompiler, the debugger, the JIT, etc.), lifts compiled code limitations (size of jumps, etc.) and is required for the Sista support. Some work remains in debugging support. - Read-only objects: They work in Pharo 6, but the in-image support needs to be improved (fall-back code of primitives, etc.) and some polishing needs to be done (primitive error code not always correct, etc.). Used for the support of different project, including GBS. - Literals immutability (based on read-only objects). In particular an analysis of the image usage of literal (string mutation) is needed. - Sista preview: A first version of Sista will be integrated, yielding 1.5x performance boost on most applications, but will be optional and will require specific constraints (not toying too much with literal mutability, partial IDE support, etc.). ### Already done - [DONE] Remove Shoreline reporter. - [DONE] Kernel should not depend on Traits: This will speed up the bootstrap and support the modular introduction of alternate traits implementation currently designed by Pablo Tesone and tested in the new generation of Moose.

July 6, 2017

Re: [Pharo-dev] tar sec files for windows Pharo VM build

by Eliot Miranda

Never mind. I've made sense of it. Forgive the noise. On Wed, Jul 5, 2017 at 8:40 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi All, > > if I'm reading the build.winMNxOP/third-party/Makefile.libXXX files > correctly there should be a libXXX.spec.win file for each library in > build.winMNxOP/third-party. There are no such files in the Cog or master > branches. Can anybody point me to these files? > > _,,,^..^,,,_ > best, Eliot > -- _,,,^..^,,,_ best, Eliot

July 6, 2017

tar sec files for windows Pharo VM build

by Eliot Miranda

Hi All, if I'm reading the build.winMNxOP/third-party/Makefile.libXXX files correctly there should be a libXXX.spec.win file for each library in build.winMNxOP/third-party. There are no such files in the Cog or master branches. Can anybody point me to these files? _,,,^..^,,,_ best, Eliot

July 6, 2017

Re: [Pharo-dev] Recommended production VM on latest macOS Sierra?

by Ben Coman

On Wed, Jul 5, 2017 at 8:39 PM, Tim Mackinnon <tim(a)testit.works> wrote: > So does that mean we should use the Pharo5 vm for production then? It > sounds like 6 is still a bit bleeding edge There is probably no stability advantage in Pharo-5-32bit over Pharo-6-32bit, and "6" will be supported longer. There was no released Pharo-5-64bit for comparison IIUC Pharo-6-64bit is "stable" but I guess as its adopted into more varied use and environments, some issues may arise. So use 6-64bit if you need its image size capability, otherwise use 6-32bit (but still develop using 6-64bit to find those outlying bugs more quickly and cover 6-32bit with CI testing) cheers -ben > and for production applications we should wait for 6.1? > What you want is for any bugs you find to be integrated into 6.1, so you need to use 6.0 to find them ;) cheers -ben > At the moment it seems a little chaotic - we all like the new features, > and 6 is definitely a breath of fresh air - however if you want to deploy > something commercially then you need stability. > > > Tim > > > On 3 Jul 2017, at 19:28, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > > > Hi, > > > >> On 3 Jul 2017, at 20:08, Bernhard Pieber <bernhard(a)pieber.com> wrote: > >> > >> Hi, > >> > >> What VM would you recommend for a production app on macOS Sierra? > >> - Pharo 6, right? > > yes > > > >> - I assume 32-bit because 64-bit is still experimental, right? > > not particularly, you can download it and use it. > > some tests fails but people is already using it (iceberg will not work, > but AFAIK, is the only thing that does not works). > > > >> - The one I get with âcurl get.pharo.org | bashâ has a reproducible > crash when typing (https://github.com/OpenSmalltalk/opensmalltalk- > vm/issues/141). So it canât be the recommended one, right? > > nevertheless is the current recommended one, and that bug is annoying us > all :) > > > > but we hope to be able to fix it eventually, of course :) > > > > cheers! > > Esteban > > > >> > >> I must admit I am quite confused. > >> > >> Cheers, > >> Bernhard > > > > > > >

July 5, 2017

Re: [Pharo-dev] Recommended production VM on latest macOS Sierra?

by Tim Mackinnon

Ahh - thats more reassuring. There is quite a lot going on at the moment and so it gets a bit overwhelming. Thanks for all the hard work youâve put into this by the way - its really starting to open up the doors we all needed. Tim > On 5 Jul 2017, at 15:02, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > pharo 6 is very stable and should be used in production. > every version has bugs, that will not change. Yes, that particular bug can be annoying but it does not happens always (for example, Iâm unable to reproduce it, which is in fact part of the reason why I cannot fix it). > > cheers, > Esteban > >> On 5 Jul 2017, at 14:39, Tim Mackinnon <tim(a)testit.works> wrote: >> >> So does that mean we should use the Pharo5 vm for production then? It sounds like 6 is still a bit bleeding edge and for production applications we should wait for 6.1? >> >> At the moment it seems a little chaotic - we all like the new features, and 6 is definitely a breath of fresh air - however if you want to deploy something commercially then you need stability. >> >> >> Tim >> >>> On 3 Jul 2017, at 19:28, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >>> >>> Hi, >>> >>>> On 3 Jul 2017, at 20:08, Bernhard Pieber <bernhard(a)pieber.com> wrote: >>>> >>>> Hi, >>>> >>>> What VM would you recommend for a production app on macOS Sierra? >>>> - Pharo 6, right? >>> yes >>> >>>> - I assume 32-bit because 64-bit is still experimental, right? >>> not particularly, you can download it and use it. >>> some tests fails but people is already using it (iceberg will not work, but AFAIK, is the only thing that does not works). >>> >>>> - The one I get with âcurl get.pharo.org | bashâ has a reproducible crash when typing (https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/141) So it canât be the recommended one, right? >>> nevertheless is the current recommended one, and that bug is annoying us all :) >>> >>> but we hope to be able to fix it eventually, of course :) >>> >>> cheers! >>> Esteban >>> >>>> >>>> I must admit I am quite confused. >>>> >>>> Cheers, >>>> Bernhard >>> >>> >> >> > >

July 5, 2017

Re: [Pharo-dev] Recommended production VM on latest macOS Sierra?

by Esteban Lorenzano

pharo 6 is very stable and should be used in production. every version has bugs, that will not change. Yes, that particular bug can be annoying but it does not happens always (for example, Iâm unable to reproduce it, which is in fact part of the reason why I cannot fix it). cheers, Esteban > On 5 Jul 2017, at 14:39, Tim Mackinnon <tim(a)testit.works> wrote: > > So does that mean we should use the Pharo5 vm for production then? It sounds like 6 is still a bit bleeding edge and for production applications we should wait for 6.1? > > At the moment it seems a little chaotic - we all like the new features, and 6 is definitely a breath of fresh air - however if you want to deploy something commercially then you need stability. > > > Tim > >> On 3 Jul 2017, at 19:28, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >> >> Hi, >> >>> On 3 Jul 2017, at 20:08, Bernhard Pieber <bernhard(a)pieber.com> wrote: >>> >>> Hi, >>> >>> What VM would you recommend for a production app on macOS Sierra? >>> - Pharo 6, right? >> yes >> >>> - I assume 32-bit because 64-bit is still experimental, right? >> not particularly, you can download it and use it. >> some tests fails but people is already using it (iceberg will not work, but AFAIK, is the only thing that does not works). >> >>> - The one I get with âcurl get.pharo.org | bashâ has a reproducible crash when typing (https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/141) So it canât be the recommended one, right? >> nevertheless is the current recommended one, and that bug is annoying us all :) >> >> but we hope to be able to fix it eventually, of course :) >> >> cheers! >> Esteban >> >>> >>> I must admit I am quite confused. >>> >>> Cheers, >>> Bernhard >> >> > >

July 5, 2017

SmartTest

by Benoit Verhaeghe

Hi everyone, Iâm working on a new plugin for Nautilus/Calypso that will help us with the test. The goal is to provide us the selection of tests we should run after a modification. It would be great if you accept to try it or want to use it. Currently, by installing the plugin, when you select a method in Calypso or Nautilus, you will see a new critique that will offer you to run the test relative to the method youâve selected (or class ^^ ) Another goal is to calculate the efficient of this new tool and how we will use it. Itâs why Iâve also developed a plugin that will record how we will use the plugin. If you accept to help me. Please tell me ( so i will be able to estimate how many data I will obtain ). Now the commands. If youâd like to help me. The command is simple : Metacello new smalltalkhubUser: 'badetitou' project: 'TestsUsageAnalyser-CoraExtends'; configuration: 'TestsUsageAnalyser_CORAExtends'; version: #stable; get; load. If youâd like to use the plugin or just want to try it. Itâs also simple Metacello new smalltalkhubUser: 'badetitou' project: 'CORA'; configuration: 'CORA'; version: #stable; get; load. For the version with Calypso ( because itâs cool ). Metacello new smalltalkhubUser: 'badetitou' project: 'CORA'; configuration: 'CORA_Calypso'; version: #stable; get; load. So, if you want to help me and use calypso you should run the first and the third command. Options for the plugin are available in the settings of Pharo in the group : âTestRegressionâ. So you can extend the plugin with your own logic of testing strategy. To disable the spy (first package) in the option please uncheck âTest Usage Analyserâ Iâm going to write a blog post that will explain all the plugin and how to use it. If you find bugs ( But there are no bugs ;) ) please tell me and i will fix it as soon as possible. If youâd like that I add a feature, tell me too. Thanks a lot for your help. BenoÃ®t Verhaeghe

July 5, 2017

Re: [Pharo-dev] Recommended production VM on latest macOS Sierra?

by Tim Mackinnon

So does that mean we should use the Pharo5 vm for production then? It sounds like 6 is still a bit bleeding edge and for production applications we should wait for 6.1? At the moment it seems a little chaotic - we all like the new features, and 6 is definitely a breath of fresh air - however if you want to deploy something commercially then you need stability. Tim > On 3 Jul 2017, at 19:28, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > Hi, > >> On 3 Jul 2017, at 20:08, Bernhard Pieber <bernhard(a)pieber.com> wrote: >> >> Hi, >> >> What VM would you recommend for a production app on macOS Sierra? >> - Pharo 6, right? > yes > >> - I assume 32-bit because 64-bit is still experimental, right? > not particularly, you can download it and use it. > some tests fails but people is already using it (iceberg will not work, but AFAIK, is the only thing that does not works). > >> - The one I get with âcurl get.pharo.org | bashâ has a reproducible crash when typing (https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/141) So it canât be the recommended one, right? > nevertheless is the current recommended one, and that bug is annoying us all :) > > but we hope to be able to fix it eventually, of course :) > > cheers! > Esteban > >> >> I must admit I am quite confused. >> >> Cheers, >> Bernhard > >

July 5, 2017

Ideas wanted: "Patterns to sustain feedback loops"

by Marcus Denker

Hi, I want to do a third talk at ESUG as the âpart 3â of the feedback loop series.. The first parts where: 1. Nomads do not build Cathedrals: https://www.slideshare.net/MarcusDenker/2014-esugcathedral 2. Perfection and Feedback Loops: Why worse is better. https://www.slideshare.net/MarcusDenker/perfection-feedback-loops-or-why-wo… The idea for part 3 is to do something along the lines of âpatterns to enable feedback loopsâ, a more practical view on what one should be doing (or not be doing) to enable and sustain a project as a feedback loop. - both things we do and those we should - things that we do wrong / anti patterns - technical as well as social (communityâ¦) - examples you have seen in the real world The idea is not to follow a formal patterns language, it will be informalâ¦ so if you have any ideas, please send a private mail to me. Marcus

July 5, 2017

Re: [Pharo-dev] Pull requests reviews

by Esteban Lorenzano

donât need it. Iâm working on tool for that :) > On 5 Jul 2017, at 10:02, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > If you don't mind, I'll transform your script into a class and commit it somewhere :) > > On Wed, Jul 5, 2017 at 8:30 AM, Pavel Krivanek <pavel.krivanek(a)gmail.com <mailto:pavel.krivanek@gmail.com>> wrote: > So loading of the PR packages is very simple. Do at the end: > > diff := IceDiff from: headCommit to: bootstrapCommit. > diff changedPackages do: #reload. > > > > 2017-07-04 17:33 GMT+02:00 Pavel Krivanek <pavel.krivanek(a)gmail.com <mailto:pavel.krivanek@gmail.com>>: > I prepared a script that should help you with the reviews of the pull requests on Pharo 7. We will later convert it into a more fancy tool. It does next steps: > > - sets the basic information: pull request number, path to your pharo repository clone, name of your fork. > - registers the repository into Iceberg and sets pull and push target remotes > - switches branch to a particular commit from which the Pharo image was bootstrapped > - registers the repository into into Monticello packages to be able to do correct diffs > - gets basic information about the pull request from GitHub (original repository, branch name) > - registers the PR original repository into remotes if needed and fetches information from it > - creates a new local branch to merge the PR > - merges the PR branch > - displays a simple tool that shows differences in done in this merged branch > > -------- > > pullRequest := 73. > target := '/path/pharo' asFileReference. > myForkName := 'myFork'. > > repository := IceRepositoryCreator new location: target; subdirectory:'src'; createRepository. > repository register. > fork := repository remotes detect: [ :remote | remote remoteName = myForkName ]. > repository pushRemote: fork. > repository pullRemote: repository origin. > repository checkoutBranch: (SystemVersion current commitHash). > > fileTreeRepository := (MCFileTreeRepository new directory: target / #src; yourself). > repositoryGroup := MCRepositoryGroup withRepositories: { fileTreeRepository. MCCacheRepository uniqueInstance. }. > MCWorkingCopy allManagers > select: [ :wc | (wc repositoryGroup repositories reject: [ :repo | repo isCache ]) isEmpty ] > thenDo: [ :wc | wc repositoryGroup: repositoryGroup ]. > > stonString := (ZnEasy get: 'https://api.github.com/repos/pharo-project/pharo/pulls/ <https://api.github.com/repos/pharo-project/pharo/pulls/>', pullRequest asString) contents. > head := (STONJSON fromString: stonString) at: 'head'. > sshUrl := (head at: #repo) at: 'ssh_url'. > branchName := head at: #ref. > user := (sshUrl withoutPrefix: 'git(a)github.com:') withoutSuffix: '/pharo.git'. > > fork := repository remotes detect: [ :remote | remote remoteName = user ] ifNone: [ > | newFork | > newFork := (IceRemote name: user url: ('git@github.com:{1}/pharo.git' format: {user})). > repository addRemote: newFork. > newFork ]. > > repository fetchFrom: fork. > > prMergedBranchName := 'pr', pullRequest asString. > repository createBranch: prMergedBranchName. > repository checkoutBranch: prMergedBranchName. > > commit := repository revparse: user, '/', branchName. > bootstrapCommit := repository revparse: (SystemVersion current commitHash). > [ repository backend merge: commit id ] > on: IceMergeAborted > do: [ :error | repository mergeConflictsWith: commit ] . > headCommit := repository revparse: 'HEAD'. > > browser := GLMTabulator new. > browser row: [:row | row column: #commits span: 2; column: #changes span: 3]; row: #diff. > browser transmit to: #commits. > browser transmit to: #changes; from: #commits; andShow: [ :a :commitInfo | > (IceDiffChangeTreeBuilder new entity: commitInfo; diff: (IceDiff from: commitInfo to: bootstrapCommit); buildOn: a) title: 'Changes'. ]. > browser transmit from: #commits; from: #changes; to: #diff; andShow: [ :a | > a diff title: 'Left: working copy / Right: incoming updates'; display: [ :commitInfo :change | > { change theirVersion ifNil: ''. change myVersion ifNil: ''. }]]. > browser openOn: {headCommit}. > > -------- > > The merge operation only changes the Git working copy, no code is loaded into the image. If you want to test the PR code, currently you need to open Iceberg and reload all packages in the Pharo repository (Packages tab, Reload all) > > Expect troubles :-) > > Cheers, > -- Pavel > > > > > > > -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> > > > Web: http://guillep.github.io <http://guillep.github.io/> > Phone: +33 06 52 70 66 13

July 5, 2017