Some more effort to make Slang and VMMaker work on Pharo [for review]
Hi all, Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is: - we can generate, compile and run a stack vm - we can generate, compile and run a cog vm - for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think) To do this work - I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on... - Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so on⦠Cheers, Guille
Inicio del mensaje reenviado:
De: Guillermo Polito <guillermopolito@gmail.com> Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review] Fecha: 9 de mayo de 2019, 17:48:03 CEST Para: Open Smalltalk Virtual Machine Development Discussion <vm-dev@lists.squeakfoundation.org>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote: Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org <http://source.squeak.org/> and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
Thanks Guile, This is super important to ease contributions from Pharo teams. Le mar. 14 mai 2019 à 16:30, Guillermo Polito <guillermopolito@gmail.com> a écrit :
Hi all,
Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is:
- we can generate, compile and run a stack vm - we can generate, compile and run a cog vm
- for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think)
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on... - Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
Cheers, Guille
Inicio del mensaje reenviado:
*De: *Guillermo Polito <guillermopolito@gmail.com> *Asunto: **Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]* *Fecha: *9 de mayo de 2019, 17:48:03 CEST *Para: *Open Smalltalk Virtual Machine Development Discussion < vm-dev@lists.squeakfoundation.org>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
[image: image.png]
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
Ah yes, and Iâve done this so far on Pharo6.1, platform where Eliot was working on initially. I donât think there is much to do to move it to Pharo7, but still Iâve not explored it.
El 14 may 2019, a las 16:29, Guillermo Polito <guillermopolito@gmail.com> escribió:
Hi all,
Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is:
- we can generate, compile and run a stack vm - we can generate, compile and run a cog vm
- for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think)
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on... - Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
Cheers, Guille
Inicio del mensaje reenviado:
De: Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review] Fecha: 9 de mayo de 2019, 17:48:03 CEST Para: Open Smalltalk Virtual Machine Development Discussion <vm-dev@lists.squeakfoundation.org <mailto:vm-dev@lists.squeakfoundation.org>>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote: Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org <http://source.squeak.org/> and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
<image.png>
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
This is really great, thx Guille. I'd love to use Pharo environment when I play with the VM and look forward to trying it. Is it feasible to add VMMaker to the Catalog and make it a one click install? Thanks also Elliot for the initial work. cheers -ben On Tue, 14 May 2019 at 23:05, Guillermo Polito <guillermopolito@gmail.com> wrote:
Ah yes, and Iâve done this so far on Pharo6.1, platform where Eliot was working on initially. I donât think there is much to do to move it to Pharo7, but still Iâve not explored it.
El 14 may 2019, a las 16:29, Guillermo Polito <guillermopolito@gmail.com> escribió:
Hi all,
Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is:
- we can generate, compile and run a stack vm - we can generate, compile and run a cog vm
- for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think)
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on... - Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
Cheers, Guille
Inicio del mensaje reenviado:
*De: *Guillermo Polito <guillermopolito@gmail.com> *Asunto: **Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]* *Fecha: *9 de mayo de 2019, 17:48:03 CEST *Para: *Open Smalltalk Virtual Machine Development Discussion < vm-dev@lists.squeakfoundation.org>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
<image.png>
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
This is super nice and this was a couple of days well invested. I thought that it would be a lot more challenging so you prove it by doing it fast and this is super cool. In addition this is excellent that you added tests while doing it. I really want to have a tutorial on extending the VM with Pharo. Stef
Hi all,
Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is:
- we can generate, compile and run a stack vm - we can generate, compile and run a cog vm
- for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think)
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on... - Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
Cheers, Guille
Inicio del mensaje reenviado:
De: Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review] Fecha: 9 de mayo de 2019, 17:48:03 CEST Para: Open Smalltalk Virtual Machine Development Discussion <vm-dev@lists.squeakfoundation.org <mailto:vm-dev@lists.squeakfoundation.org>>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote: Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org <http://source.squeak.org/> and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
<image.png>
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
Hi Guille, On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to keep everyone informed, last week Iâve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot. The summary of what I did is:
- we can generate, compile and run a stack vm - we can generate, compile and run a cog vm
Great news!
- for this, Iâve done a couple of patches to VMMaker - the ones specific to Pharo has been pushed - the ones that touch cross dialect code are still in review (I think)
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.
- Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.
Cheers, Guille
Inicio del mensaje reenviado:
*De: *Guillermo Polito <guillermopolito@gmail.com> *Asunto: **Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]* *Fecha: *9 de mayo de 2019, 17:48:03 CEST *Para: *Open Smalltalk Virtual Machine Development Discussion < vm-dev@lists.squeakfoundation.org>
I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST. I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable... Maybe we can do an iteration afterwards.
Cheers, Guille
On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
On Wed, May 8, 2019 at 2:17 AM David T. Lewis <lewis@mail.msen.com> wrote:
On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:
In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of asString. asString is a horrible hack. printString says what we mean.
Sure. I've submitted another version to the inbox:
Name: VMMaker.oscog-GAP.2534 Author: GAP Time: 9 May 2019, 5:12:22.055089 pm UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1 Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation using #printString. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
==== Patch 1 ====
Name: VMMaker.oscog-GAP.2533 Author: GAP Time: 7 May 2019, 10:57:16.034354 am UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee Ancestors: VMMaker.oscog-eem.2532
Convert numeric constants to strings for concatenation. Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.
I trust you on these. I'm not really in a position to test this out right now. I suggest you push the VMMakerCompatibilityForPharo6 packages to VMMaker as soon as you'd like. You can write VMMaker right?
+1
Guille, if you want commit access to the repo, you can make an account on source.squeak.org and I'll add you the the VMMaker project. Or if it's easier, just say which packages you are ready to move and either I or Eliot (or any of the others with commit access) will move it to VMMaker for you.
I note that VMMaker.oscog-GAP.2533 package is part of the main VMMaker.oscog package. If you need to have that tested on Squeak I can probably do that for you within the next couple of days if no one else gets to it first.
Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo. So, I'd say I've tried it myself already.
For the Pharo specific package, I've already uploaded it to the VMMaker repository. I had already rights to do it :).
Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11 Author: GuillermoPolito Time: 9 May 2019, 5:15:50.00473 pm UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10
With these changes I can already compile and run Cog and almost all tests pass :)
[image: image.png]
I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs. Would you like that I push them inside the VMMaker package or make a separate package?
Thanks! Guille
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
-- _,,,^..^,,,_ best, Eliot
Hi Eliot,
El 15 may 2019, a las 3:20, Eliot Miranda <eliot.miranda@gmail.com> escribió:
Hi Guille,
On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.
From what I have now, I can see three main kind of differences (I actually compared with source code I generated myself from squeak and the existing sources in the git repo). I did not work on them because the VM seemed to work, so I left them for later:
1) some comments are placed in different places in the output source code. This is entirely cosmetic, but it brings unnecessary noise to C source diffs. 2) labels in the gnuâed code are numbered differently. I did not spot any âbugâ in that yet, I did not dig enough into the gnuification to see why the difference, and Iâm not entirely sure that this difference is caused by that either :)⦠Anyhow, this is mostly annoying because it also brings lots of noise to diffs. 3) some for loops inline the condition while they probably shouldn't. For example: 0 to: (self numSlotsOf: array1) - 1 do: [ ⦠] gets in my output translated to for (i = 0; i < (numSlotsOf(array1)); i += 1) { ... while in the version generated from squeak for (i = 0, iLimiT = ((numSlotsOf(array1)) - 1); i <= iLimiT; i += 1) { ⦠This seems the most âdangerous" difference I have so far specially if the limit condition has some side effect. We will probably agree in that this third point should be fixed ^^.
- Iâve written several unit tests to ensure that future migrations are easier to do - Iâve introduced several compatibility classes/methods related to PackageInfo, Time and so onâ¦
This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.
Maybe in the next weeks I can try to massage the tests to make them less dependent on the source (wether squeak's or pharoâs) AST. The thing is that Pharoâs AST is closer to the source code, while Squeakâs has already some pre-cooked code transformations (like #ifNil: & co), and my tests assume a specific AST structure to traverse the tree and compare⦠Cheers, Guille
Guille, This is excellent work. Thank you also for your careful explanations. Dave On Wed, May 15, 2019 at 10:33:38AM +0200, Guillermo Polito wrote:
Hi Eliot,
El 15 may 2019, a las 3:20, Eliot Miranda <eliot.miranda@gmail.com> escribi??:
Hi Guille,
On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.
From what I have now, I can see three main kind of differences (I actually compared with source code I generated myself from squeak and the existing sources in the git repo). I did not work on them because the VM seemed to work, so I left them for later:
1) some comments are placed in different places in the output source code. This is entirely cosmetic, but it brings unnecessary noise to C source diffs.
2) labels in the gnu???ed code are numbered differently. I did not spot any ???bug??? in that yet, I did not dig enough into the gnuification to see why the difference, and I???m not entirely sure that this difference is caused by that either :)??? Anyhow, this is mostly annoying because it also brings lots of noise to diffs.
3) some for loops inline the condition while they probably shouldn't. For example:
0 to: (self numSlotsOf: array1) - 1 do: [ ??? ]
gets in my output translated to
for (i = 0; i < (numSlotsOf(array1)); i += 1) { ...
while in the version generated from squeak
for (i = 0, iLimiT = ((numSlotsOf(array1)) - 1); i <= iLimiT; i += 1) { ???
This seems the most ???dangerous" difference I have so far specially if the limit condition has some side effect. We will probably agree in that this third point should be fixed ^^.
- I???ve written several unit tests to ensure that future migrations are easier to do - I???ve introduced several compatibility classes/methods related to PackageInfo, Time and so on???
This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.
Maybe in the next weeks I can try to massage the tests to make them less dependent on the source (wether squeak's or pharo???s) AST. The thing is that Pharo???s AST is closer to the source code, while Squeak???s has already some pre-cooked code transformations (like #ifNil: & co), and my tests assume a specific AST structure to traverse the tree and compare???
Cheers, Guille
On 14/05/19 7:59 PM, Guillermo Polito wrote:
To do this work
- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Guillermo, This is a superb effort! Thanks. Are you planning to write a paper on this part of your work? A blog? It can open new vistas for research students in Pharo. Regards .. Subbu
Hi Subbu
To do this work - I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Guillermo,
This is a superb effort! Thanks. Are you planning to write a paper on this part of your work? A blog? It can open new vistas for research students in Pharo.
We will see if we can produce a tutorial in how to extend the VM. Igor did it in 2011 and we will redo it. Life is a cycle. Then we would love to see if we can have a VMMaker repackaged (it should not be difficult) with less code so that we can do lectures on the Pharo VM and exercises for students with it. It is not really pedagogical to tell them to jump over many not used classes. And more important since we want to attract smart guys they will not take us seriously if we present a system that is monolithic - we got several times students asking why VMMaker could not be cut into different packages and well they are right. What I was thinking is that we should have soon a travis to build it automatically with Pharo. If you are interested to help please join the effort. What guille showed us is that it was not difficult to do it in fact. Stef
Regards .. Subbu
Hi Subbu, I do not believe this deserves a paper :P. At most a blogpost. What I can do quickly here is to draft the âmethodologyâ I followed by instinct, but that can be reproduced by anyone. 1) generate sources from pharo 2) try to compile (and run) 3) in case of failure, make a diff with the latest version of the same file in the git repository of opensmalltalk vm 4) spot an interesting difference (most common cases were related to loops and conditions, missing or extra inlinings) => get ONE method with the problem 4.1) Check 1: verify that the generating that method from squeak does **not** have the problem 4.2) Check 2: verify that the generating that method from pharo **does have** the problem For these checks I used some of the already given VMMaker scripts 5) generate a small artificial but representative case for testing. For example: methodWithIfNil self something ifNil: [ 1 ] ifNotNil: [ 2 ] Then I have two level of tests: AST-to-AST transformation, and generation tests. testIfNilIfNotNilBecomesIfTrueIfFalse | translation thisAST | thisAST := (self class >> #methodWithIfNil) ast. translation := thisAST asTranslationMethodOfClass: TMethod. self assert: translation statements first selector equals: #ifTrue:ifFalse: testSimpleIfNilAssignment | translation thisAST codeGenerator result | thisAST := (self class >> #methodWithIfNil) ast. translation := thisAST asTranslationMethodOfClass: TMethod. codeGenerator := CCodeGeneratorGlobalStructure new. codeGenerator generateDeadCode: false. codeGenerator addMethod: translation. codeGenerator doInlining: true. result := String streamContents: [ :stream | translation parseTree statements first emitCCodeOn: stream level: 0 generator: codeGenerator. ]. self assert: result equals: 'if ((something()) == null) { } else { }â 6) Fix it * Most of the âbugsâ were related to making the different #asTranslatorNodeIn: methods in the RB AST compliant to what it was doing in squeak. * Sometimes I ended up doing parallel stepping in Pharo and Squeak to spot where the translation diverged * Something important that Iâve learnt in the process is that Squeakâs AST contain already some transformations/expansions like what you see/write: self x ifNil: [ ⦠] what you actually have: self x == nil ifTrue: [ ⦠] BUT, the most confusing point is that even if the AST internally is structured as in the version in the right, the printString of the AST in the inspector showed the left version. Nothing new under the sun ^^. Guille
El 15 may 2019, a las 9:34, ducasse <stepharo@netcourrier.com> escribió:
Hi Subbu
To do this work - I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
Guillermo,
This is a superb effort! Thanks. Are you planning to write a paper on this part of your work? A blog? It can open new vistas for research students in Pharo.
We will see if we can produce a tutorial in how to extend the VM. Igor did it in 2011 and we will redo it. Life is a cycle.
Then we would love to see if we can have a VMMaker repackaged (it should not be difficult) with less code so that we can do lectures on the Pharo VM and exercises for students with it. It is not really pedagogical to tell them to jump over many not used classes. And more important since we want to attract smart guys they will not take us seriously if we present a system that is monolithic - we got several times students asking why VMMaker could not be cut into different packages and well they are right.
What I was thinking is that we should have soon a travis to build it automatically with Pharo. If you are interested to help please join the effort. What guille showed us is that it was not difficult to do it in fact.
Stef
Regards .. Subbu
participants (7)
-
Ben Coman -
David T. Lewis -
ducasse -
Eliot Miranda -
Guillermo Polito -
K K Subbu -
Nicolas Cellier