Re: [Pharo-dev] [Vm-dev] Fwd: Nix package for Pharo flavor of opensmalltalk-vm
On Thu, Apr 20, 2017 at 10:19 PM, Ben Coman <btc@openinworld.com> wrote:
Moved thread back to pharo-dev for image-side nix packaging discussion.
Whoops. Try again, back to pharo-dev.
On Thu, Apr 20, 2017 at 3:59 PM, Luke Gorrie <luke@snabb.co> wrote:
On 20 April 2017 at 06:03, Ben Coman <btc@openinworld.com> wrote:
On Thu, Apr 20, 2017 at 12:21 AM, Luke Gorrie <luke@snabb.co> wrote:
Then you can start one of those images with these shell commands:
pharo-6.0 pharo64-6.0 pharo-5.0 moose-6.1 pharo-launcher-2017.02.28
Since Pharo 6 is not yet released, what are the implications of naming those pharo-6.dev & pharo64-6.dev [minor edit]
On reflection I will split up my packaging work into "vm" and "image" components. I will send the vm upstream ASAP. I will leave the images on a development branch, perhaps indefinitely, because I am not sure that I am on the right track there at all.
That sounds like a good approach.
So, here are more thoughts on image packaging, but with the disclaimer that I will probably not actively pursue this topic further at the moment:
I assert that Pharo 5 and Pharo 6 are both released,
The "release" event will create the PharoV60.sources file, null out the .changes file (it gets incorporated into PharoV60.sources) and Pharo 7 development on will commence. None of those things has happened yet. Last I read, this was due in May, similar timeframe to Pharo 5 release last year.
and are both fast-moving targets.
The attached chart shows the dev-image build rate. There is a clear change in Pharo 5 build rate after it was released on 12-May. What happens is that during Pharo-6-development, a critical bug was fixed and deemed worthy to backport to Pharo 5. That doesn't put released-Pharo-5 and in-development-Pharo-6 on equal footing.
Perhaps for packaging for distributions we need a modified process for post-release-updates, such that the released image remains constant and upon startup notifies that there are updates available to install.
On http://pharo.org/download there is a prominent link to Pharo 6 as the "development version" ready for download. Anecdotally the Pharo community that I interact with (mailing list, twitter, slack chat, etc) seem more focused on Pharo 6 than on Pharo 5.
The greater focus on Pharo 6 is because it is "in development" and you are talking to pharo-developers.
Occasionally people express surprise when I tell them I am "still" using Pharo 5.
On the one hand, that might be considered overzealous recommending alpha/beta software for your primary platform. On the other hand, in practice many find the bleeding edge consistently very stable. (opinion depends on the exact critical nature the application)
The thing is, if you hit a bug in the in-development version, it can get fixed quite quickly. More Pharo-developers work on the HEAD, so its self interest to get things working smoothly. Policy is that *all* fixes must *first* be done in the in-development-version. Only critical fixes get backported to the released-version. So if you hit a bug in a released-version, you might not see the fix before the next release.
So there is a lot of benefit following the tip of development.
On http://files.pharo.org/image/50/ there is a new release of Pharo 5 every few weeks. This means that each time I have downloaded "Pharo 5" in the past I have probably gotten a different image and a different VM. So from my end-user perspective I have no idea what software I am installing regardless of whether I choose Pharo 5 or Pharo 6.
I consider this moving target for a released version (i.e. Pharo 5) a problem. Personally I'd like to see post-release versionning like 5.0.x, but other seem to disagree, or maybe its just resourcing.
Pharo 6 is "in-development" so different rules apply. Personally I never use the "latest" download. I manually choose the latest build using PharoLauncher (couldn't live without it)
So as a nix user it would make sense for me to be able to download both Pharo 5 and Pharo 6 and for the nix package definitions to determine exactly which software I am running (as opposed to getting an invisible update from Pharo upstream every few days/weeks potentially leading to "but it worked fine on my other machine..." sorts of problems that are exactly what can nix protect me from.)
I agree with you wrt to a released-Pharo-5, but not in-development-Pharo-6.
Packing the other images is maybe also wrong. How would you deal with user state being stored in the image when a new build download in available?
The ad-hoc idea that I ran with is this:
Running a script like pharo-5.0 (or moose-6.1, etc) will make a temporary copy of the image in the current directory. The changes file will be read-write (requirement) but the image file will be read-only. The model is "you saved it, you own it" i.e. if you want the image to survive after exit you need to "Save image as..." and once you do that then you are responsible for it.
I hardly ever use "Save image as...", just "Save" and I think that might confuse existing users if they *need* to do it. Probably workable would be having the "Save image as..." dialog appear the first that "Save" is chosen.
Code: script to run an image[1] and script to start the right vm[2].
[1] https://github.com/lukego/nixpkgs/blob/pharo6/pkgs/development/pharo/images/... [2] https://github.com/lukego/nixpkgs/blob/pharo6/pkgs/development/pharo/wrapper...
I am not sure why "Save image as..." is misbehaving for me. I have tested every combination of read/write permissions on the working directory, the image, the changes file, etc, but it was always the same. I am sure that I can work this out via the Smalltalk debugger -- but at the moment it seems to make more sense to focus on the VM than on the images. (I want to wrap this up.)
The downside to skipping the images is that I don't maintain feature parity with Damien's original nix packaging. He packaged the launcher image and then that was the gateway to everything else. This solution doesn't feel very solid to me, since now in 2017 I don't think the launcher is accounting for all the different VM requirements properly, but I am not willing to pick up maintenance of that image so that is life.
Skipping Images is different to skipping Launcher. The Launcher provides a *static* vm+image that is used to manage Images to solve "movign target" problem you are hitting. Launcher could perhaps do with some further extension to distinguish 64-bit Images, since that requirement has only arisen in the past year, and also perhaps be able to manage multiple VMs, instead of just the pre-Spur & Spur VMs configured in its settings. But otherwise Launcher works fairly well.
If feasible, I'd suggest that you separately package the VMs, and then have a package for PharoLauncher that depends on that package.
cheers -ben
Cheers, -Luke
On Thu, Apr 20, 2017 at 6:43 PM, Luke Gorrie <luke@snabb.co> wrote:
On 20 April 2017 at 09:59, Luke Gorrie <luke@snabb.co> wrote:
On reflection I will split up my packaging work into "vm" and "image" components. I will send the vm upstream ASAP. I will leave the images on a development branch, perhaps indefinitely, because I am not sure that I am on the right track there at all.
On further reflection it is possible that I am on the wrong track with the VM packaging too.
Pharo upstream are distributing releases as binary vm+image pairs with some third party libraries included. The most faithful way to package this for nix would be to take these binaries and make them work (using patchelf to fix up the shared library paths.) This is how people package other binary software for nix e.g. Skype.
The advantage of this approach would be to faithfully reproduce the software as it is released from the Pharo project. This way nix users would expect the software to work equally well as on Ubuntu or Fedora, say. This would also ease support because Pharo upstream would never be bothered by users having problems due to my packaging e.g. using the latest VM with an older image release if that is not supposed to be supported.
This is the advantage of using PharoLauncher, and I wonder if nix is what inspired Damien to develop PharoLauncher. It pairs a static VM and a static image for *itself*, with settings to determine which VM is used. Your packaging should not enter the picture for which VMs and Images users combine together. Although you might
The disadvantage is that this does not suit my own use case. I want to develop a Pharo application, deploy it with nix, and support it myself. The binary releases are too opaque for my taste and I don't want to work with them. In this context I see a net benefit to building everything from sources and tracking all the dependencies explicitly on the nix level. So I would continue to use this source-based packaging myself, and I would not be interested in spending the time to package up the binaries for other people.
Perhaps you can achieve what you want by packaging PharoLauncher separately from the VMs. Then IIUC, the VM-package-version for your application could move on while the VM-package-version for PharoLauncher could remain pinned?? You may find PharoLauncher useful while developing your application. It excels when you want a new clean image from which to test a bug, or generate a submit a bug report or fix, or install a slice of a fix to review.
So - I need to rethink whether I am the right person to maintain the generic pharo packages for nix. (Could be that I should instead treat this as an opensmalltalk-vm packaging that is "use at own risk" with pharo images.)
Hopefully you see some value in keeping PharoLauncher. Perhaps I'll join you on nix after a while. It looks really interesting. But I can understand not wanting to overcommit. cheers -ben
Hi Ben, Thanks for explaining the process for maintaining the stable vs development versions and illustrating it with data. I need to understand the Pharo Launcher better. Could indeed be the best strategy is to focus on supporting that well. That's basically the approach that Damien has taken with the current nix packages. Hopefully we can work out the optimal way to package Pharo once 6.0 has shipped and people have more bandwidth. Just now I have a hard time understanding what VM+image combinations are considered to be "supported" vs which ones are likely to have obscure plugin problems, etc.
On Fri, Apr 21, 2017 at 3:11 AM, Luke Gorrie <luke@snabb.co> wrote:
Hi Ben,
Thanks for explaining the process for maintaining the stable vs development versions and illustrating it with data.
I need to understand the Pharo Launcher better. Could indeed be the best strategy is to focus on supporting that well. That's basically the approach that Damien has taken with the current nix packages.
Hopefully we can work out the optimal way to package Pharo once 6.0 has shipped and people have more bandwidth. Just now I have a hard time understanding what VM+image combinations are considered to be "supported"
In times past, the VM from the previous release has usually remained stable for the most of the next development cycle. Except last year there was a big change from pre-Spur to Spur, and this year a big change from 32-bit to 64-bit, as you can see from the rate of change... https://github.com/OpenSmalltalk/opensmalltalk-vm/graphs/contributors These two very positive events obviously came with more churn in updating the VM.
vs which ones are likely to have obscure plugin problems, etc.
I've not seen many plugin problems reported. Now unless you join the consortium or otherwise pay for support, I would guess you'll get the "best" support on the tip of development, with the risk of being the first to categorise a new bug. What might be good is to periodically define some recommended in-development VM-Image combinations, so we get more people working with a particular combination - to more quickly hit bugs in a particular combo, and provide confidence for the more cautious to move their applications forward. That is, not the effort of a full release, (@Esteban?) Very broadly, I'd stick with the Pharo 6 release for the first 3 - 6 months while any high impact stuff that has been waiting in the wings during the Pharo 6 feature freeze are integrated. Then as Pharo-6 bugs fixed in Pharo-7-dev accumulate, consider moving your personal development to Pharo-7-dev, keeping release on Pharo-6. It probably a good idea port your app to Pharo-7-dev at least three months prior to its release - so any bugs you hit get fixed *in* Pharo 7, otherwise (unless they are critical, or you pay) you may end up waiting until Pharo 8 release. Probably its good to have CI running for both Pharo 6 Release and Pharo-7-dev-tip so your tests pick up any new bugs. https://ci.inria.fr/pharo-contribution/ @Damien might advise how compatible this is with nix. (Disclaimer: Pharo is currently just a (significant) hobby for me, so in practice YMMV.) cheers -ben
participants (2)
-
Ben Coman -
Luke Gorrie