- Pharo-dev - lists.pharo.org

Re: [Pharo-vm] New VM Release v10.3.2

by Norbert Hartl

> Am 05.12.2024 um 14:26 schrieb Norbert Hartl <norbert(a)hartl.name>: > >> >> Also, anybody having a couple of minutes to test and help, if you see this issue locally, could you >> - build a VM (https://github.com/pharo-project/pharo-vm/) checking out tag v10.3.2 >> - run locally the failing tests > > I will take that for creating a vm build on my CI. a small update. I managed in 20 minutes to create a Dockerfile that seems to work: --- # # Dockerfile for building pharo-10 branch under ubuntu 22.04 # FROM ubuntu:22.04 RUN \ apt-get update && \ apt-get install -y wget curl unzip inetutils-ping gnupg libcurl3-gnutls && \ apt-get install -y \ build-essential \ cmake \ git \ python3-minimal \ pkg-config \ libgit2-dev \ libssh-dev \ libsdl2-dev \ libfreetype-dev \ libffi-dev && \ rm -rf /var/lib/apt/lists/* WORKDIR /opt RUN mkdir -p /opt/src WORKDIR /opt/src RUN git clone https://github.com/pharo-project/pharo-vm.git && cd pharo-vm && git checkout pharo-10 RUN cmake -S pharo-vm -B pharo-vm-build RUN cmake --build pharo-vm-build --target install RUN rm -rf pharo-vm-build/build/dist/pharo-vm RUN mv pharo-vm-build/build/dist /opt/pharo-vm RUN rm -rf pharo-vm-build pharo-vm --- Doing a docker build . in that directory builds the vm and installs it under /opt/pharo-vm/pharo In the end it copies the files to a special place. But this could also be a way to zip them and put them somewhere else. The most simple use would be to have a Dockerfile per combination OS-pharoVmVersion. Maybe this would work on github. If I find time I might try somethig along that line. That would make zeroconf survive and still we can support a bit more. Of course it duplicates some work on the OBS but that deals with much more to integrate in the system. For me two uses cases. What I do now is that the Dockerfile above builds and installs the vm in the docker image. I upload these and make them the basis of my docker builds so this is the closest I could get and apart from obstacles that might occur it works. Just in case some has interests in the Dockerfile or volunteering building a repo with those Dockerfiles. Norbert

Dec. 6, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by stephane ducasse

Thanks you for your time and energy fighting with this. This is really great to have you around. I hope that we will get out this situation with a nicer infrastructure. S > On 5 Dec 2024, at 14:04, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > > >> Le 5 dÃ©c. 2024 Ã 1:49â¯PM, Guillermo Polito <guillermopolito(a)gmail.com> a Ã©crit : >> >> >>> Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. >> >> If you want to control what you download, right now the safest way is to go to files.pharo.org <http://files.pharo.org/>, more precisely to https://files.pharo.org/vm/pharo-spur64/, and look for your architecture. >> >> There you will find the archives with the binaries we built. >> Details about naming conventions are found here: https://github.com/pharo-project/pharo-vm/wiki/About-Build-and-Artifacts >> >> That file server stores both released artifacts and intermediate build results. >> The released artifact is the first artifact with the corresponding version tag (e.g., 10.3.1, 10.2.0). >> This is generally the oldest in timestime, but you can confirm with the tags on github. > > Also, we improved the artifact naming to include also: > - suffixes (alpha, RC, SNAPSHOT) > - the distance to the released tag in the format [+x] where x is the commits. > > https://github.com/pharo-project/pharo-vm/pull/870 > > We have tested this in the dev branch, Iâll backport this in the stable branch. > This will allow us to clearly identify the release commit (the one without the +x tag in the name). > > G > _______________________________________________ > Pharo-vm mailing list -- pharo-vm(a)lists.pharo.org > To unsubscribe send an email to pharo-vm-leave(a)lists.pharo.org StÃ©phane Ducasse http://stephane.ducasse.free.fr 06 30 93 66 73 "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.â Calvin & Hobbes

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Jan VranÃ½

Hi, > > > > Overall this is quite unfortunate. I hope there will be a better > > solution then everyone has to build their own vms. I think I need > > to plan some time to do it. > > We understand, and believe me, itâs unfortunate for us too. > We are working on it and hope to have some news in the following > weeks. > > > Speaking of this, it is worse than having to build your own. I'd perfectly happy to do so if I could, but I do not know the secret sauce needed so and nobody seems to know either [1]. In this particular case, I managed in the end to compile it from generated C source [2]. Now I need to back-port some fixes but again, I do not know the secret sauce needed to build a VMMaker image. I'd be great to fix this too (if not already). Best, Jan [1]: https://lists.pharo.org/empathy/thread/M5IOYAHP2QZ2I33OL74JFSRYVUJJR7HI [2]: https://github.com/janvrany/pharo-vm/commit/feb4e8f02aecbac5505b390d8bafb10…

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Guillermo Polito

> Le 5 dÃ©c. 2024 Ã 2:26â¯PM, Norbert Hartl <norbert(a)hartl.name> a Ã©crit : > >> you should have some procedure how to get data so we can help. And everyone complaining about instability is suspect to exactly follow that :) Haha, this is indeed a nice idea, itâs true we have a lot of this in our heads. Iâll come up with something.

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Norbert Hartl

> Am 05.12.2024 um 14:04 schrieb Guillermo Polito <guillermopolito(a)gmail.com>: > > > >> Le 5 dÃ©c. 2024 Ã 1:49â¯PM, Guillermo Polito <guillermopolito(a)gmail.com> a Ã©crit : >> >> >>> Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. >> >> If you want to control what you download, right now the safest way is to go to files.pharo.org <http://files.pharo.org/>, more precisely to https://files.pharo.org/vm/pharo-spur64/, and look for your architecture. >> >> There you will find the archives with the binaries we built. >> Details about naming conventions are found here: https://github.com/pharo-project/pharo-vm/wiki/About-Build-and-Artifacts >> >> That file server stores both released artifacts and intermediate build results. >> The released artifact is the first artifact with the corresponding version tag (e.g., 10.3.1, 10.2.0). >> This is generally the oldest in timestime, but you can confirm with the tags on github. > > Also, we improved the artifact naming to include also: > - suffixes (alpha, RC, SNAPSHOT) > - the distance to the released tag in the format [+x] where x is the commits. perfect. We use git describe | sed -e 's/\-$[0-9][0-9]*$\-.*/.\1/' for that. > > https://github.com/pharo-project/pharo-vm/pull/870 > > We have tested this in the dev branch, Iâll backport this in the stable branch. > This will allow us to clearly identify the release commit (the one without the +x tag in the name). > super! Thank you for the good work!! Norbert > G

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Norbert Hartl

> Am 05.12.2024 um 13:49 schrieb Guillermo Polito <guillermopolito(a)gmail.com>: > >>> I've rolled the stable version back to 10.3.1, please test. >> >> thanks, that means the 10.3.1 is still build with ubuntu and 10.3.2 is debian based? > > The binary for version 10.3.1 was built on ubuntu 16 or 18 I think, maybe Christophe or Pablo can give more details, I donât recall it right now. > > 10.3.2 was built on the new linux server infrastructure. > > $ uname -a > Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux > > We are evaluating different alternatives (e.g., downgrading the build server for a while). > >> Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. > > If you want to control what you download, right now the safest way is to go to files.pharo.org <http://files.pharo.org/>, more precisely to https://files.pharo.org/vm/pharo-spur64/, and look for your architecture. ok, thanks. > > There you will find the archives with the binaries we built. > Details about naming conventions are found here: https://github.com/pharo-project/pharo-vm/wiki/About-Build-and-Artifacts > > That file server stores both released artifacts and intermediate build results. > The released artifact is the first artifact with the corresponding version tag (e.g., 10.3.1, 10.2.0). > This is generally the oldest in timestime, but you can confirm with the tags on github. > > Moreover, the general advice is to avoid zeroconf as much as possible. > We donât have the manpower to maintain build farms for each possible linux flavor. Agreed. It is really hard to do. The only way would be to extract them from the OBS build. OBS's purpose is to have that build farm building all flavors. So if there would be a way to extract files from there and adjusting the zeroconf bash script it would be feasible. But probably not really easy to do. > > Internally, we recently started discussing the deprecation of zeroconf and the migration path to OBS for *nixes. That would be really sad because it is super handy. But it would also be understandable to shut down it down when it does the wrong thing. > The early sketch is that each released version will have its own package in OBS. This is inevitable. > Then a virtual package will gather them all together (and help people update or downgrade on need) This is nice but less important then the above. > More of this will come soon. > cool > >> I think on one of the issues there was a mentioning that Kris add some patch for their CI. > > Iâm not aware of this, can you post a link here? > https://discord.com/channels/223421264751099906/492279831216783360/13141692… >> Maybe this is a good addition to zeroconf? Or can this be achieved differently > > Zeroconf worked very well until now, and we have put lot of effort in its modernization lately. > However, as it is it has problems to scale: > - executing an untrusted script that downloads unverified binaries that is a user choice, putting a warning label is sufficient. > - we manage only one nix build, which works only as soon as all nixes keep compatibility with the version we built on see above > - it is designed to expose two versions: latest and stable. to my memory these versions are fully arbitrary so you could remove them to gain at least clarity. > > Much of this is solved by OBS, and we hope this improves our stability and visibility. > >> >> Overall this is quite unfortunate. I hope there will be a better solution then everyone has to build their own vms. I think I need to plan some time to do it. > > We understand, and believe me, itâs unfortunate for us too. > We are working on it and hope to have some news in the following weeks. > >>> >>> @Norbert et al, we identified at least two issues: >>> - a glibC incompatibility with Ubuntu 20.04 that arises because recently the build servers changed to a Debian, so all zeroconf users are impacted (except those using stable debian). >> >> I can confirm that. The segfault was always in ZdcSocketStream when accessing a native glibc function. Sadly the console I cannot see anymore because too many builds in between. > > Thanks, could you share your OS/setup ? (uname -a probably is enough). Ubuntu 22.04 Linux fe1102acfb5f 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux that is based on a docker ubuntu:22.04 > > Also, anybody having a couple of minutes to test and help, if you see this issue locally, could you > - build a VM (https://github.com/pharo-project/pharo-vm/) checking out tag v10.3.2 > - run locally the failing tests I will take that for creating a vm build on my CI. > > This information could help us reduce the search space, and identify if itâs just the build environment or there is something else we have to deal with. > Yes, you should have some procedure how to get data so we can help. And everyone complaining about instability is suspect to exactly follow that :) thanks again, Norbert > Cheers, > G

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Guillermo Polito

> Le 5 dÃ©c. 2024 Ã 1:49â¯PM, Guillermo Polito <guillermopolito(a)gmail.com> a Ã©crit : > > >> Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. > > If you want to control what you download, right now the safest way is to go to files.pharo.org <http://files.pharo.org/>, more precisely to https://files.pharo.org/vm/pharo-spur64/, and look for your architecture. > > There you will find the archives with the binaries we built. > Details about naming conventions are found here: https://github.com/pharo-project/pharo-vm/wiki/About-Build-and-Artifacts > > That file server stores both released artifacts and intermediate build results. > The released artifact is the first artifact with the corresponding version tag (e.g., 10.3.1, 10.2.0). > This is generally the oldest in timestime, but you can confirm with the tags on github. Also, we improved the artifact naming to include also: - suffixes (alpha, RC, SNAPSHOT) - the distance to the released tag in the format [+x] where x is the commits. https://github.com/pharo-project/pharo-vm/pull/870 We have tested this in the dev branch, Iâll backport this in the stable branch. This will allow us to clearly identify the release commit (the one without the +x tag in the name). G

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Guillermo Polito

>> I've rolled the stable version back to 10.3.1, please test. > > thanks, that means the 10.3.1 is still build with ubuntu and 10.3.2 is debian based? The binary for version 10.3.1 was built on ubuntu 16 or 18 I think, maybe Christophe or Pablo can give more details, I donât recall it right now. 10.3.2 was built on the new linux server infrastructure. $ uname -a Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux We are evaluating different alternatives (e.g., downgrading the build server for a while). > Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. If you want to control what you download, right now the safest way is to go to files.pharo.org <http://files.pharo.org/>, more precisely to https://files.pharo.org/vm/pharo-spur64/, and look for your architecture. There you will find the archives with the binaries we built. Details about naming conventions are found here: https://github.com/pharo-project/pharo-vm/wiki/About-Build-and-Artifacts That file server stores both released artifacts and intermediate build results. The released artifact is the first artifact with the corresponding version tag (e.g., 10.3.1, 10.2.0). This is generally the oldest in timestime, but you can confirm with the tags on github. Moreover, the general advice is to avoid zeroconf as much as possible. We donât have the manpower to maintain build farms for each possible linux flavor. Internally, we recently started discussing the deprecation of zeroconf and the migration path to OBS for *nixes. The early sketch is that each released version will have its own package in OBS. Then a virtual package will gather them all together (and help people update or downgrade on need) More of this will come soon. > I think on one of the issues there was a mentioning that Kris add some patch for their CI. Iâm not aware of this, can you post a link here? > Maybe this is a good addition to zeroconf? Or can this be achieved differently Zeroconf worked very well until now, and we have put lot of effort in its modernization lately. However, as it is it has problems to scale: - executing an untrusted script that downloads unverified binaries - we manage only one nix build, which works only as soon as all nixes keep compatibility with the version we built on - it is designed to expose two versions: latest and stable. Much of this is solved by OBS, and we hope this improves our stability and visibility. > > Overall this is quite unfortunate. I hope there will be a better solution then everyone has to build their own vms. I think I need to plan some time to do it. We understand, and believe me, itâs unfortunate for us too. We are working on it and hope to have some news in the following weeks. >> >> @Norbert et al, we identified at least two issues: >> - a glibC incompatibility with Ubuntu 20.04 that arises because recently the build servers changed to a Debian, so all zeroconf users are impacted (except those using stable debian). > > I can confirm that. The segfault was always in ZdcSocketStream when accessing a native glibc function. Sadly the console I cannot see anymore because too many builds in between. Thanks, could you share your OS/setup ? (uname -a probably is enough). Also, anybody having a couple of minutes to test and help, if you see this issue locally, could you - build a VM (https://github.com/pharo-project/pharo-vm/) checking out tag v10.3.2 - run locally the failing tests This information could help us reduce the search space, and identify if itâs just the build environment or there is something else we have to deal with. Cheers, G

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Norbert Hartl

> Am 05.12.2024 um 11:20 schrieb Guille Polito <guillermo.polito(a)inria.fr>: > > Hi all, > > I've rolled the stable version back to 10.3.1, please test. thanks, that means the 10.3.1 is still build with ubuntu and 10.3.2 is debian based? Just to be sure. In that case it would be nice to have a way to download precisely 10.3.1 in order to stay safe and we have time to plan countermeasure. I think on one of the issues there was a mentioning that Kris add some patch for their CI. Maybe this is a good addition to zeroconf? Or can this be achieved differently Overall this is quite unfortunate. I hope there will be a better solution then everyone has to build their own vms. I think I need to plan some time to do it. > > @Norbert et al, we identified at least two issues: > - a glibC incompatibility with Ubuntu 20.04 that arises because recently the build servers changed to a Debian, so all zeroconf users are impacted (except those using stable debian). I can confirm that. The segfault was always in ZdcSocketStream when accessing a native glibc function. Sadly the console I cannot see anymore because too many builds in between. Norbert > - a vm crash that we are still investigating > > If you have the second issue, and you have a small reproducible case (like some small test set in your open source project) that we can use to reproduce the issue, that will accelerate our debugging process. > > Thanks all for understanding, > G > >> Le 5 dÃ©c. 2024 Ã 10:45â¯AM, Norbert Hartl <norbert(a)hartl.name> a Ã©crit : >> >> Thanks. >> >> And please tell us what data we can generate in order to help. It happened on our CI and this is a place where I can introduce stuff which generates data if it does not become too slow. >> >> Norbert >> >>> Am 05.12.2024 um 08:42 schrieb Guille Polito <guillermo.polito(a)inria.fr>: >>> >>> Hi all, >>> >>> We had reports of issues observed with this version, we will rollback during the morning and after proceed to study the indicent. >>> >>> G >>> >>>> Le 4 dÃ©c. 2024 Ã 10:56â¯AM, Guille Polito <guillermo.polito(a)inria.fr> a Ã©crit : >>>> >>>> Hi all, >>>> >>>> Weâve published a new VM release (v10.3.2) to be used with Pharo versions 10 through 13. >>>> Please check the change log here: >>>> >>>> https://github.com/pharo-project/pharo-vm/releases/tag/v10.3.2 >>>> >>>> The build is on the oven, new VMs should be available as soon as the build farm finishes. >>>> >>>> Cheers, >>>> G >>>> _______________________________________________ >>>> Pharo-vm mailing list -- pharo-vm(a)lists.pharo.org >>>> To unsubscribe send an email to pharo-vm-leave(a)lists.pharo.org >>> >> >> _______________________________________________ >> Pharo-vm mailing list -- pharo-vm(a)lists.pharo.org >> To unsubscribe send an email to pharo-vm-leave(a)lists.pharo.org >

Dec. 5, 2024

Re: [Pharo-vm] New VM Release v10.3.2

by Guille Polito

Hi all, I've rolled the stable version back to 10.3.1, please test. @Norbert et al, we identified at least two issues: - a glibC incompatibility with Ubuntu 20.04 that arises because recently the build servers changed to a Debian, so all zeroconf users are impacted (except those using stable debian). - a vm crash that we are still investigating If you have the second issue, and you have a small reproducible case (like some small test set in your open source project) that we can use to reproduce the issue, that will accelerate our debugging process. Thanks all for understanding, G > Le 5 dÃ©c. 2024 Ã 10:45â¯AM, Norbert Hartl <norbert(a)hartl.name> a Ã©crit : > > Thanks. > > And please tell us what data we can generate in order to help. It happened on our CI and this is a place where I can introduce stuff which generates data if it does not become too slow. > > Norbert > >> Am 05.12.2024 um 08:42 schrieb Guille Polito <guillermo.polito(a)inria.fr>: >> >> Hi all, >> >> We had reports of issues observed with this version, we will rollback during the morning and after proceed to study the indicent. >> >> G >> >>> Le 4 dÃ©c. 2024 Ã 10:56â¯AM, Guille Polito <guillermo.polito(a)inria.fr> a Ã©crit : >>> >>> Hi all, >>> >>> Weâve published a new VM release (v10.3.2) to be used with Pharo versions 10 through 13. >>> Please check the change log here: >>> >>> https://github.com/pharo-project/pharo-vm/releases/tag/v10.3.2 >>> >>> The build is on the oven, new VMs should be available as soon as the build farm finishes. >>> >>> Cheers, >>> G >>> _______________________________________________ >>> Pharo-vm mailing list -- pharo-vm(a)lists.pharo.org >>> To unsubscribe send an email to pharo-vm-leave(a)lists.pharo.org >> > > _______________________________________________ > Pharo-vm mailing list -- pharo-vm(a)lists.pharo.org > To unsubscribe send an email to pharo-vm-leave(a)lists.pharo.org

Dec. 5, 2024