Hi guys, Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo? I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built? or do you prefer install packages on building? (as done by pip -r requirements.txt) Anyone using xpra binding from subuser [1]? Anyone could make it work Pharo on Alpine since it uses different libc? Anyway sharing opinions would be also appreciated. [1] http://subuser.org/news/0.3.html Cheers. Hernán
On Mon, Oct 14, 2019 at 04:07:49AM -0300, Hernán Morales Durand wrote:
Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo? Anyone could make it work Pharo on Alpine since it uses different libc?
Hi Hernán, Please see following links: - https://github.com/pharo-contributions/Docker-Alpine - https://hub.docker.com/r/pierceng/pharovm-alpine - https://hub.docker.com/r/pierceng/pharo7-alpine - https://www.samadhiweb.com/blog/2019.07.20.alpine.pharo.minimal.html - https://www.samadhiweb.com/blog/2019.08.11.minimizing.pharo.html The Alpine-based VM can be made smaller by removing unused/irrelevant modules. Image-wise, currently I am using Pharo 7 minimal, which is just under 30MB, and that forms the lower bound of the Pharo image size. However there is also the Pharo Candle effort that aims to make really small images, which I hope to try out soon.
I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built?
For Pharo, I start with the regular or minimal image, then load my packages. For Docker, I first build a container image for the VM only. Using that I build a new container image from the Pharo image and other required files.
Interesting. I use Firejail for big programs such as Chromium. Pierce
El lun., 14 oct. 2019 a las 5:23, Pierce Ng (<pierce@samadhiweb.com>) escribió:
On Mon, Oct 14, 2019 at 04:07:49AM -0300, Hernán Morales Durand wrote:
Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo? Anyone could make it work Pharo on Alpine since it uses different libc?
Hi Hernán,
Please see following links:
- https://github.com/pharo-contributions/Docker-Alpine - https://hub.docker.com/r/pierceng/pharovm-alpine - https://hub.docker.com/r/pierceng/pharo7-alpine - https://www.samadhiweb.com/blog/2019.07.20.alpine.pharo.minimal.html - https://www.samadhiweb.com/blog/2019.08.11.minimizing.pharo.html
Nice! It works with latest minimal 7.0.4 too. BTW you used .sources whith seem to be for Pharo 7 (Pharo7.0-32bit-0903ade.sources), but when I tried with latest stable minimal and for some reason requires the V60 sources: $ docker run --rm --ulimit rtprio=2 pharo7_docker_alpine printVersion Pharo cannot locate the sources file named /pkg/image/PharoV60.sources. I found also another approach using rtprio=2:2 suggested from https://github.com/ba-st/docker-pharo/blob/master/docs/rtprio.md (soft:hard ulimits). The Alpine-based VM can be made smaller by removing unused/irrelevant
modules.
Image-wise, currently I am using Pharo 7 minimal, which is just under 30MB, and that forms the lower bound of the Pharo image size.
The built image here is aprox. 99 Mbytes REPOSITORY TAG IMAGE ID CREATED SIZE pharo7_docker_alpine latest 595b9ece2625 3 hours ago 99.4MB but I think you didn't sumed the .sources.
However there is also the Pharo Candle effort that aims to make really small images, which I hope to try out soon.
I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built?
For Pharo, I start with the regular or minimal image, then load my packages.
For Docker, I first build a container image for the VM only. Using that I build a new container image from the Pharo image and other required files.
Thank you for sharing Pierce. Hernán
Example: # Stage 1: Load the project on an empty image, in this case this is a development image (should change to use a minimal image) FROM basmalltalk/pharo:7.0-image AS loader COPY load-project.st ./ RUN pharo Pharo.image load-project.st --save --quit # Stage 2: Copy the resulting Pharo.image with our project loaded # into a new docker image with just the vm # start in a new stage to keep docker image minimal FROM basmalltalk/pharo:7.0 USER root WORKDIR /opt/app COPY start.sh . COPY --from=loader /opt/pharo/Pharo.image ./ COPY --from=loader /opt/pharo/Pharo.changes ./ COPY --from=loader /opt/pharo/Pharo*.sources ./ RUN mkdir logs \ && chmod a+x start.sh \ && chown --recursive pharo:pharo /opt/app USER pharo EXPOSE 8080 CMD ["./start.sh"] The resulting image size is ~80mb including image, changes & sources Reference: https://docs.docker.com/develop/develop-images/multistage-build/ As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image. It's important to keep the docker image thin, but also to keep layers at a minimum and not adding / removing stuff during the build, because those are stored in the middle layers (multi stage helps to remove that). Cheers. On Tue, 15 Oct 2019 at 20:00, Hernán Morales Durand < hernan.morales@gmail.com> wrote:
El lun., 14 oct. 2019 a las 5:23, Pierce Ng (<pierce@samadhiweb.com>) escribió:
On Mon, Oct 14, 2019 at 04:07:49AM -0300, Hernán Morales Durand wrote:
Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo? Anyone could make it work Pharo on Alpine since it uses different libc?
Hi Hernán,
Please see following links:
- https://github.com/pharo-contributions/Docker-Alpine - https://hub.docker.com/r/pierceng/pharovm-alpine - https://hub.docker.com/r/pierceng/pharo7-alpine - https://www.samadhiweb.com/blog/2019.07.20.alpine.pharo.minimal.html - https://www.samadhiweb.com/blog/2019.08.11.minimizing.pharo.html
Nice! It works with latest minimal 7.0.4 too.
BTW you used .sources whith seem to be for Pharo 7 (Pharo7.0-32bit-0903ade.sources), but when I tried with latest stable minimal and for some reason requires the V60 sources:
$ docker run --rm --ulimit rtprio=2 pharo7_docker_alpine printVersion Pharo cannot locate the sources file named /pkg/image/PharoV60.sources.
I found also another approach using rtprio=2:2 suggested from https://github.com/ba-st/docker-pharo/blob/master/docs/rtprio.md (soft:hard ulimits).
The Alpine-based VM can be made smaller by removing unused/irrelevant
modules.
Image-wise, currently I am using Pharo 7 minimal, which is just under 30MB, and that forms the lower bound of the Pharo image size.
The built image here is aprox. 99 Mbytes
REPOSITORY TAG IMAGE ID CREATED SIZE pharo7_docker_alpine latest 595b9ece2625 3 hours ago 99.4MB
but I think you didn't sumed the .sources.
However there is also the Pharo Candle effort that aims to make really small images, which I hope to try out soon.
I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built?
For Pharo, I start with the regular or minimal image, then load my packages.
For Docker, I first build a container image for the VM only. Using that I build a new container image from the Pharo image and other required files.
Thank you for sharing Pierce.
Hernán
El mié., 16 oct. 2019 a las 19:49, Julián Maestri (<serpi90@gmail.com>) escribió:
Example:
# Stage 1: Load the project on an empty image, in this case this is a development image (should change to use a minimal image)
FROM basmalltalk/pharo:7.0-image AS loader COPY load-project.st ./ RUN pharo Pharo.image load-project.st --save --quit
# Stage 2: Copy the resulting Pharo.image with our project loaded # into a new docker image with just the vm # start in a new stage to keep docker image minimal FROM basmalltalk/pharo:7.0
USER root
WORKDIR /opt/app COPY start.sh . COPY --from=loader /opt/pharo/Pharo.image ./ COPY --from=loader /opt/pharo/Pharo.changes ./ COPY --from=loader /opt/pharo/Pharo*.sources ./
RUN mkdir logs \ && chmod a+x start.sh \ && chown --recursive pharo:pharo /opt/app
USER pharo EXPOSE 8080
CMD ["./start.sh"]
The resulting image size is ~80mb including image, changes & sources Reference: https://docs.docker.com/develop/develop-images/multistage-build/
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
It's important to keep the docker image thin, but also to keep layers at a minimum and not adding / removing stuff during the build, because those are stored in the middle layers (multi stage helps to remove that).
Cool!! Thank you Julián, Cheers, Hernán
Cheers.
On Tue, 15 Oct 2019 at 20:00, Hernán Morales Durand < hernan.morales@gmail.com> wrote:
El lun., 14 oct. 2019 a las 5:23, Pierce Ng (<pierce@samadhiweb.com>) escribió:
On Mon, Oct 14, 2019 at 04:07:49AM -0300, Hernán Morales Durand wrote:
Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo? Anyone could make it work Pharo on Alpine since it uses different libc?
Hi Hernán,
Please see following links:
- https://github.com/pharo-contributions/Docker-Alpine - https://hub.docker.com/r/pierceng/pharovm-alpine - https://hub.docker.com/r/pierceng/pharo7-alpine - https://www.samadhiweb.com/blog/2019.07.20.alpine.pharo.minimal.html - https://www.samadhiweb.com/blog/2019.08.11.minimizing.pharo.html
Nice! It works with latest minimal 7.0.4 too.
BTW you used .sources whith seem to be for Pharo 7 (Pharo7.0-32bit-0903ade.sources), but when I tried with latest stable minimal and for some reason requires the V60 sources:
$ docker run --rm --ulimit rtprio=2 pharo7_docker_alpine printVersion Pharo cannot locate the sources file named /pkg/image/PharoV60.sources.
I found also another approach using rtprio=2:2 suggested from https://github.com/ba-st/docker-pharo/blob/master/docs/rtprio.md (soft:hard ulimits).
The Alpine-based VM can be made smaller by removing unused/irrelevant
modules.
Image-wise, currently I am using Pharo 7 minimal, which is just under 30MB, and that forms the lower bound of the Pharo image size.
The built image here is aprox. 99 Mbytes
REPOSITORY TAG IMAGE ID CREATED SIZE pharo7_docker_alpine latest 595b9ece2625 3 hours ago 99.4MB
but I think you didn't sumed the .sources.
However there is also the Pharo Candle effort that aims to make really small images, which I hope to try out soon.
I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built?
For Pharo, I start with the regular or minimal image, then load my packages.
For Docker, I first build a container image for the VM only. Using that I build a new container image from the Pharo image and other required files.
Thank you for sharing Pierce.
Hernán
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well. Norbert
I havenât tried in a while, but in 2017 with PharoLambda I had a combined Pharo image and VM size if 21 mb using the early Pharo minimal (I recall it was an early 7.0 image ). I was loading a simple hello Alexa app, so not a ton of code (but it had Neo Json and other AWS libs as a dependency I recall). Iâm interested in trying these Docker experiments, so Iâll have to look at some point and see if I can get similar sizes. As I scripted my build, I have the steps laid out. I recall there were many vm plugins not needed for a server install (sound etc) and I also ran a cleanup step in the image as there was lots of metacello stuff cached ... so Iâm sure tinier is possible even without candle. Tim Sent from my iPhone
On 18 Oct 2019, at 07:48, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well.
Norbert
Tim, is there a document anywhere explaining how to do a lambda image? Norbert
Am 18.10.2019 um 11:00 schrieb Tim Mackinnon <tim@testit.works>:
I havenât tried in a while, but in 2017 with PharoLambda I had a combined Pharo image and VM size if 21 mb using the early Pharo minimal (I recall it was an early 7.0 image ). I was loading a simple hello Alexa app, so not a ton of code (but it had Neo Json and other AWS libs as a dependency I recall).
Iâm interested in trying these Docker experiments, so Iâll have to look at some point and see if I can get similar sizes.
As I scripted my build, I have the steps laid out. I recall there were many vm plugins not needed for a server install (sound etc) and I also ran a cleanup step in the image as there was lots of metacello stuff cached ... so Iâm sure tinier is possible even without candle.
Tim
Sent from my iPhone
On 18 Oct 2019, at 07:48, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well.
Norbert
Hi Norbert, Last year I did some experiments with Pharo Lambda functions, see: https://github.com/jvdsandt/pharo-aws-toolbox/blob/master/doc/pharo-lambda-r... I didn't spend a lot of time on minimizing the image. There is still a lot of room for improvement. Jan. On Fri, Oct 18, 2019 at 12:30 PM Norbert Hartl <norbert@hartl.name> wrote:
Tim,
is there a document anywhere explaining how to do a lambda image?
Norbert
Am 18.10.2019 um 11:00 schrieb Tim Mackinnon <tim@testit.works>:
I havenât tried in a while, but in 2017 with PharoLambda I had a combined Pharo image and VM size if 21 mb using the early Pharo minimal (I recall it was an early 7.0 image ). I was loading a simple hello Alexa app, so not a ton of code (but it had Neo Json and other AWS libs as a dependency I recall).
Iâm interested in trying these Docker experiments, so Iâll have to look at some point and see if I can get similar sizes.
As I scripted my build, I have the steps laid out. I recall there were many vm plugins not needed for a server install (sound etc) and I also ran a cleanup step in the image as there was lots of metacello stuff cached ... so Iâm sure tinier is possible even without candle.
Tim
Sent from my iPhone
On 18 Oct 2019, at 07:48, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well.
Norbert
-- Jan van de Sandt gsm: +31 (0)6 3039 5998
Hi Norbert - itâs all in the gitlab repo (the idea was to fork it and configure your own pipeline e vars) However the key stuff was in the scripts dir, and this file - /scripts/build.sh Which also loads some .st files for image fix ups . Tim Sent from my iPhone Sent from my iPhone
On 18 Oct 2019, at 16:07, Jan van de Sandt <jvdsandt@gijjes.nl> wrote:
Hi Norbert,
Last year I did some experiments with Pharo Lambda functions, see:
https://github.com/jvdsandt/pharo-aws-toolbox/blob/master/doc/pharo-lambda-r...
I didn't spend a lot of time on minimizing the image. There is still a lot of room for improvement.
Jan.
On Fri, Oct 18, 2019 at 12:30 PM Norbert Hartl <norbert@hartl.name> wrote: Tim,
is there a document anywhere explaining how to do a lambda image?
Norbert
Am 18.10.2019 um 11:00 schrieb Tim Mackinnon <tim@testit.works>:
I havenât tried in a while, but in 2017 with PharoLambda I had a combined Pharo image and VM size if 21 mb using the early Pharo minimal (I recall it was an early 7.0 image ). I was loading a simple hello Alexa app, so not a ton of code (but it had Neo Json and other AWS libs as a dependency I recall).
Iâm interested in trying these Docker experiments, so Iâll have to look at some point and see if I can get similar sizes.
As I scripted my build, I have the steps laid out. I recall there were many vm plugins not needed for a server install (sound etc) and I also ran a cleanup step in the image as there was lots of metacello stuff cached ... so Iâm sure tinier is possible even without candle.
Tim
Sent from my iPhone
On 18 Oct 2019, at 07:48, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well.
Norbert
-- Jan van de Sandt gsm: +31 (0)6 3039 5998
This is a better link: https://workingcopy.app/git/#path=scripts/build.sh&repo=git@gitlab.com:macta... Sent from my iPhone
On 18 Oct 2019, at 16:18, Tim Mackinnon <tim@testit.works> wrote:
Hi Norbert - itâs all in the gitlab repo (the idea was to fork it and configure your own pipeline e vars)
However the key stuff was in the scripts dir, and this file - /scripts/build.sh
Which also loads some .st files for image fix ups .
Tim
Sent from my iPhone
Sent from my iPhone
On 18 Oct 2019, at 16:07, Jan van de Sandt <jvdsandt@gijjes.nl> wrote:
Hi Norbert,
Last year I did some experiments with Pharo Lambda functions, see:
https://github.com/jvdsandt/pharo-aws-toolbox/blob/master/doc/pharo-lambda-r...
I didn't spend a lot of time on minimizing the image. There is still a lot of room for improvement.
Jan.
On Fri, Oct 18, 2019 at 12:30 PM Norbert Hartl <norbert@hartl.name> wrote: Tim,
is there a document anywhere explaining how to do a lambda image?
Norbert
Am 18.10.2019 um 11:00 schrieb Tim Mackinnon <tim@testit.works>:
I havenât tried in a while, but in 2017 with PharoLambda I had a combined Pharo image and VM size if 21 mb using the early Pharo minimal (I recall it was an early 7.0 image ). I was loading a simple hello Alexa app, so not a ton of code (but it had Neo Json and other AWS libs as a dependency I recall).
Iâm interested in trying these Docker experiments, so Iâll have to look at some point and see if I can get similar sizes.
As I scripted my build, I have the steps laid out. I recall there were many vm plugins not needed for a server install (sound etc) and I also ran a cleanup step in the image as there was lots of metacello stuff cached ... so Iâm sure tinier is possible even without candle.
Tim
Sent from my iPhone
On 18 Oct 2019, at 07:48, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.10.2019 um 02:00 schrieb Julián Maestri <serpi90@gmail.com>:
As a side note, the final image size is not what really matters, if you have 20 different images all starting from the same base image (eg ubuntu:18.04) the base layer is shared among all images so the network / disk usage is less than the total size of the image.
The overlayfs does only help here with the disk storage that is not multiplied. As the image is a memory dump of an individual image nothing can be shared there. So if you have 20 images you have 20 times the heap. So a small image is actually important. The vm is different. It is the same static file which will be paged into shared memory and the vm binary should be shared for all 20 runtimes. But the interpreter memory will not be shared so a small vm pays out as well.
Norbert
-- Jan van de Sandt gsm: +31 (0)6 3039 5998
Hi,
Am 14.10.2019 um 09:07 schrieb Hernán Morales Durand <hernan.morales@gmail.com>:
Hi guys,
Because I am lazy and want to avoid searching through all DockerHub repository pages... Do you know a Dockerfile to generate the smallest possible docker image for Pharo?
Please have a look at https://github.com/pharo-contributions/Docker-Alpine <https://github.com/pharo-contributions/Docker-Alpine> This was the first step in unify work on this. Itâs based on alpine. It may need to be ironed out a bit as alpine has a different libc implementation. Further steps should include building a pharo minimal image to bring it down in size. 10 MB per docker image is theoretically possible.
I am also interested in which stage you use to install packages into the image, and why? Do you copy the contained image into docker already built? or do you prefer install packages on building? (as done by pip -r requirements.txt) Anyone using xpra binding from subuser [1]? Anyone could make it work Pharo on Alpine since it uses different libc?
Building the docker image you can do either way. I have one public project where I provide the software as docker image. Doing this is easier when you build the pharo image inside the docker container. Because you can then use dockerhub [1] to build it. You just add your project there and it builds it and releases it. In our inhouse products I use building the image outside of the docker container. The process is building the product image then load the test code and test it and when everything is fine use the image and package as a docker image. I donât know subuser but using xpra will bloat the docker image because AFAIK it works over X11 which means you need those libraries in the docker container and in the pharo vm. What Iâm working on is to be able to connect a running docker container directly from my laptop and use TelePharo [2] on it. Norbert
Anyway sharing opinions would be also appreciated.
[1] http://subuser.org/news/0.3.html <http://subuser.org/news/0.3.html>
Cheers.
Hernán
[1] https://hub.docker.com <https://hub.docker.com/>[2] https://github.com/pharo-ide/TelePharo <https://github.com/pharo-ide/TelePharo>
participants (6)
-
Hernán Morales Durand -
Jan van de Sandt -
Julián Maestri -
Norbert Hartl -
Pierce Ng -
Tim Mackinnon