[DEV] Systemd socket activation patch for the VM
Hi folks As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary. Systemd Socket Activation: Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket. Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html Our scenario for socket activation: At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time. With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that donât receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in. The VM Patch: The attached patch tries hard to be minimal. It defines a new socket type (ProvidedTCPSocketType) which you will also find in the image (Socket>>newSystemd). ProvidedTCPSocketType is special in that it can refer to an existing TCPSocket and does not necessarily entail the creation of a new socket. The changes in the image: - Socket>>newSystemd tries to open a new Systemd socket. - TCPListener>>haveSystemdAssignedPort tests if the LISTEN_FDS environment variable has been set by Systemd for the current process. - TCPListener>>primEnvironmentAtSymbol is a utility method that we need to check for the environment variable (IMPORTANT: copied from OSProcess; uses the OSProcess plugin) - TCPListener>>pvtNewListener: has been modified to include the check for the Systemd port. If the socket has been assigned by Systemd, use a Systemd socket, a TCP socket otherwise Quick Howto: - Adapt path to vm and image in pharo@.service - Take a Seaside image and FileIn systemd.cs (tested with http://seaside.st/distributions/Seaside-2.8.4.zip) - Save as 9999.image - Copy pharo@.service and pharo@.socket to /etc/systemd/system - As root run: systemctl daemon-reload systemctl start pharo@9999.socket - Open http://localhost:9999/seaside/config [- systemctl stop pharo@9999.socket] -> Upon connection pharo@9999.socket starts: pharo@9999.service (which would start .../9999.image) You can see this in action in the following screencast: http://youtu.be/MVPLZNKg5j8 We hope that this patch (or some version of it) will find its way into the VM. If you have any questions please feel free to ask. Cheers, Max (on behalf of Nik and the rest of Netstyle.ch)
Thank you for posting this! Dave
Hi folks
As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary.
Systemd Socket Activation: Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket.
Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html
Our scenario for socket activation: At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time. With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that donÂt receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in.
The VM Patch: The attached patch tries hard to be minimal. It defines a new socket type (ProvidedTCPSocketType) which you will also find in the image (Socket>>newSystemd). ProvidedTCPSocketType is special in that it can refer to an existing TCPSocket and does not necessarily entail the creation of a new socket.
The changes in the image: - Socket>>newSystemd tries to open a new Systemd socket. - TCPListener>>haveSystemdAssignedPort tests if the LISTEN_FDS environment variable has been set by Systemd for the current process. - TCPListener>>primEnvironmentAtSymbol is a utility method that we need to check for the environment variable (IMPORTANT: copied from OSProcess; uses the OSProcess plugin) - TCPListener>>pvtNewListener: has been modified to include the check for the Systemd port. If the socket has been assigned by Systemd, use a Systemd socket, a TCP socket otherwise
Quick Howto: - Adapt path to vm and image in pharo@.service - Take a Seaside image and FileIn systemd.cs (tested with http://seaside.st/distributions/Seaside-2.8.4.zip) - Save as 9999.image - Copy pharo@.service and pharo@.socket to /etc/systemd/system - As root run: systemctl daemon-reload systemctl start pharo@9999.socket - Open http://localhost:9999/seaside/config
[- systemctl stop pharo@9999.socket]
-> Upon connection pharo@9999.socket starts: pharo@9999.service (which would start .../9999.image)
You can see this in action in the following screencast: http://youtu.be/MVPLZNKg5j8
We hope that this patch (or some version of it) will find its way into the VM. If you have any questions please feel free to ask.
Cheers, Max (on behalf of Nik and the rest of Netstyle.ch)
this is really good! sebastian o/
On 08/07/2014, at 11:08, Max Leske <maxleske@gmail.com> wrote:
Hi folks
As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary.
Systemd Socket Activation: Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket.
Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html
Our scenario for socket activation: At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time. With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that donât receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in.
The VM Patch: The attached patch tries hard to be minimal. It defines a new socket type (ProvidedTCPSocketType) which you will also find in the image (Socket>>newSystemd). ProvidedTCPSocketType is special in that it can refer to an existing TCPSocket and does not necessarily entail the creation of a new socket.
The changes in the image: - Socket>>newSystemd tries to open a new Systemd socket. - TCPListener>>haveSystemdAssignedPort tests if the LISTEN_FDS environment variable has been set by Systemd for the current process. - TCPListener>>primEnvironmentAtSymbol is a utility method that we need to check for the environment variable (IMPORTANT: copied from OSProcess; uses the OSProcess plugin) - TCPListener>>pvtNewListener: has been modified to include the check for the Systemd port. If the socket has been assigned by Systemd, use a Systemd socket, a TCP socket otherwise
Quick Howto: - Adapt path to vm and image in pharo@.service - Take a Seaside image and FileIn systemd.cs (tested with http://seaside.st/distributions/Seaside-2.8.4.zip) - Save as 9999.image - Copy pharo@.service and pharo@.socket to /etc/systemd/system - As root run: systemctl daemon-reload systemctl start pharo@9999.socket - Open http://localhost:9999/seaside/config
[- systemctl stop pharo@9999.socket]
-> Upon connection pharo@9999.socket starts: pharo@9999.service (which would start .../9999.image)
You can see this in action in the following screencast: http://youtu.be/MVPLZNKg5j8
We hope that this patch (or some version of it) will find its way into the VM. If you have any questions please feel free to ask.
Cheers, Max (on behalf of Nik and the rest of Netstyle.ch)
<systemd.cs> <pharo@.service> <squeak-svn.diff> <pharo@.socket>
On Tue, Jul 8, 2014 at 9:08 AM, Max Leske <maxleske@gmail.com> wrote:
Hi folks
As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary.
Systemd Socket Activation: Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket.
Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html
Our scenario for socket activation: At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time. With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that donât receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in.
OMG, I want that!
Hello all. Has anyone tried this in recent stable Pharo? If anyone can tell me which combinations of image / vm versions would work fine... I have in a Centos 7 vps an 1.4 image and latest stable vm running fine, but when I try to load systemd.cs I get an MNU receiver of "methodsFor:stamp:" is nil because there is no TcpListener class in this image... Regards.. 2014-07-08 11:08 GMT-03:00 Max Leske <maxleske@gmail.com>:
Hi folks
As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary.
Systemd Socket Activation: Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket.
Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html
Our scenario for socket activation: At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time. With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that donât receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in.
The VM Patch: The attached patch tries hard to be minimal. It defines a new socket type (ProvidedTCPSocketType) which you will also find in the image (Socket>>newSystemd). ProvidedTCPSocketType is special in that it can refer to an existing TCPSocket and does not necessarily entail the creation of a new socket.
The changes in the image: - Socket>>newSystemd tries to open a new Systemd socket. - TCPListener>>haveSystemdAssignedPort tests if the LISTEN_FDS environment variable has been set by Systemd for the current process. - TCPListener>>primEnvironmentAtSymbol is a utility method that we need to check for the environment variable (IMPORTANT: copied from OSProcess; uses the OSProcess plugin) - TCPListener>>pvtNewListener: has been modified to include the check for the Systemd port. If the socket has been assigned by Systemd, use a Systemd socket, a TCP socket otherwise
Quick Howto: - Adapt path to vm and image in pharo@.service - Take a Seaside image and FileIn systemd.cs (tested with http://seaside.st/distributions/Seaside-2.8.4.zip) - Save as 9999.image - Copy pharo@.service and pharo@.socket to /etc/systemd/system - As root run: systemctl daemon-reload systemctl start pharo@9999.socket - Open http://localhost:9999/seaside/config
[- systemctl stop pharo@9999.socket]
-> Upon connection pharo@9999.socket starts: pharo@9999.service (which would start .../9999.image)
You can see this in action in the following screencast: http://youtu.be/MVPLZNKg5j8
We hope that this patch (or some version of it) will find its way into the VM. If you have any questions please feel free to ask.
Cheers, Max (on behalf of Nik and the rest of Netstyle.ch)
participants (5)
-
Chris Muller -
David T. Lewis -
Gastón Dall' Oglio -
Max Leske -
Sebastian Sastre