TM
Tim Mackinnon
Wed, Apr 17, 2024 11:31 PM
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
SD
stephane ducasse
Thu, Apr 18, 2024 5:43 AM
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io http://dockerize.io/ - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com http://render.com/ - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
just out of my mind and before breakfast :)
did you see Stdio ?
S
> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>
> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>
> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>
> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>
> But as most things go to the Transcript, and that goes to stdout - it should be good.
>
> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>
> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>
> From memory, you often need to have a Transcript cr. to flush your last line.
>
> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>
> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>
> So I tried: Transcript cr; show: msg; flush
>
> But that didn't seem to work (which I don't understand)
>
> Eventually I did: Transcript show: msg; cr; flush
>
> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>
> Ideally I want to use: Transcript cr'; show: msg; flush
>
> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>
> Anyway - I'm curious if anyone else has done work in this area to shed light?
>
> Thanks,
>
> Tim
>
>
> As an aside - for deployment - several years ago I came across dockerize.io <http://dockerize.io/> - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com <http://render.com/> - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
TM
Tim Mackinnon
Thu, Apr 18, 2024 10:15 AM
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com http://render.com/ - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
> just out of my mind and before breakfast :)
>
> did you see Stdio ?
>
> S
>
>> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>>
>> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>>
>> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>>
>> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>>
>> But as most things go to the Transcript, and that goes to stdout - it should be good.
>>
>> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>>
>> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>>
>> From memory, you often need to have a Transcript cr. to flush your last line.
>>
>> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>>
>> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>>
>> So I tried: Transcript cr; show: msg; flush
>>
>> But that didn't seem to work (which I don't understand)
>>
>> Eventually I did: Transcript show: msg; cr; flush
>>
>> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>>
>> Ideally I want to use: Transcript cr'; show: msg; flush
>>
>> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>>
>> Anyway - I'm curious if anyone else has done work in this area to shed light?
>>
>> Thanks,
>>
>> Tim
>>
>>
>> As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com <http://render.com/> - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
>
> Stéphane Ducasse
> http://stephane.ducasse.free.fr
> 06 30 93 66 73
>
> "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
>
>
TM
Tim Mackinnon
Thu, Apr 18, 2024 3:16 PM
Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com http://render.com/ - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
> Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
>
> It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
> "When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
>
> I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
>
> But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
>
> I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
>
> Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
>
> Tim
>
> On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
>> just out of my mind and before breakfast :)
>>
>> did you see Stdio ?
>>
>> S
>>
>>> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>>>
>>> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>>>
>>> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>>>
>>> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>>>
>>> But as most things go to the Transcript, and that goes to stdout - it should be good.
>>>
>>> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>>>
>>> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>>>
>>> From memory, you often need to have a Transcript cr. to flush your last line.
>>>
>>> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>>>
>>> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>>>
>>> So I tried: Transcript cr; show: msg; flush
>>>
>>> But that didn't seem to work (which I don't understand)
>>>
>>> Eventually I did: Transcript show: msg; cr; flush
>>>
>>> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>>>
>>> Ideally I want to use: Transcript cr'; show: msg; flush
>>>
>>> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>>>
>>> Anyway - I'm curious if anyone else has done work in this area to shed light?
>>>
>>> Thanks,
>>>
>>> Tim
>>>
>>>
>>> As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com <http://render.com/> - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
>>
>> Stéphane Ducasse
>> http://stephane.ducasse.free.fr
>> 06 30 93 66 73
>>
>> "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
>>
>>
>
GP
Guillermo Polito
Fri, Apr 19, 2024 7:16 AM
Hi Tim,
In general, doing a flush should suffice.
Generally, the buffer is flushed automatically when full.
You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
G
El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon tim@testit.works escribió:
Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io http://dockerize.io/ - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com http://render.com/ - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
Hi Tim,
In general, doing a flush should suffice.
Generally, the buffer is flushed automatically when full.
You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
G
> El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon <tim@testit.works> escribió:
>
> Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
>
>
>
> On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
>> Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
>>
>> It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
>> "When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
>>
>> I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
>>
>> But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
>>
>> I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
>>
>> Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
>>
>> Tim
>>
>> On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
>>> just out of my mind and before breakfast :)
>>>
>>> did you see Stdio ?
>>>
>>> S
>>>
>>>> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>>>>
>>>> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>>>>
>>>> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>>>>
>>>> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>>>>
>>>> But as most things go to the Transcript, and that goes to stdout - it should be good.
>>>>
>>>> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>>>>
>>>> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>>>>
>>>> From memory, you often need to have a Transcript cr. to flush your last line.
>>>>
>>>> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>>>>
>>>> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>>>>
>>>> So I tried: Transcript cr; show: msg; flush
>>>>
>>>> But that didn't seem to work (which I don't understand)
>>>>
>>>> Eventually I did: Transcript show: msg; cr; flush
>>>>
>>>> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>>>>
>>>> Ideally I want to use: Transcript cr'; show: msg; flush
>>>>
>>>> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>>>>
>>>> Anyway - I'm curious if anyone else has done work in this area to shed light?
>>>>
>>>> Thanks,
>>>>
>>>> Tim
>>>>
>>>>
>>>> As an aside - for deployment - several years ago I came across dockerize.io <http://dockerize.io/> - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com <http://render.com/> - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
>>>
>>>
>>> Stéphane Ducasse
>>> http://stephane.ducasse.free.fr
>>> 06 30 93 66 73
>>>
>>> "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
RO
Richard O'Keefe
Fri, Apr 19, 2024 11:43 AM
In the C world,
- flush forces data from user-space buffers to OS buffers
- sync forces data from OS buffers to devices
- there is no standard documented way to force devices to move data
from buffers to persistent storage even when the device has such an
operation and the OS knows how to use it.
Is it the same in Pharo?
On Fri, 19 Apr 2024 at 19:16, Guillermo Polito
guillermopolito@gmail.com wrote:
Hi Tim,
In general, doing a flush should suffice.
Generally, the buffer is flushed automatically when full.
You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
G
El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon tim@testit.works escribió:
Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
In the C world,
- flush forces data from user-space buffers to OS buffers
- sync forces data from OS buffers to devices
- there is no standard documented way to force devices to move data
from buffers to persistent storage even when the device has such an
operation and the OS knows how to use it.
Is it the same in Pharo?
On Fri, 19 Apr 2024 at 19:16, Guillermo Polito
<guillermopolito@gmail.com> wrote:
>
> Hi Tim,
>
> In general, doing a flush should suffice.
> Generally, the buffer is flushed automatically when full.
> You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
>
> About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
>
> G
>
> El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon <tim@testit.works> escribió:
>
> Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
>
>
>
> On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
>
> Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
>
> It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
> "When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
>
> I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
>
> But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
>
> I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
>
> Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
>
> Tim
>
> On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
>
> just out of my mind and before breakfast :)
>
> did you see Stdio ?
>
> S
>
> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>
> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>
> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>
> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>
> But as most things go to the Transcript, and that goes to stdout - it should be good.
>
> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>
> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>
> From memory, you often need to have a Transcript cr. to flush your last line.
>
> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>
> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>
> So I tried: Transcript cr; show: msg; flush
>
> But that didn't seem to work (which I don't understand)
>
> Eventually I did: Transcript show: msg; cr; flush
>
> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>
> Ideally I want to use: Transcript cr'; show: msg; flush
>
> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>
> Anyway - I'm curious if anyone else has done work in this area to shed light?
>
> Thanks,
>
> Tim
>
>
> As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
>
>
> Stéphane Ducasse
> http://stephane.ducasse.free.fr
> 06 30 93 66 73
>
> "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
>
>
T
Tim@testit.works
Sat, Apr 20, 2024 11:44 AM
Hey Guillermo what I’ve notice is that if you need to guarantee a line of text is output (as you do for logging before a critical operation) then you :
Transcript show: ‘some log string’; cr
Doesn’t show output when piped to stdio (as when headless mode running on a server). If non critical then I’m sure you can wait for a full buffer, but to get guaranteed output the magic line (and I notice that the BA Bell library does the same ) is:
Transcript show: ‘some log string’; cr; flush
You need the cr AND the flush - the image comment mentions sync - BUT this give a prim failure on Ubuntu Linux - for which I will raise an issue (although Richard says it’s a grey area anyway).
Given the prevalent use of “cr; show:” in the image, I had hoped we could flush a line without a cr, but that doesn’t work (and probably doesn’t make sense in the wider world where log tools want a guaranteed line to parse into log format).
If we want Pharo to play nicely in server environments we should discourage crTrace writing in the image and adopt traceCr writing OR better still have some log writing interface we all use (with some common logfmt conventions like error level) that maps to the transcript but has indirection to map to another log provider. This said - Beacon works well, and tools generating events do let you easily plug in your own and easily write show:’’;cr conventions yourself. This was the case with Zinc which was nice.
Tim
On 19 Apr 2024, at 08:17, Guillermo Polito guillermopolito@gmail.com wrote:
Hi Tim,
In general, doing a flush should suffice.
Generally, the buffer is flushed automatically when full.
You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
G
El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon tim@testit.works escribió:
Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
"When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
Tim
On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
just out of my mind and before breakfast :)
did you see Stdio ?
S
On 18 Apr 2024, at 01:31, Tim Mackinnon tim@testit.works wrote:
Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
But as most things go to the Transcript, and that goes to stdout - it should be good.
HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
From memory, you often need to have a Transcript cr. to flush your last line.
BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
So I tried: Transcript cr; show: msg; flush
But that didn't seem to work (which I don't understand)
Eventually I did: Transcript show: msg; cr; flush
And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
Ideally I want to use: Transcript cr'; show: msg; flush
As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
Anyway - I'm curious if anyone else has done work in this area to shed light?
Thanks,
Tim
As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com http://render.com/ - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
Stéphane Ducasse
http://stephane.ducasse.free.fr
06 30 93 66 73
"If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
Hey Guillermo what I’ve notice is that if you need to guarantee a line of text is output (as you do for logging before a critical operation) then you :
Transcript show: ‘some log string’; cr
Doesn’t show output when piped to stdio (as when headless mode running on a server). If non critical then I’m sure you can wait for a full buffer, but to get guaranteed output the magic line (and I notice that the BA Bell library does the same ) is:
Transcript show: ‘some log string’; cr; flush
You need the cr AND the flush - the image comment mentions sync - BUT this give a prim failure on Ubuntu Linux - for which I will raise an issue (although Richard says it’s a grey area anyway).
Given the prevalent use of “cr; show:” in the image, I had hoped we could flush a line without a cr, but that doesn’t work (and probably doesn’t make sense in the wider world where log tools want a guaranteed line to parse into log format).
If we want Pharo to play nicely in server environments we should discourage crTrace writing in the image and adopt traceCr writing OR better still have some log writing interface we all use (with some common logfmt conventions like error level) that maps to the transcript but has indirection to map to another log provider. This said - Beacon works well, and tools generating events do let you easily plug in your own and easily write show:’’;cr conventions yourself. This was the case with Zinc which was nice.
Tim
> On 19 Apr 2024, at 08:17, Guillermo Polito <guillermopolito@gmail.com> wrote:
> Hi Tim,
>
> In general, doing a flush should suffice.
> Generally, the buffer is flushed automatically when full.
> You generally don’t need to flush manually unless you are in a special case (like, you’re about to quit the process and you want to flush the buffer before quitting).
>
> About the comment, would you mind opening an issue with the exact information, I don’t really get the issue from the blob of text :)
>
> G
>
>> El 18 abr 2024, a las 5:16 p. m., Tim Mackinnon <tim@testit.works> escribió:
>>
>> Just as a comment on this - I tried using #sync as an experiment and I get a primFailed on an Ubuntu docker image - so I think that comment in the image is misleading (or its a bug, or unfinished work)
>>
>>
>>
>> On Thu, 18 Apr 2024, at 11:15 AM, Tim Mackinnon wrote:
>>> Hi Stef - I was aware of that class - but a good reminder to re-read it for any extra info, but I don't see anything in particular mentioned about how data is flushed - and I think this is one of those black arts from in the field.
>>>
>>> It looks like stdio uses an Stdiostream - and I did note an interesting comment in #flush which mention #sync and
>>> "When writing, this syncs any written/flushed data still in the kernel file system buffers to disk. This should generally be used after #flush, but on Windows they do the same thing."
>>>
>>> I've never heard of #sync before - and I wonder if the trick to getting output without the cr is to do a "flush; sync" operation?
>>>
>>> But even then, I am thinking that a server based logging tool is going to need a cr on a log line to know how to interpret it and parse it into syslog format, so perhaps I can't escape moving the a trailing cr policy - and just try to make sure most of my logging adopts that standard (and just accept the odd corruption when other libraries write out crTrace?).
>>>
>>> I'm just curious how others handle this kind of stuff in the wild - maybe I should ask on a Seaside list as they are more likely to delve into this (or maybe the Gemstone guys).
>>>
>>> Its a world you don't normally pay attention to until you try and run something on a server and want to use the modern tools available in that world.
>>>
>>> Tim
>>>
>>> On Thu, 18 Apr 2024, at 6:43 AM, stephane ducasse wrote:
>>>> just out of my mind and before breakfast :)
>>>>
>>>> did you see Stdio ?
>>>>
>>>> S
>>>>
>>>>> On 18 Apr 2024, at 01:31, Tim Mackinnon <tim@testit.works> wrote:
>>>>>
>>>>> Hi - I've been messing around with deploying a hobby pharo app to the web.. which has become a lot simpler over the years, although the tech keeps changing and you have to relearn things.
>>>>>
>>>>> Anyway, I have my image in one of the wonderful BA Docker containers, and it runs well - and the host I'm using will show the logs for you, so you can figure out what is going on... well that is if your logs come out properly (and of course, if it gets really hairy then you can get a VNC session onto the image and figure stuff out)
>>>>>
>>>>> So logs are handy, and pharo these days has a nice headless mode that redirects the Transcript to stdout - and there are also a few decent logging frameworks as well.
>>>>>
>>>>> But as most things go to the Transcript, and that goes to stdout - it should be good.
>>>>>
>>>>> HOWEVER - flushing is the killer, as if things happen and the last thing goes wrong, but the output isn't flushed, then you aren't going to see it.
>>>>>
>>>>> So my question is how to properly flush? And I'm sure I've read something about this before, but I can't find it.
>>>>>
>>>>> From memory, you often need to have a Transcript cr. to flush your last line.
>>>>>
>>>>> BUT, most things in the image seem to use "self crTrace:" these days, which is a cr to ensure the previous msg is separated from what you want to write, and then you write your line out. However, as there is now cr - you might not see it.
>>>>>
>>>>> So I tried changing my stuff to use "self traceCr:" (which is in the image), and that still didn't seem to work - the last failing line wasn't being output. Worse still, its confusing, as many things in the image are using crTrace: and so you get intermingled messages, which are hard to decipher.
>>>>>
>>>>> So I tried: Transcript cr; show: msg; flush
>>>>>
>>>>> But that didn't seem to work (which I don't understand)
>>>>>
>>>>> Eventually I did: Transcript show: msg; cr; flush
>>>>>
>>>>> And this seems to ensure things do reliably get outputted - but I'm wondering if anyone can shed light on this areas?
>>>>>
>>>>> Ideally I want to use: Transcript cr'; show: msg; flush
>>>>>
>>>>> As this plays much better with everything that is in the image - but is there some way to do this? And indeed, will log tools the Bettersatack or papertail play ball with output like this (as I guess they operate on complete lines to interpret log levels etc),
>>>>>
>>>>> Anyway - I'm curious if anyone else has done work in this area to shed light?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Tim
>>>>>
>>>>>
>>>>> As an aside - for deployment - several years ago I came across dockerize.io - which lets you upload a Docker image to a host, and it will run it for you. Sadly that service didn't survive... but there are quite a few like it now, and so I'm trying Render.com <http://render.com/> - which is similar, but the twist is you need to store a Docker image in a registry somewhere (I use gitlab from my CI pipeline), and then it will retrieve it and run it for you (for either free in 40 minute chunks, or for $7/m - which is pretty good, and possibly bit simpler than Digital Ocean). Its pretty cool, and maybe I will write up about it sometime
>>>>
>>>> Stéphane Ducasse
>>>> http://stephane.ducasse.free.fr
>>>> 06 30 93 66 73
>>>>
>>>> "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.” Calvin & Hobbes
