Le 12/01/2015 18:32, phil@highoctane.be a écrit :

There are lots of things. Hi Phil, thank you, what things would you advice ? I'm not a java zealot, it's just that I found only that one, and that it works in Java without effort - I'd prefer not to do my own.

But Log4s works and yeah, it is similar to Log4j in concept.

Phil

On Mon, Jan 12, 2015 at 6:27 PM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/__Log4s.html/Wiki <http://ss3.gemstone.com/ss/Log4s.html/Wiki>

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ? TIA -- Regards,

Alain

-- Regards, Alain

Le 12/01/2015 18:56, Sven Van Caekenberghe a écrit :

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

Thank you Sven, Googling for those names gave me more interesting results :) I'll dig into that

...

TIA -- Regards,

Alain

-- Regards, Alain

Le 12/01/2015 19:11, stepharo a écrit :

Le 12/1/15 18:56, Sven Van Caekenberghe a écrit :

...

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ? Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

we will merge SystemLogger and Beacon. Now there is a full documentation about SystemLogger at https://github.com/SquareBracketAssociates/PharoReadyForReviews

Cool chapters. And I found the projects on smalltalk hub. thank you Stef

...

...

TIA -- Regards,

Alain

-- Regards, Alain

Le 12/01/2015 19:30, stepharo a écrit :

If you need the pdf just ask we should put back the ci crushing pillar and producing pdf the text is readable even with tags, and figures in png and pdf, so it's ok for me, thanks anyway. may be time for me to try using pillar, I saw the doc of Damien Cassou and delayed trying it but and it looks cool.

Le 12/1/15 19:25, Alain Rastoul a écrit :

...

Le 12/01/2015 19:11, stepharo a écrit :

...

Le 12/1/15 18:56, Sven Van Caekenberghe a écrit :

...

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ? Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

we will merge SystemLogger and Beacon. Now there is a full documentation about SystemLogger at https://github.com/SquareBracketAssociates/PharoReadyForReviews

Cool chapters. And I found the projects on smalltalk hub. thank you Stef

...

...

...

TIA -- Regards,

Alain

-- Regards, Alain

Hi, You can details about beacon here: http://www.humane-assessment.com/blog/beacon Cheers, Doru On Mon, Jan 12, 2015 at 7:52 PM, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

Le 12/01/2015 18:56, Sven Van Caekenberghe a écrit :

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

...

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

found your video on ZnLogEvent on youtube +1 for logging objects ... :)

...

TIA

...

-- Regards,

Alain

-- Regards,

Alain

-- www.tudorgirba.com "Every thing has its own flow"

These are nice. There is still value in Log4s with its variery of appenders.

what is an appender? And yes soon I will come back to Beacon+SystemLogger+

Thanks Le 13/1/15 17:10, S Krish a écrit :

Appenders in Log4J describe an end point to send the log data/ event object to.

So you can have a rolling_file , socket TCP connection, MQ , DB table , console ... any you need, many already provided with their custom code to pipe the info to the end point.

Appenders are now configurable ( in code ) typically in XML to be able to pipe it to a file in Production / or a DB . While in dev it is to the console.

You can have the same log sent to multiple Appenders with events filtered logically for each of them, with one of the Appender getting the whole dump of events.

The critical issue in logging is performance impact of logging too much.

On Tue, Jan 13, 2015 at 5:46 PM, stepharo <stepharo@free.fr <mailto:stepharo@free.fr>> wrote:

These are nice. There is still value in Log4s with its variery of appenders.

what is an appender?

And yes soon I will come back to Beacon+SystemLogger+

2015-01-13 13:42 GMT-03:00 stepharo <stepharo@free.fr>:

You see if people using logging frameworks would have look for real at Beacon or systemLogger and add one single appenders and publish the code then we would have more of them. We will merge Beacon and SystemLogger with Doru and Norbert and we will slowly add some backends.

Of course you can do whatever you want in Pharo, but people will still choose.

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :( So if Object logging is implementing #asLog and #logClass methods maybe I should add them so Log4s can log "Object" :). But seriously, such comparison is not fair. Let's compare real logging features: priorities, levels, appenders, output destinations, hierarchical logging, formatting layout, etc. Many of them are implemented in Log4s. Cheers, Hernán 2015-01-12 14:56 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

...

TIA -- Regards,

Alain

2015-01-13 5:17 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 13 Jan 2015, at 07:00, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

Maybe we should do away with Events as objects, Methods as objects, Exceptions as objects, Announcements as objects, Ring, MC, Commands as objects, Windows, Menus, ... - well, this whole objects everywhere thing is silly, let's use strings instead ;-)

Sarcasm is never the answer. Let's focus on logging, and specifically the case with remote image where you don't have access to "local log objects". I suppose there are other (specially web) developers with the same requirement.

We'll keep on trying to explain and to explore the possibilities.

Log objects are much more powerful, less expensive than you think, and backwards compatible with textual output.

If I have to use a bayesian model with markov chains, which could take up to 1 million of samplings only to begin stabilization, and I want to log them... do you expect to have every entry as a Log object... right? That would be less expensive in terms of memory and speed than flushing to a text file?

Logging is deceptively simple, managing megabytes of log files is a PITA. Structure, classification, intelligence, behaviour are the answers.

BTW, it is not that you are not allowed to log some simple things to the Transcript, it is that it does not really scale, conceptually.

In my system I am not logging to the Transcript (which obviously does not scale). Cheers, Hernán

2015-01-13 14:50 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 13 Jan 2015, at 18:27, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

2015-01-13 5:17 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 13 Jan 2015, at 07:00, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

Maybe we should do away with Events as objects, Methods as objects, Exceptions as objects, Announcements as objects, Ring, MC, Commands as objects, Windows, Menus, ... - well, this whole objects everywhere thing is silly, let's use strings instead ;-)

Sarcasm is never the answer. Let's focus on logging, and specifically the case with remote image where you don't have access to "local log objects". I suppose there are other (specially web) developers with the same requirement.

The analogy was to make a point, but you don't seem to see it, too bad. It is technically easy to export log objects out of an image.

Instead of analogies, I would love to read about how your Log system proposals works with headless images. Because to me it is really easy to write a grep command (I don't have to *explore* logs).

...

We'll keep on trying to explain and to explore the possibilities.

Log objects are much more powerful, less expensive than you think, and backwards compatible with textual output.

If I have to use a bayesian model with markov chains, which could take up to 1 million of samplings only to begin stabilization, and I want to log them... do you expect to have every entry as a Log object... right? That would be less expensive in terms of memory and speed than flushing to a text file?

That would only be true if you kept them all in memory. That is an orthogonal point.

So you didn't answered my questions. Do I have to create 1 million of log objects and that is less expensive than flushing them as they occur into a file? What's the point of creating objects if I don't need them?

<sorry-for-the-sarcasm>A String instance is an object too, composing/formatting a log message is a computation too, similar to serialising an object in some format.</sorry-for-the-sarcasm>

I see. In the end we (still) read Strings, whatever conceptualization is above them.

...

Logging is deceptively simple, managing megabytes of log files is a PITA. Structure, classification, intelligence, behaviour are the answers.

BTW, it is not that you are not allowed to log some simple things to the Transcript, it is that it does not really scale, conceptually.

In my system I am not logging to the Transcript (which obviously does not scale).

I was using the Transcript as an example of a text based logging facility. I meant that it doesn't scale conceptually, it becomes one big mess of dozens of log files that helps very little.

Look, you don't have to use any form of Object Logging if you fail to see the point, there is nothing wrong with logging to a file. I was just trying to explain.

The point is isn't just me. I haven't seen any intensive computation software which creates log objects for self-exploration. And if you find one, I could cite 100 of them which does "String logging", and they just work, and people using it is not stupid, they know they have limitations, it happens that they prioritize other requirements.

We have dozens of servers producing many, many log files, most of them produced by log4j, writing more than 1Gb a day. 90% of that is a waste, but it is never enough. And it is a big mess. Even our Java developers say so. And what are they looking at ? Right, storing JSON log objects in searchable databases. (LogStash, ElasticSearch, blabla, ..).

It is weird to read sometimes how Java developers have it so wrong, and sometimes they are visionary souls which drive us into the future. Hernán

2015-01-13 17:08 GMT-03:00 Norbert Hartl <norbert@hartl.name>:

Am 13.01.2015 um 19:41 schrieb Hernán Morales Durand < hernan.morales@gmail.com>:

2015-01-13 14:50 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

...

On 13 Jan 2015, at 18:27, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

2015-01-13 5:17 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 13 Jan 2015, at 07:00, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

Maybe we should do away with Events as objects, Methods as objects, Exceptions as objects, Announcements as objects, Ring, MC, Commands as objects, Windows, Menus, ... - well, this whole objects everywhere thing is silly, let's use strings instead ;-)

Sarcasm is never the answer. Let's focus on logging, and specifically the case with remote image where you don't have access to "local log objects". I suppose there are other (specially web) developers with the same requirement.

The analogy was to make a point, but you don't seem to see it, too bad. It is technically easy to export log objects out of an image.

Instead of analogies, I would love to read about how your Log system proposals works with headless images. Because to me it is really easy to write a grep command (I don't have to *explore* logs).

It can work the way you want it to. The main difference might be that you don't produce strings inside your code when logging. You create objects with contextual information and you emit them. A convert/outputter/appender can then convert the objects into a desired format before it leaves the image. Or you can keep the objects in the image. The only difference might be that we keep objects as long as possible and postpone to convert it to less rich format as long as possible. But you don't need to produce special objects, you can just log strings as the most basic thing as well (because string is an object). So it is up to you if you write those objects as strings to a file, if you generate a hierarchical format like json and transmit it via tcp or you keep the objects in the image having a http handler where you can specify a query for the objects and format to be returned. We just want it to be small and extensible. Using objects in the core is the best way to do it. Because then it is up to you if you want to log string messages or a whole stack.

Thanks for the clarification Norbert. Though it could seem I want to kill your efforts, I really want you succeed with your projects. Today, I see your logging objects as unnecessary entities for my requirements. Unfortunately it seems nobody had enough time to benchmark and compare loggers, if that's the case we wouldn't had this thread. Is the sense that tools and projects should be promoted with more humility in this community. If people chose other tools doesn't mean they don't get it right nor they fail miserably. Hernán

Norbert

...

We'll keep on trying to explain and to explore the possibilities.

...

Log objects are much more powerful, less expensive than you think, and

backwards compatible with textual output.

...

If I have to use a bayesian model with markov chains, which could take

up to 1 million of samplings only to begin stabilization, and I want to log them... do you expect to have every entry as a Log object... right? That would be less expensive in terms of memory and speed than flushing to a text file?

That would only be true if you kept them all in memory. That is an orthogonal point.

So you didn't answered my questions. Do I have to create 1 million of log objects and that is less expensive than flushing them as they occur into a file? What's the point of creating objects if I don't need them?

...

<sorry-for-the-sarcasm>A String instance is an object too, composing/formatting a log message is a computation too, similar to serialising an object in some format.</sorry-for-the-sarcasm>

I see. In the end we (still) read Strings, whatever conceptualization is above them.

...

...

Logging is deceptively simple, managing megabytes of log files is a PITA. Structure, classification, intelligence, behaviour are the answers.

BTW, it is not that you are not allowed to log some simple things to the Transcript, it is that it does not really scale, conceptually.

In my system I am not logging to the Transcript (which obviously does not scale).

I was using the Transcript as an example of a text based logging facility. I meant that it doesn't scale conceptually, it becomes one big mess of dozens of log files that helps very little.

Look, you don't have to use any form of Object Logging if you fail to see the point, there is nothing wrong with logging to a file. I was just trying to explain.

The point is isn't just me. I haven't seen any intensive computation software which creates log objects for self-exploration. And if you find one, I could cite 100 of them which does "String logging", and they just work, and people using it is not stupid, they know they have limitations, it happens that they prioritize other requirements.

...

We have dozens of servers producing many, many log files, most of them produced by log4j, writing more than 1Gb a day. 90% of that is a waste, but it is never enough. And it is a big mess. Even our Java developers say so. And what are they looking at ? Right, storing JSON log objects in searchable databases. (LogStash, ElasticSearch, blabla, ..).

It is weird to read sometimes how Java developers have it so wrong, and sometimes they are visionary souls which drive us into the future.

Hernán

Log4J does do Event object logging by default. I see it may be relevant to roll it into log4s also. I remember Toothpick probably did that On Tue, Jan 13, 2015 at 11:30 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

So if Object logging is implementing #asLog and #logClass methods maybe I should add them so Log4s can log "Object" :). But seriously, such comparison is not fair. Let's compare real logging features: priorities, levels, appenders, output destinations, hierarchical logging, formatting layout, etc. Many of them are implemented in Log4s.

Cheers,

Hernán

2015-01-12 14:56 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

...

TIA -- Regards,

Alain

Doru,

Am 13.01.2015 um 10:54 schrieb Tudor Girba <tudor@tudorgirba.com>:

Hi,

Log4j types of logging systems do a reasonable job at capturing log entries, but they are poor at helping the human make sense of the produced log. Of course, they do attempt, but given that they see the log as a long string, there is a significant mechanism built around formatting.

This is broken because it implies that the human should read the log as a way of understanding it. This works fine for a couple of lines but fails miserably for even Mb logs (not to mention larger logs). One could tweak the formatting to make it machine readable, but in practice all logs I have seen in enterprise systems are mostly unstructured and one has to spend great effort to extract the useful information out of them. For example, I use the information in logs to understand performance and diagnose errors, and I am using automatic tools for that. But, to build those tools, the hardest part is to parse the log. That cost should not exist.

The object as an item of logging preserves the original information so that the machine can read it. String is just a way of serializing an object, and we should only use it only in this way. Formatting still has a place, but it should be clear that it is just one tool - most often the least useful one. That is the reason why in Beacon, there is no formatting in the core.

Priority is another thing I find rather artificial. For example, developers should mark as DEBUG things that are not useful in production, but in the end, they mark as DEBUG things that are too large to have in production all the time. The problem with this is that sometimes you really want to have access to a very specific event regardless of "priority". So, a better way of looking at filtering is by allowing the developer to cherry pick any event at any given moment.

We definitely have to get good at the backend, but logging objects should provide a significant advantage.

will you attend pharodays? Maybe this would be a good target to sit together and merge SystemLogger and Beacon. Norbert

On Tue, Jan 13, 2015 at 10:05 AM, S Krish <krishnamachari.sudhakar@gmail.com <mailto:krishnamachari.sudhakar@gmail.com>> wrote:

Log4J does do Event object logging by default. I see it may be relevant to roll it into log4s also. I remember Toothpick probably did that

On Tue, Jan 13, 2015 at 11:30 AM, Hernán Morales Durand <hernan.morales@gmail.com <mailto:hernan.morales@gmail.com>> wrote: First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j <http://en.wikipedia.org/wiki/Log4j>

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

So if Object logging is implementing #asLog and #logClass methods maybe I should add them so Log4s can log "Object" :). But seriously, such comparison is not fair. Let's compare real logging features: priorities, levels, appenders, output destinations, hierarchical logging, formatting layout, etc. Many of them are implemented in Log4s.

Cheers,

Hernán

2015-01-12 14:56 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu <mailto:sven@stfx.eu>>:

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki <http://ss3.gemstone.com/ss/Log4s.html/Wiki>

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

...

TIA -- Regards,

Alain

-- www.tudorgirba.com <http://www.tudorgirba.com/>

"Every thing has its own flow"

Am 13.01.2015 um 13:18 schrieb stepharo <stepharo@free.fr>:

Norbert

we could read beacon and give a try.

Ok, that would be nice! Norbert

Its a matter of viewpoint here.. while I would agree that the needs outlined in your note are valid. Log4J does a good job of what I would consider the laymen spirit of logging production , UAT ( at client site ) with real data and workflow textual logs. The default event logging done captures the specific of class, line number, timestamp, method et als. * the rendered string that is 90% of the logging use is formatted string to a file / channel with options of logging the event object to DB / MQ et als * the events logged do have a fair amount of information that can be used for automation tooling around it * the rendered string too can be usefully queried for much information * the 7 levels of logging of info, trace, debug, warn, error allows for optimization in production run to keep it at "warn" and reduce the performance. The purpose is to reduce the number of logging events routed to an appender, performance is impacted if it goes 10 / 100 times larger.. While UAT it is typical to work with debug to get more information of the replicated issue. The 80-20 rule in typical enterprise is to capture the log trace of the exception / error and use that to fix a bug, mostly trivial use of the logging information. Rarely if ever have I seen the log is used for deep dive analysis for improving the product, though it certainly can be and is done. So in log4j it is upto the configuration that allows one to log event object or formatted strings and to have finer control over amount of log events channelled and to any number / types of appenders eventually as per customer convenience. Given there may be 100's of customers with varied needs, it is ok to have that flexibility. I do find log4j as such a very enterprise like framework, which may seem to posses gaps you point out, but for almost 99% of its usage it fits in very well. On Tue, Jan 13, 2015 at 3:24 PM, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi,

Log4j types of logging systems do a reasonable job at capturing log entries, but they are poor at helping the human make sense of the produced log. Of course, they do attempt, but given that they see the log as a long string, there is a significant mechanism built around formatting.

This is broken because it implies that the human should read the log as a way of understanding it. This works fine for a couple of lines but fails miserably for even Mb logs (not to mention larger logs). One could tweak the formatting to make it machine readable, but in practice all logs I have seen in enterprise systems are mostly unstructured and one has to spend great effort to extract the useful information out of them. For example, I use the information in logs to understand performance and diagnose errors, and I am using automatic tools for that. But, to build those tools, the hardest part is to parse the log. That cost should not exist.

The object as an item of logging preserves the original information so that the machine can read it. String is just a way of serializing an object, and we should only use it only in this way. Formatting still has a place, but it should be clear that it is just one tool - most often the least useful one. That is the reason why in Beacon, there is no formatting in the core.

Priority is another thing I find rather artificial. For example, developers should mark as DEBUG things that are not useful in production, but in the end, they mark as DEBUG things that are too large to have in production all the time. The problem with this is that sometimes you really want to have access to a very specific event regardless of "priority". So, a better way of looking at filtering is by allowing the developer to cherry pick any event at any given moment.

We definitely have to get good at the backend, but logging objects should provide a significant advantage.

Cheers, Doru

On Tue, Jan 13, 2015 at 10:05 AM, S Krish < krishnamachari.sudhakar@gmail.com> wrote:

...

Log4J does do Event object logging by default. I see it may be relevant to roll it into log4s also. I remember Toothpick probably did that

On Tue, Jan 13, 2015 at 11:30 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

So if Object logging is implementing #asLog and #logClass methods maybe I should add them so Log4s can log "Object" :). But seriously, such comparison is not fair. Let's compare real logging features: priorities, levels, appenders, output destinations, hierarchical logging, formatting layout, etc. Many of them are implemented in Log4s.

Cheers,

Hernán

2015-01-12 14:56 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

...

TIA -- Regards,

Alain

-- www.tudorgirba.com

"Every thing has its own flow"

On Tue, Jan 13, 2015 at 2:00 PM, Atlas <erde@chello.at> wrote:

It's all a matter of viewpoint, that's why we should include as many of them possible :)

But, even though the answer was not directed as me, I'd like to step in:

...

Log4J does a good job of what I would consider the laymen spirit of logging production > he default event logging done captures the specific of class, line number, timestamp, method et als.

A dog's breakfast actually.

Strings lack self-description and this causes many unnecessary architectural problems.

As Doru already pointed out, you have to write parsers and analysers to interpret the output. And since might be many different formats for the same thing (dialects), you lose uniformity and have to handle all corner cases. Worse yet, there is the implicit assumption that the software (or human) inspecting the log has to understand all formats, i.e. has to have knowledge about: What the names of the classes and methods mean, which subsystems (MQ, databases) are in use and how their communication protocols look like etc. And that does not factor in possible format changes or versions of subsystems.

It's hard to write generic browsers and analysers (the automated tooling Doru pointed out) that way. Also, there is the problem of reusing these parsers and analyers and code duplication, maintenance issues etc.

With that said, ultimately, it depends on the use cases.* For a fixed set * of inspection use cases, this logging strategy might work fine. But it certainly lacks the real flexibility to make a system more accessible and to adopt the generated information to the person and it's needs and viewpoints. You cannot address that easily with Log4j (or any logging facility for that matter). As for Log4j and its 7 levels of logging: Why not 8, 9 or a 100? What does INFO really mean? To whom? It all depends on the context.

There are many, many usage scenarios and even more might pop up with more accessible and comprehensible logging schemes.

We need to go far beyond logging.

Sure but with tools like http://www.fluentd.org/, and http://riemann.io/concepts.html, we are really blown out of the water with those logs we have now. Text is here to stay once we get into the sysadmin realm. People do not want to deal with code internals when they are doing the piping. Just my experience dealing with such admins working with a lot of different systems. We cannot ask them to become polyglot coders as their hands are full already. Phil

-- View this message in context: http://forum.world.st/What-Logging-framework-for-Pharo-tp4799073p4799254.htm... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

-- --- Philippe Back Visible Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be | Web: http://philippeback.eu Blog: http://philippeback.be | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium Pharo Consortium Member - http://consortium.pharo.org/ Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller

Am 13.01.2015 um 14:08 schrieb phil@highoctane.be:

On Tue, Jan 13, 2015 at 2:00 PM, Atlas <erde@chello.at <mailto:erde@chello.at>> wrote: It's all a matter of viewpoint, that's why we should include as many of them possible :)

But, even though the answer was not directed as me, I'd like to step in:

...

Log4J does a good job of what I would consider the laymen spirit of logging production > he default event logging done captures the specific of class, line number, timestamp, method et als.

A dog's breakfast actually.

Strings lack self-description and this causes many unnecessary architectural problems.

As Doru already pointed out, you have to write parsers and analysers to interpret the output. And since might be many different formats for the same thing (dialects), you lose uniformity and have to handle all corner cases. Worse yet, there is the implicit assumption that the software (or human) inspecting the log has to understand all formats, i.e. has to have knowledge about: What the names of the classes and methods mean, which subsystems (MQ, databases) are in use and how their communication protocols look like etc. And that does not factor in possible format changes or versions of subsystems.

It's hard to write generic browsers and analysers (the automated tooling Doru pointed out) that way. Also, there is the problem of reusing these parsers and analyers and code duplication, maintenance issues etc.

With that said, ultimately, it depends on the use cases.* For a fixed set * of inspection use cases, this logging strategy might work fine. But it certainly lacks the real flexibility to make a system more accessible and to adopt the generated information to the person and it's needs and viewpoints. You cannot address that easily with Log4j (or any logging facility for that matter). As for Log4j and its 7 levels of logging: Why not 8, 9 or a 100? What does INFO really mean? To whom? It all depends on the context.

There are many, many usage scenarios and even more might pop up with more accessible and comprehensible logging schemes.

We need to go far beyond logging.

Sure but with tools like http://www.fluentd.org/ <http://www.fluentd.org/>, and http://riemann.io/concepts.html <http://riemann.io/concepts.html>, we are really blown out of the water with those logs we have now.

What do you mean? What is (besides performance) the special thing to do? I had to love looking at the riemann site because the stream filter example remembers me about something I have done myself root@2d-misc:/opt/application/mc-partner# cat launch.d/95_enable-callback ZnEasy post: 'http://192.168.10.47:3000/callbacks' data: (ZnEntity with: (NeoJSONWriter toString: { 'name' -> 'multicity partner'. 'url' -> 'http://192.168.10.43:7850/pushtrigger'. 'filter' -> [ :event | ((event origin = 'deviceManagement') & (#( 'MessageSent' 'MessageNotSent') includes: event type)) and: [ ((event message at: 'application') at: 'name') = 'multicity' ] ] sourceNode formattedCode } asDictionary)) It registers a pharo instance at my own event server for some events. The block is transferred as code and stored in the database on the event server. Whenever the block matches the callback url is triggered with the event data as json.

Text is here to stay once we get into the sysadmin realm. People do not want to deal with code internals when they are doing the piping. Just my experience dealing with such admins working with a lot of different systems. We cannot ask them to become polyglot coders as their hands are full already.

The shell is line based and the odds are quite low this will change. That doesn't mean all of the information needs to be line based just the results. I'm using jq [1] since a while now and there you work on hierarchical json data to produce line based results. So if you would query from a shell to the logging engine you could still get line based results. One example is how I delete older backups in elasticsearch #!/bin/sh BACKUP_URL="http://localhost:9200/_snapshot/backup" SNAPSHOTS=`curl -s "$BACKUP_URL/_all" | jq ".snapshots[].snapshot" | sed -e 's/"//g' | sort -u | head -n-14` for snapshot in $SNAPSHOTS; do curl -XDELETE "$BACKUP_URL/$snapshot" > /dev/null 2>&1 done Norbert [1] http://stedolan.github.io/jq/ <http://stedolan.github.io/jq/>

-- --- Philippe Back Visible Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be <mailto:Mail%3Aphil@highoctane.be> | Web: http://philippeback.eu <http://philippeback.eu/> Blog: http://philippeback.be <http://philippeback.be/> | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos <http://www.youtube.com/user/philippeback/videos>

High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium

Pharo Consortium Member - http://consortium.pharo.org/ <http://consortium.pharo.org/> Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com <http://spamcast.libsyn.com/> Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller

Phil as usually (sadly) with the loggin framework rant people are confusing logging in the image and outside. WE MUST HAVE A LOGGING infrastructure and it will not be 'J4log' whatever. I want to log compiler error and be able to click on them and jump in the debugger. Right now I can only see errors if I open a transcript and look for strings!!! Now if we would all use the time to write mails to write code....

Le 13/01/2015 17:39, stepharo a écrit :

Sorry to look harsh but you often kill my fun and energy with strange expectations. We have one and half full time engineer working for Pharo now. And we are pretty successful. Now imagine how pharo would look like if you would participate for real each at its own degree

Stef, I'm really sorry if my question and the answers, opinions and expectations of this thread made you feel bad. It was a not a strange expectation or a general discussion on logging at all, but a true question about existing logging facility in Pharo that I do not know, and the fact is there was Beacon, SystemLogger and ZnLogger, and albeit it may look depressing to you, this thread has some very interesting pointers to me. I think everybody's answer has some valid arguments from it's own point of view, none of them is really controversial, and the merging you talked about should settle everyone's expectations right. For me, I was looking for a logging tool for a project I (re)started some time ago and broke recently, trying to refactor it. More precisely I don't know why a client socket connection is closing, here I think Beacon will really help me on that, ZnLogger or SystemLogger could too. And even if the sl4s package looks good, it would help less for debugging (I 'm not saying it is not good), inspecting logged objects will be really a big plus. I 'll do some experiments with beacon and try to make something like "sensors" (signalers) plugged in the code (like sql server extended events framework plugged in sql server code) to replace a BlackBox I did in my system that is really weak. I would be very interested in following, participating or helping (docs, or whatever useful, eventually code or tests with my very limited skills and with some guidance) on this logging subject if needed, but I do not feel skilled enough on the Pharo part to solve most (if not any) fogbugz cases I saw, I would certainly only do a mess (I'm sure I'm not the only one in that case, the sep is very high between using and solving cases). So to me just using it for now is a good way to get deeper into it and probably one day be skilled enough to help ... :) And I'm very sorry too that I cannot go to Pharo day, I would have loved to see presentations , meet some people, talk about it (logging tools or whatever) but I definitely can't . too bad :(

Stef

-- Regards, Alain

Hi guys As a general point of view, often the discussions in this mailing-list are comparing Pharo with projects where - people really produces open-source software and share it for real (by opposition as talking nicely close the fireplace with friends). - it often sounds like we should stop doing things just because people already did something else. - if I would really read these discussions I would drop everything and go do something else. So as a conclusion: - ask yourself what you did for Pharo recently (not just to get money for you) but for the community in terms of open-source software, documentation, bug fixes, blog entries.... - what will you do to change the situation by helping Pharo? Not just using it! Sorry to look harsh but you often kill my fun and energy with strange expectations. We have one and half full time engineer working for Pharo now. And we are pretty successful. Now imagine how pharo would look like if you would participate for real each at its own degree Stef

hernanmd wrote

That depends in what you want to interpret. A log entry format is something you invent for your system. For log analysis you don't usually write a parser, you can just grep over the output to see why things didn't worked. Or you may use one of the tools for output analysis.

Provided that the parts of the system *I* am interested in have been enriched with logging code. And provided the log messages are informative -- and that's a highly subjective matter. Also, different users and developers might have different opinions as to which parts of the system to instrument. (Of course, I could use an aspect weaver to do instrumentation, but that's a rather indirect way to address the problem) hernanmd wrote

Exactly, you define the meaning of INFO for your system. There should not be just one conception of INFO.

I might define what INFO means for my system from *my *point of view, but there might be many other different viewpoints. How can these be incorporated, if the decisions what and how to log and how are determined by someone else? hernanmd wrote

The point is isn't just me. I haven't seen any intensive computation software which creates log objects for self-exploration. And if you find one, I could cite 100 of them which does "String logging", and they just work, and people using it is not stupid, they know they have limitations, it happens that they prioritize other requirements.

They should prioritize other requirements, yes. There should be no need to write (much) logging code. In an object-oriented system, I am interested about the information flow / messages between objects. With hierarchical decomposition, I can zoom in and out, see the messages and their effects, which in my opinion, is a more holistic perspective on system analysis. With grep, I would have to reconstruct this information (caller graph, state changes etc.) from the logs or use a tool which only works if the log messages follow a specific format (and these formats might change). Of course, I am not saying that string messages are useless and cannot be informative, but they cover only one aspect of the problem. hernanmd wrote

I see. In the end we (still) read Strings, whatever conceptualization is above them.

This rests on the assumption that the string and/or strings from related log messages carry enough information (and additional context) to infer the concepts. -- View this message in context: http://forum.world.st/What-Logging-framework-for-Pharo-tp4799073p4799398.htm... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Hernan,

Am 13.01.2015 um 07:00 schrieb Hernán Morales Durand <hernan.morales@gmail.com>:

First of all, I am not the developer of Log4s, but I have been using it for a while and I see no significant gain switching to SystemLogger or ZnLogEvent. As Alain said, it is based in Log4j, which has his own Wikipedia page: http://en.wikipedia.org/wiki/Log4j <http://en.wikipedia.org/wiki/Log4j>

I don't think there is too much sense in having "Log Objects" around, where probably I want to tail a daily rolled log from a headless remote image. I read the documentation of SystemLogger hoping to find a mechansim which dynamically selects compression strategies based in what's logged, or some other advanced feature, but I didn't found any :(

if your only concern is to have syslog style logging (maybe even just file based) than all of what we've done is not of any value. If you peek at current logging trends you will see things like logstash et al. In those log collectors they provide huge parsing libraries just to be able to parse every available log format out there in order to produce objects from it that you can store in e.g. elasticsearch. Especially if you have GBs of logging information it makes sense to be able to query the log information for things you are interested in. So think about moving the DEBUG quality from the logger to the viewer of the log. I personally do enjoy using SystemLogger. Instead of a huge logging framework and the need to parse them back into a system like logstash I did a shortcut by emitting the log objects and send them as json into a elasticsearch database. From there I have queries for finding problems and second I produce stats from it that give me some insight of the health of the system. To me there a plenty of advantages to do it that way. From a simple (well :) ) query I can see something like this. This things are important for me. Norbert

So if Object logging is implementing #asLog and #logClass methods maybe I should add them so Log4s can log "Object" :). But seriously, such comparison is not fair. Let's compare real logging features: priorities, levels, appenders, output destinations, hierarchical logging, formatting layout, etc. Many of them are implemented in Log4s.

Cheers,

Hernán

2015-01-12 14:56 GMT-03:00 Sven Van Caekenberghe <sven@stfx.eu <mailto:sven@stfx.eu>>:

...

On 12 Jan 2015, at 18:27, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

hi all,

Googling a bit for a logging framework in Pharo, I found sl4j, which sounds familiar to me. http://ss3.gemstone.com/ss/Log4s.html/Wiki <http://ss3.gemstone.com/ss/Log4s.html/Wiki>

I suppose the API is the same as the java package, as claimed by the project page. I wonder if anybody is using it , or has tried it and have an advice on that package ? (I saw there was no test and no comments ... :( )

I think it could be great for Pharo to have a standard package like that, a logging facility is mandatory for most -if not all- applications

Any advice or pointers ?

Object logging instead of String logging is the way to go. SystemLogger, Beacon, Zinc's ZnLogEvent are examples. Search the mailing list for past discussions.

...

TIA -- Regards,

Alain

Le 13/01/2015 07:03, Hernán Morales Durand a écrit :

Hi Alain,

You have a Log4s podcast here

http://www.jarober.com/blog/blogView?entry=3509625595

and documentation here

http://www.instantiations.com/docs/852/wwhelp/wwhimpl/js/html/wwhelp.htm#hre...

I have a real web application using Log4s here: http://www.smalltalkhub.com/#!/~hernan/IGEVET and a repository where I try to post fixes or any improvement: http://www.smalltalkhub.com/#!/~hernan/Log4s (don't hesitate to ask me for access if you want to contribute)

Hi Hernan, Thank you for the pointers. I think today there are two very different use cases of a logging system : in production, where you need a robust, efficient and if possible standard (for example one that logs to system logs in a standard way, why appenders are importants IMHO), and in development, where you need to log information about your program to help you in your developement. I also asked that myself, what makes logging objects different from "standard" logging systems? May be as you say, is not so different, and may be also with growth of the "platform as a service" trend, administrators and operation systems engineers will become developers (or builders) of their own production platform, some kind of new developers? And using standard logger with objects, appenders could make the difference What I also like in the Beacon approach is that it is event driven. Having sensors plugged in the code looks very attractive.

Let us know if you have specific questions.

Cheers,

Hernán

-- Regards, Alain