Am 13.01.2015 um 14:08 schrieb phil@highoctane.be:

On Tue, Jan 13, 2015 at 2:00 PM, Atlas <erde@chello.at> wrote:
It's all a matter of viewpoint, that's why we should include as many of them
possible :)

But, even though the answer was not directed as me, I'd like to step in:

> Log4J does a good job of what I would consider the laymen spirit of
> logging production > he default event logging done captures the specific
> of class, line number, timestamp, method et als.

A dog's breakfast actually.

Strings lack self-description and this causes many unnecessary architectural
problems.

As Doru already pointed out, you have to write parsers and analysers to
interpret the output. And since might be many different formats for the same
thing (dialects), you lose uniformity and have to handle all corner cases.
Worse yet, there is the implicit assumption that the software (or human)
inspecting the log has to understand all formats, i.e. has to have knowledge
about:
What the names of the classes and methods mean, which subsystems (MQ,
databases) are in use and how their communication protocols look like etc.
And that does not factor in possible format changes or versions of
subsystems.

It's hard to write generic browsers and analysers (the automated tooling
Doru pointed out) that way.
Also, there is the problem of reusing these parsers and analyers and code
duplication, maintenance issues etc.

With that said, ultimately, it depends on the use cases.* For a fixed set *
of inspection use cases, this logging strategy might work fine. But it
certainly lacks the real flexibility to make a system more accessible and to
adopt the generated information to the person and it's needs and viewpoints.
You cannot address that easily with Log4j (or any logging facility for that
matter). As for Log4j and its 7 levels of logging: Why not 8, 9 or a 100?
What does INFO really mean? To whom? It all depends on the context.

There are many, many usage scenarios and even more might pop up with more
accessible and comprehensible logging schemes.

We need to go far beyond logging.


Sure but with tools like http://www.fluentd.org/, and http://riemann.io/concepts.html, we are really blown out of the water with those logs we have now.

What do you mean? What is (besides performance) the special thing to do? I had to love looking at the riemann site because the stream filter example remembers me about something I have done myself

root@2d-misc:/opt/application/mc-partner# cat launch.d/95_enable-callback
ZnEasy
   post: 'http://192.168.10.47:3000/callbacks'
   data: (ZnEntity
      with: (NeoJSONWriter toString: {
         'name'   -> 'multicity partner'.
         'url'    -> 'http://192.168.10.43:7850/pushtrigger'.
         'filter' -> [ :event |
            ((event origin = 'deviceManagement') & (#( 'MessageSent' 'MessageNotSent') includes: event type)) and: [
               ((event message at: 'application') at: 'name') = 'multicity' ] ] sourceNode formattedCode } asDictionary))

It registers a pharo instance at my own event server for some events. The block is transferred as code and stored in the database on the event server. Whenever the block matches the callback url is triggered with the event data as json. 



Text is here to stay once we get into the sysadmin realm. People do not want to deal with code internals when they are doing the piping.
Just my experience dealing with such admins working with a lot of different systems. We cannot ask them to become polyglot coders as their hands are full already.

The shell is line based and the odds are quite low this will change. That doesn't mean all of the information needs to be line based just the results. I'm using jq [1] since a while now and there you work on hierarchical json data to produce line based results. So if you would query from a shell to the logging engine you could still get line based results. One example is how I delete older backups in elasticsearch

#!/bin/sh

BACKUP_URL="http://localhost:9200/_snapshot/backup"
SNAPSHOTS=`curl -s "$BACKUP_URL/_all" | jq ".snapshots[].snapshot" | sed -e 's/"//g' | sort -u | head -n-14`

for snapshot in $SNAPSHOTS; do
   curl -XDELETE "$BACKUP_URL/$snapshot" > /dev/null 2>&1
done

Norbert

[1] http://stedolan.github.io/jq/

-- 
---
Philippe Back
Visible Performance Improvements
Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027
Blog: http://philippeback.be | Twitter: @philippeback

High Octane SPRL
rue cour Boisacq 101 | 1301 Bierges | Belgium

Pharo Consortium Member - http://consortium.pharo.org/
Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com
Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller