Beacon for Pharo 7
Hi. I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions. But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning. What you think to create new org "pharo-logging" for this purpose? It looks like current approach to manage common projects (pharo-vcs, pharo-ide). Another question is what we want: copy of BeaconCore into Pharo or load it separately like Iceberg and Calypso? We can start with loading then we will decide. Best regards, Denis
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos? Stephan
2018-05-25 17:21 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos?
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
Stephan
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that? Stephan
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do
not
want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
Stephan
+1 Doru
On May 26, 2018, at 7:44 AM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>: Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
Stephan
-- www.tudorgirba.com www.feenk.com "Not knowing how to do something is not an argument for how it cannot be done."
Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do
not
want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
You do not have a use case for separate repos, but one for duplicates of the same repo. You need that anyhow because your images depend on different branches and versions. You might want to solve that by having only one image responsible for source code management, and all others connecting to that instead of using libgit directly. Stephan
2018-05-26 9:30 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do
not
want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
You do not have a use case for separate repos, but one for duplicates of the same repo.
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
You need that anyhow because your images depend on different branches and versions. You might want to solve that by having only one image responsible for source code management, and all others connecting to that instead of using libgit directly.
Stephan
Hi all, Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere". If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7? Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them? If I put this side by side with the versionning questions above, this ones have more weight to me... Cheers, Guille
Hi, I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately. As for Beacon, I am using it since two years, and it is as stable as software can get. The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism. What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else. And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options. Cheers, Doru
On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
-- www.tudorgirba.com www.feenk.com "Some battles are better lost than fought."
+1 Norbert
Am 26.05.2018 um 10:57 schrieb Tudor Girba <tudor@tudorgirba.com>:
Hi,
I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately. As for Beacon, I am using it since two years, and it is as stable as software can get.
The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism.
What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else.
And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options.
Cheers, Doru
On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
-- www.tudorgirba.com www.feenk.com
"Some battles are better lost than fought."
Hi, On Sat, May 26, 2018 at 10:57 AM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately.
Really? Since years we are chasing bugs and memory leaks created by glamour and spotter... And what I see is a lot of effort on debuggging those from the people in RMod to fix those. Is that what you call stable? What I'd like is that the effort of maintaining Glamour or even the Glamorous Toolkit is shared, because you have the knowledge to debug it while me, I have to develop that knowledge... Right now in a vanilla Pharo 6 I've been using for some hours I do GTInspector allInstances. And I have three instances, while only one inspector is open.
As for Beacon, I am using it since two years, and it is as stable as software can get.
The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism.
No, the problem will be there even without Announcements. Beacon relies on a global registration mechanism, otherwise it doesn't make sense. If I register a logger in Beacon, that logger will be hanging around for ever. So we need tools to handle those. Announcements have other problems on their own, as we were talking with Pablo the other day. They are not reentrant (what happens if an announcement is raised during the treatment of an announcement? Does it loop?), they block the caller until all announcements are processed (not so decoupled, huh :P), right now without looking at the code it's not clear to me if they can handle the subscription of arbitrary instances instead of just Announcement (sub)classes (announcer on: arbitraryObject send: ...). Not even mentioning the weak/ephemeron mess :).
What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else.
I want to be concrete, please. I do not want the Kernel to rely on Beacon nor announcements to work. To me the idea of "all transcript usages should be rewritten" is way too abstract. What is the list of packages that should be rewritten in Pharo? I want to be concrete. Maybe some Transcript usages need to be removed, maybe some should be replaced. The solution "We introduce it and eventually we replace users" will decay into "We introduce it and never replace the users" unless there is an active effort to do it, we have all seen it. So, who is willing to do that effort? Because, integrating Beacon into Pharo has also its drawbacks for users: People will not be able to freely install/upgrade the version of Beacon they want without putting the system in risk. On the other side, keeping it as an external library, people can have as a simple dependency whatever they need for their application. Also, didn't we agree that in Pharo7 we should try to remove something for each thing that is added?
And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options.
So when people say "We shoud integrate Beacon", what exactly are we talking about? - What are the packages/features included? What loggers? What kind of events/logs? - What are the good/bad practices of using Beacon?
Cheers, Doru
On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
-- www.tudorgirba.com www.feenk.com
"Some battles are better lost than fought."
-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
2018-05-26 12:26 GMT+03:00 Guillermo Polito <guillermopolito@gmail.com>:
Also, didn't we agree that in Pharo7 we should try to remove something for each thing that is added?
But it is not the case. We do not have logging system in Pharo. So Beacon will not replace anything. It is new feature
Hi, Yes, Glamour is stable since years. Yes, we did have some leak issues that appeared along the way. Last year, we discovered that a memory leak came from Rubric (specifically there was a global variable in it). People saw inspectors laying around, so naturally, the issue was seen as being due to GT. And now there is another one, which is still under investigation. For me, the main issue is not the memory leak, but the fact that we have no tool to reason about them. I am not being vague about what needs to change in the image, but I think there are people that are better equipped for answering that. For example, Denis provided a concrete answer that the Kernel should not use Beacon. But, to facilitate the conversation: (SystemNavigation default allGlobalRefsOn: #Transcript) reject: [ :each | each package name = #Kernel ] gets us 124 methods, and (((SystemNavigation default allGlobalRefsOn: #Transcript) reject: [ :each | each package name = #Kernel ]) groupedBy: [:each | each package name ]) returns 58 packages. When we talk about integrating Beacon, we talk at most about: spec package: #'Beacon-Core'; package: #'Beacon-Core-GT' with: [ spec requires: #('Beacon-Core' ). ]; package: #'Beacon-Core-Tests' with: [ spec requires: #('Beacon-Core' ). ]; package: #'Beacon-Syslog'; package: #'Beacon-Fileâ; package: #'Beacon-Zinc'. where, Beacon-Syslog and Beacon-File must not necessarily be in the image. I do agree with the idea of requiring something to be removed when we add something new. However, in this case, Beacon adds a new capability, and I do not see how this rule would apply. Cheers, Doru
On May 26, 2018, at 11:26 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi,
On Sat, May 26, 2018 at 10:57 AM, Tudor Girba <tudor@tudorgirba.com> wrote: Hi,
I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately.
Really? Since years we are chasing bugs and memory leaks created by glamour and spotter... And what I see is a lot of effort on debuggging those from the people in RMod to fix those. Is that what you call stable? What I'd like is that the effort of maintaining Glamour or even the Glamorous Toolkit is shared, because you have the knowledge to debug it while me, I have to develop that knowledge...
Right now in a vanilla Pharo 6 I've been using for some hours I do
GTInspector allInstances.
And I have three instances, while only one inspector is open.
As for Beacon, I am using it since two years, and it is as stable as software can get.
The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism.
No, the problem will be there even without Announcements. Beacon relies on a global registration mechanism, otherwise it doesn't make sense. If I register a logger in Beacon, that logger will be hanging around for ever. So we need tools to handle those.
Announcements have other problems on their own, as we were talking with Pablo the other day. They are not reentrant (what happens if an announcement is raised during the treatment of an announcement? Does it loop?), they block the caller until all announcements are processed (not so decoupled, huh :P), right now without looking at the code it's not clear to me if they can handle the subscription of arbitrary instances instead of just Announcement (sub)classes (announcer on: arbitraryObject send: ...). Not even mentioning the weak/ephemeron mess :).
What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else.
I want to be concrete, please. I do not want the Kernel to rely on Beacon nor announcements to work.
To me the idea of "all transcript usages should be rewritten" is way too abstract. What is the list of packages that should be rewritten in Pharo? I want to be concrete. Maybe some Transcript usages need to be removed, maybe some should be replaced. The solution "We introduce it and eventually we replace users" will decay into "We introduce it and never replace the users" unless there is an active effort to do it, we have all seen it.
So, who is willing to do that effort?
Because, integrating Beacon into Pharo has also its drawbacks for users: People will not be able to freely install/upgrade the version of Beacon they want without putting the system in risk. On the other side, keeping it as an external library, people can have as a simple dependency whatever they need for their application.
Also, didn't we agree that in Pharo7 we should try to remove something for each thing that is added?
And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options.
So when people say "We shoud integrate Beacon", what exactly are we talking about? - What are the packages/features included? What loggers? What kind of events/logs? - What are the good/bad practices of using Beacon?
Cheers, Doru
On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
-- www.tudorgirba.com www.feenk.com
"Some battles are better lost than fought."
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
-- www.tudorgirba.com www.feenk.com "It's not how it is, it is how we see it."
On 05/26/2018 05:21 AM, Tudor Girba wrote:
Yes, Glamour is stable since years. Yes, we did have some leak issues that appeared along the way. Last year, we discovered that a memory leak came from Rubric (specifically there was a global variable in it). People saw inspectors laying around, so naturally, the issue was seen as being due to GT. And now there is another one, which is still under investigation. For me, the main issue is not the memory leak, but the fact that we have no tool to reason about them.
I found a memory leak in GLMHintableActionButtonBrick. It has an asyncTask class instance variable that holds a GLMAsyncTask. After running for a while -- opening and closing inspectors and other tools, I closed several windows, did a GC and expected to not have any AST nodes hanging around. However, I found several, so I used my ReferenceFinder tool to find a reference path. That path went through the GLMHintableActionButtonBrick to a GLMAsyncTask to a block that holds a reference to a GLMHintableActionButtonBrick instance. Following references out of the GLMHintableActionButtonBrick instance eventually leads to a NodeMorph which has a NodeModel that contained my AST node. I'm running a Pharo 7 build from a few weeks ago (Pharo-7.0+alpha.build.843.sha.edceb623c1a3ac3358f7e46ca1048d455657e10c) so this may have been fixed. John Brant
On 2 Jun 2018, at 17:53, John Brant <brant@refactoryworkers.com> wrote:
On 05/26/2018 05:21 AM, Tudor Girba wrote:
Yes, Glamour is stable since years. Yes, we did have some leak issues that appeared along the way. Last year, we discovered that a memory leak came from Rubric (specifically there was a global variable in it). People saw inspectors laying around, so naturally, the issue was seen as being due to GT. And now there is another one, which is still under investigation. For me, the main issue is not the memory leak, but the fact that we have no tool to reason about them.
I found a memory leak in GLMHintableActionButtonBrick. It has an asyncTask class instance variable that holds a GLMAsyncTask. After running for a while -- opening and closing inspectors and other tools, I closed several windows, did a GC and expected to not have any AST nodes hanging around. However, I found several, so I used my ReferenceFinder tool to find a reference path. That path went through the GLMHintableActionButtonBrick to a GLMAsyncTask to a block that holds a reference to a GLMHintableActionButtonBrick instance. Following references out of the GLMHintableActionButtonBrick instance eventually leads to a NodeMorph which has a NodeModel that contained my AST node.
Thanks, I have added it to the issue tracker to not lose it: https://pharo.fogbugz.com/f/cases/22163/Memory-Leak-GLMHintableActionButtonB...
To be honest you sound a bit harsh. And Iâm not sure I can remember that such kind of arguments have ever been exchanged before something was integrated. You want to have a concrete definition. But you just said you donât want the core the rely neither on Beacon nor on Announcements. So maybe it would be good to define what is in the core so we can understand? I hope we can agree that having a logging system is a good thing. And I hope Beacon is considered the best candidate for it. In order to make it so we designed the core to be minimal for easy integration. The only consumers included in the system are the memory logger and the transcript logger. So the dependencies are none because Transcript is here to stay for the time being. Transcript logger can also be the default fallback so the system appears like it is today. One of the biggest decisions might be what method names are allowed to be highjacked by a logging framework. Will there be something like Object>>#log: (#emit:) ??? If beacon is considered to be close to the core this will be ok. If it is considered just a logger to be loaded it is not. Designing something that makes beacon replacable will most likely exceed the code base beacon is. It is kept small for that reason. Regarding Announcements. Are Announcements to be removed from the system? If not it is fine Beacon is using them. Btw. the global usage of beacon is just one use case. And a needed one because you want to have it usable from a playground, too. So it is a good fallback. The problems of Announcements should be solved in Announcements not in Beacon. Or Announcements should be deprecated. So please define what is the core of pharo and we can talk about how Beacon is best integrated. Norbert
Am 26.05.2018 um 11:26 schrieb Guillermo Polito <guillermopolito@gmail.com>:
Hi,
On Sat, May 26, 2018 at 10:57 AM, Tudor Girba <tudor@tudorgirba.com> wrote: Hi,
I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately.
Really? Since years we are chasing bugs and memory leaks created by glamour and spotter... And what I see is a lot of effort on debuggging those from the people in RMod to fix those. Is that what you call stable? What I'd like is that the effort of maintaining Glamour or even the Glamorous Toolkit is shared, because you have the knowledge to debug it while me, I have to develop that knowledge...
Right now in a vanilla Pharo 6 I've been using for some hours I do
GTInspector allInstances.
And I have three instances, while only one inspector is open.
As for Beacon, I am using it since two years, and it is as stable as software can get.
The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism.
No, the problem will be there even without Announcements. Beacon relies on a global registration mechanism, otherwise it doesn't make sense. If I register a logger in Beacon, that logger will be hanging around for ever. So we need tools to handle those.
Announcements have other problems on their own, as we were talking with Pablo the other day. They are not reentrant (what happens if an announcement is raised during the treatment of an announcement? Does it loop?), they block the caller until all announcements are processed (not so decoupled, huh :P), right now without looking at the code it's not clear to me if they can handle the subscription of arbitrary instances instead of just Announcement (sub)classes (announcer on: arbitraryObject send: ...). Not even mentioning the weak/ephemeron mess :).
What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else.
I want to be concrete, please. I do not want the Kernel to rely on Beacon nor announcements to work.
To me the idea of "all transcript usages should be rewritten" is way too abstract. What is the list of packages that should be rewritten in Pharo? I want to be concrete. Maybe some Transcript usages need to be removed, maybe some should be replaced. The solution "We introduce it and eventually we replace users" will decay into "We introduce it and never replace the users" unless there is an active effort to do it, we have all seen it.
So, who is willing to do that effort?
Because, integrating Beacon into Pharo has also its drawbacks for users: People will not be able to freely install/upgrade the version of Beacon they want without putting the system in risk. On the other side, keeping it as an external library, people can have as a simple dependency whatever they need for their application.
Also, didn't we agree that in Pharo7 we should try to remove something for each thing that is added?
And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options.
So when people say "We shoud integrate Beacon", what exactly are we talking about? - What are the packages/features included? What loggers? What kind of events/logs? - What are the good/bad practices of using Beacon?
Cheers, Doru
On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito@gmail.com> wrote:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon? - And which ones do not? - **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
-- www.tudorgirba.com www.feenk.com
"Some battles are better lost than fought."
--
Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
Guillermo Polito wrote
Announcements have other problems on their own, as we were talking with Pablo the other day. They are not reentrant (what happens if an announcement is raised during the treatment of an announcement? Does it loop?)
What? The new event is delivered, of course. If the complaint is "When I write an infinite loop using announcements, it is an infinite loop", I don't know what to tell you... Guillermo Polito wrote
, they block the caller until all announcements are processed (not so decoupled, huh :P),
Avoiding subscribers who take too long to process the subscription, *should* be the responsibility of each subscriber. If you think announcement overhead is bad now, imagine what happens if you fork of a process for every delivery. Depending on priority, you'd still have a problem in that it either in effect will still block the sender, or not be processed at all until sender completes. Also, good luck debugging. Show me *one* other event registration framework in a green-threaded system that hasn't made the same tradeoff, and I'll buy you a beer. Guillermo Polito wrote
right now without looking at the code it's not clear to me if they can handle the subscription of arbitrary instances instead of just Announcement (sub)classes (announcer on: arbitraryObject send: ...).
The fact you can easily search for users of a given announcement in a standard browser is one of its big advantages compared to the seaside announcements, if you ask me. However, the only hard requirement should be to implement handlesAnnouncement: on whatever is passed to when: (though, there seems to have been a late addition of #prepareForDelivery, which should be handled by announcement initialization, not the delivery mechanism, at least for the use indicated by its comment...). Which still doesn't make it a good idea to implement on arbitrary objects (though it's implemented on Symbol for backwards compatibility with the old event registration mechanism). Guillermo Polito wrote
Not even mentioning the weak/ephemeron mess :).
This I can get behind though, the world would be a better place if everyone unregistered their subscriptions properly. ;) (and/or we had a working ephemeron implementation without the same self-referential pitfalls as the weak one) Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
Very good questions. 2018-05-26 10:00 GMT+03:00 Guillermo Polito <guillermopolito@gmail.com>:
Hi all,
Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere".
If people would like to introduce Beacon, I'd like to have at least the following questions answered: - What tools/core libraries of Pharo should be rewritten using Beacon?
We will need to replace transcript users with Beacon signals. In simplest cases it would be: StringSignal emit: aString But we should try use objects instead of strings. And we need to decide what to do with #logCr.
- And which ones do not?
Logging inside bootstrapped kernel will stay independent from Beacon.
- **Who** is going to take care for those integrations/rewritings? - How will this delay (even more) the release of Pharo 7?
We can just include this tool without cleanup. So no delay. Idea to finally have standard logging system extendable by third party loggers (backends). So when users will ask how to log the answer will be the Beacon and a link to the place with existing loggers.
Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems...
It is simple inspector on Beacon instance. We can add world menu entry to open it. It is based on Announcer inspector. So any improvements will improve announcer in general.
- What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? - **Who** is going to implement them?
If I put this side by side with the versionning questions above, this ones have more weight to me...
Cheers, Guille
Denis Kudriashov <dionisiydk@gmail.com> wrote:
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
They donât, and shouldnât. That is the way Pillar killed Pier. No separate maintaining of core constantly breaking users. Stephan
2018-05-26 10:20 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
They donât, and shouldnât. That is the way Pillar killed Pier. No separate maintaining of core constantly breaking users.
Ok. I understand your point. But I don't agree with you.
Stephan
On 26 May 2018, at 09:20, Stephan Eggermont <stephan@stack.nl> wrote:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
They donât, and shouldnât. That is the way Pillar killed Pier. No separate maintaining of core constantly breaking users.
Pillar did not kill Pier. Pier was killed by the fact that nobody is using it (then, there is few efforts to maintain it). which is sad, because Pier was/is very powerful⦠Anyway, we can spell it differently: at least, with Pillar, a part of Pier remains in use, and not just in our hearts :) Esteban
Stephan
+1 Doru
On May 26, 2018, at 10:13 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 26 May 2018, at 09:20, Stephan Eggermont <stephan@stack.nl> wrote:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
They donât, and shouldnât. That is the way Pillar killed Pier. No separate maintaining of core constantly breaking users.
Pillar did not kill Pier. Pier was killed by the fact that nobody is using it (then, there is few efforts to maintain it). which is sad, because Pier was/is very powerfulâ¦
Anyway, we can spell it differently: at least, with Pillar, a part of Pier remains in use, and not just in our hearts :)
Esteban
Stephan
-- www.tudorgirba.com www.feenk.com "We cannot reach the flow of things unless we let go."
Esteban Lorenzano <estebanlm@gmail.com> wrote: .
Pillar did not kill Pier. Pier was killed by the fact that nobody is using it (then, there is few efforts to maintain it). which is sad, because Pier was/is very powerfulâ¦
No. Pillar killed Pier by breaking it unnecessarily. A lot of times. No Pillar maintainer ever looked at our Pier builds and tried fixing things in a compatible way, or committed a fix when they broke something. Pier is stable and works just fine. It is infrastructure and nobody makes money with it, so maintenance is never going to be high priority. If you take up responsibility for changing core, you make sure the higher level still works. If you donât want that, fork Stephan
Am 26.05.2018 um 11:02 schrieb Stephan Eggermont <stephan@stack.nl>:
Esteban Lorenzano <estebanlm@gmail.com> wrote: .
Pillar did not kill Pier. Pier was killed by the fact that nobody is using it (then, there is few efforts to maintain it). which is sad, because Pier was/is very powerfulâ¦
No. Pillar killed Pier by breaking it unnecessarily. A lot of times. No Pillar maintainer ever looked at our Pier builds and tried fixing things in a compatible way, or committed a fix when they broke something. Pier is stable and works just fine. It is infrastructure and nobody makes money with it, so maintenance is never going to be high priority.
How can pillar kill pier? Pillar is a wonderful effort to rescue a good document model that was burried in a CMS. And pillar is not a CMS. So either you need a CMS and you use pier or you need a document model and you use pilkar. The requirements are completely different so one cannot kill the other. Norbert
If you take up responsibility for changing core, you make sure the higher level still works. If you donât want that, fork
Am 26.05.2018 um 08:41 schrieb Denis Kudriashov <dionisiydk@gmail.com>:
2018-05-26 9:30 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably do
not
want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
You do not have a use case for separate repos, but one for duplicates of the same repo.
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
Sure, but for me that does not count for FileLogger. Your approach might be the right way theoretically. But putting a single class with a separate baseline in a separate repository is just overkill. For something like FileLogger I donât want to have separate versions. It is so basic and fundamental that the only thing necessary is to be able not to load it and that is fine with groups. Furthermore if I change FileLogger and BeaconCore at the same time how can I use the changed versions together? Changing FileLogger to point to BeaconCore master and when Iâm done release a new BeaconCore version to point at from FileLogger? I hope you see that your approach enables certain things and makes others more complicated. So I agree that not every logging client should be in the beacon repo. But there is a line to draw for which it is useful to separate and for which it is not. And for FileLogger it is not useful. Even for syslog I doubt it. For my elasticsearch logger it is of course not the right place and that need to be separate. Btw. this is kind if a discussion I wanted to have for a long time. We need a definition of what pharo really includes. To me there is the minimal core containing only the necessary things, then there is the pharo eco system that includes all the tools the community provides and maintains. And then there are user projects that is everything else. FileLogger is surely part of the eco system. Syslog as well Norbert
You need that anyhow because your images depend on different branches and versions. You might want to solve that by having only one image responsible for source code management, and all others connecting to that instead of using libgit directly.
Stephan
2018-05-26 12:19 GMT+03:00 Norbert Hartl <norbert@hartl.name>:
Am 26.05.2018 um 08:41 schrieb Denis Kudriashov <dionisiydk@gmail.com>:
2018-05-26 9:30 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 20:03 GMT+03:00 Stephan Eggermont <stephan@stack.nl>:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Because when you will fix or improve Beacon-SysLog you will probably
do not
want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
I seem to be missing something here. Why would you update the baseline for that?
Question is not about baselines. Separate repositories for Core and loggers are required to version them separately. It will allow users to load Core version 1.1, SysLog version 2.0 and FileLogger 3.0. And it will allow maintainers improve these projects separately. As I was described the fix in SysLog will not require updating BeaconCore which is included in Pharo.
You do not have a use case for separate repos, but one for duplicates of the same repo.
If two entities needs separate versioning they should be in separate repositories. Do you agree with this?
Sure, but for me that does not count for FileLogger. Your approach might be the right way theoretically. But putting a single class with a separate baseline in a separate repository is just overkill. For something like FileLogger I donât want to have separate versions. It is so basic and fundamental that the only thing necessary is to be able not to load it and that is fine with groups. Furthermore if I change FileLogger and BeaconCore at the same time how can I use the changed versions together? Changing FileLogger to point to BeaconCore master and when Iâm done release a new BeaconCore version to point at from FileLogger? I hope you see that your approach enables certain things and makes others more complicated. So I agree that not every logging client should be in the beacon repo. But there is a line to draw for which it is useful to separate and for which it is not. And for FileLogger it is not useful. Even for syslog I doubt it. For my elasticsearch logger it is of course not the right place and that need to be separate. Btw. this is kind if a discussion I wanted to have for a long time. We need a definition of what pharo really includes. To me there is the minimal core containing only the necessary things, then there is the pharo eco system that includes all the tools the community provides and maintains. And then there are user projects that is everything else. FileLogger is surely part of the eco system. Syslog as well
Ok. I agree
Norbert
You need that anyhow because your images depend on different
branches and versions. You might want to solve that by having only one image responsible for source code management, and all others connecting to that instead of using libgit directly.
Stephan
Hi, Yes, we want to have the concrete loggers managed with separate baselines. I would also put them in individual repositories. Cheers, Doru
On May 25, 2018, at 5:58 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 17:21 GMT+03:00 Stephan Eggermont <stephan@stack.nl>: Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos?
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
Stephan
-- www.tudorgirba.com www.feenk.com "In a world where everything is moving ever faster, one might have better chances to win by moving slower."
Really?? Donât you think the overhead is massive compared to the gain? Norbert
Am 25.05.2018 um 21:08 schrieb Tudor Girba <tudor@tudorgirba.com>:
Hi,
Yes, we want to have the concrete loggers managed with separate baselines. I would also put them in individual repositories.
Cheers, Doru
On May 25, 2018, at 5:58 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 17:21 GMT+03:00 Stephan Eggermont <stephan@stack.nl>: Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos?
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
Stephan
-- www.tudorgirba.com www.feenk.com
"In a world where everything is moving ever faster, one might have better chances to win by moving slower."
Norbert Hartl <norbert@hartl.name> wrote:
Really?? Donât you think the overhead is massive compared to the gain?
As long as the baselines are nice trees, described only one level deep, it should be ok-ish. That never seems to be the case though, and as soon as you get diamond dependency issues the duplication of volatile information creates a lot of commit ripple Iâd would like to see a description of how that is supposed to work with all the duplicated baselines and repos. Stephan
Hi Norbert 2018-05-25 23:50 GMT+03:00 Norbert Hartl <norbert@hartl.name>:
Really?? Donât you think the overhead is massive compared to the gain?
I am sure that you were agreed with this when we push Beacon two yeas ago. But look at my explanation in previous mail.
Norbert
Am 25.05.2018 um 21:08 schrieb Tudor Girba <tudor@tudorgirba.com>:
Hi,
Yes, we want to have the concrete loggers managed with separate baselines. I would also put them in individual repositories.
Cheers, Doru
On May 25, 2018, at 5:58 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 17:21 GMT+03:00 Stephan Eggermont <stephan@stack.nl>: Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos?
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
Stephan
-- www.tudorgirba.com www.feenk.com
"In a world where everything is moving ever faster, one might have better chances to win by moving slower."
Hi, I do not have a strong opinion about repositories, but I do think that the concrete loggers will be inevitably created by different people with different interests, and this setup will naturally be more suitable for different repositories. I do have a strong opinion about baselines: The baseline problem exists already at a significant scale and is not a local one. they are too costly to maintain now, and we need to build tools anyway to handle them cheaply. Without those tools we are confined to manual work, and optimizing design around manual work is not a good direction. So, trying to optimize one project is not particularly useful. About Metacello groups: I think that in most cases using them for dependency purposes is less productive than a separate baseline because they are not reified completely, and they are also not polymorphic with baselines which makes it hard to build tools to handle them. Cheers, Doru
On May 25, 2018, at 10:50 PM, Norbert Hartl <norbert@hartl.name> wrote:
Really?? Donât you think the overhead is massive compared to the gain?
Norbert
Am 25.05.2018 um 21:08 schrieb Tudor Girba <tudor@tudorgirba.com>:
Hi,
Yes, we want to have the concrete loggers managed with separate baselines. I would also put them in individual repositories.
Cheers, Doru
On May 25, 2018, at 5:58 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2018-05-25 17:21 GMT+03:00 Stephan Eggermont <stephan@stack.nl>: Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
I was looking at Beacon again to prepare integration. Some changes was needed in baseline for better granularity. So I send pull request because I have no permissions.
But now I remember that we had idea to integrate only core part of Beacon which requires MemoryLogger and TranscriptLogger. And we wanted to manage logger backends using separate baselines. So we need separate repos to have separate versioning.
Why would you want separate baselines for that? And on top of that separate repos?
Because when you will fix or improve Beacon-SysLog you will probably do not want to update Beacon-Core version which will force you to update Pharo (where SysLog is not loaded).
Stephan
-- www.tudorgirba.com www.feenk.com
"In a world where everything is moving ever faster, one might have better chances to win by moving slower."
-- www.tudorgirba.com www.feenk.com "We are all great at making mistakes."
Tudor Girba <tudor@tudorgirba.com> wrote:
I do have a strong opinion about baselines: The baseline problem exists already at a significant scale and is not a local one. they are too costly to maintain now, and we need to build tools anyway to handle them cheaply. Without those tools we are confined to manual work, and optimizing design around manual work is not a good direction. So, trying to optimize one project is not particularly useful.
We can only start from where we are. Iâm sure that better tools will help, and pushing for things to be structured in a way that gives us more work now makes it take longer to get those tools. Groups are at the moment essential to minimize manual work. Stephan
participants (9)
-
Denis Kudriashov -
Esteban Lorenzano -
Guillermo Polito -
Henrik Sperre Johansen -
John Brant -
Marcus Denker -
Norbert Hartl -
Stephan Eggermont -
Tudor Girba