two issues: GC and wrong pointers
Hi, During our work we encountered two issues we think should be considered critical. The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone. The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it. Would anyone be available for a collaboration in the following days? Cheers, Doru -- feenk.com "Not knowing how to do something is not an argument for how it cannot be done."
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
This one I want to have a look at⦠I think the solution is to remove the magic and maybe even think about integrating the FFI plugin into the compiler. I am available Tuesday afternoon. Marcus
Hi doru Can you tell us more? How many threads are running? how many times did you save the image? Pablo is investigating a bug related to the GC that happens when we have many threads and saved the image multiple times. He was on it since a couple of days. Now Pablo is attending Smalltalks and will be on vacation until end of the month. Guille is on father leave and coming back to work one day a week. So we are concerned about this problem but we cannot really allocate people on it before beginning of december. S.
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
Would anyone be available for a collaboration in the following days?
Cheers, Doru
-- feenk.com
"Not knowing how to do something is not an argument for how it cannot be done."
Hi, For the GC problem from https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391, there is no threading involved. Just invoking full GC in a large image. Cheers, Doru
On Nov 11, 2019, at 1:47 PM, ducasse <stepharo@netcourrier.com> wrote:
Hi doru
Can you tell us more? How many threads are running? how many times did you save the image? Pablo is investigating a bug related to the GC that happens when we have many threads and saved the image multiple times. He was on it since a couple of days.
Now Pablo is attending Smalltalks and will be on vacation until end of the month. Guille is on father leave and coming back to work one day a week.
So we are concerned about this problem but we cannot really allocate people on it before beginning of december.
S.
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
Would anyone be available for a collaboration in the following days?
Cheers, Doru
-- feenk.com
"Not knowing how to do something is not an argument for how it cannot be done."
-- feenk.com "Value is always contextual."
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
I think my PR fixes it⦠the PR is green and the downloaded artefact seems to not show any problematic methods anymore: https://github.com/pharo-project/pharo/pull/5120 <https://github.com/pharo-project/pharo/pull/5120> Marcus
marcus I read your fix and I do not understand why the previous way was a problem. What was your hypothesis? Stef
On 11 Nov 2019, at 14:46, Marcus Denker <marcus.denker@inria.fr> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 <https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391> We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
I think my PR fixes it⦠the PR is green and the downloaded artefact seems to not show any problematic methods anymore:
https://github.com/pharo-project/pharo/pull/5120 <https://github.com/pharo-project/pharo/pull/5120>
Marcus
On 11 Nov 2019, at 15:04, ducasse <stepharo@netcourrier.com> wrote:
marcus I read your fix and I do not understand why the previous way was a problem. What was your hypothesis?
I think the problem is that it forces recompiling methods while already loading code, which leads to some strange effect.. that I think the MethodChanged that monticello does at the end references the old method, but that got recompiled in the meantime. Something like that.
Stef
On 11 Nov 2019, at 14:46, Marcus Denker <marcus.denker@inria.fr <mailto:marcus.denker@inria.fr>> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 <https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391> We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 <https://github.com/pharo-project/pharo/issues/4967> This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
I think my PR fixes it⦠the PR is green and the downloaded artefact seems to not show any problematic methods anymore:
https://github.com/pharo-project/pharo/pull/5120 <https://github.com/pharo-project/pharo/pull/5120>
Marcus
Thanks Marcus! Alistair reviewed the issue (https://github.com/pharo-project/pharo/pull/5120) and the pull request looks great. Cool work :). Cheers, Doru
On Nov 11, 2019, at 3:51 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 11 Nov 2019, at 15:04, ducasse <stepharo@netcourrier.com> wrote:
marcus I read your fix and I do not understand why the previous way was a problem. What was your hypothesis?
I think the problem is that it forces recompiling methods while already loading code, which leads to some strange effect.. that I think the MethodChanged that monticello does at the end references the old method, but that got recompiled in the meantime.
Something like that.
Stef
On 11 Nov 2019, at 14:46, Marcus Denker <marcus.denker@inria.fr> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
I think my PR fixes it⦠the PR is green and the downloaded artefact seems to not show any problematic methods anymore:
https://github.com/pharo-project/pharo/pull/5120
Marcus
-- feenk.com "To utilize feedback, you first have to acquire it."
May be we have two problems then :( Because we have problem without a large image. Stef
On 11 Nov 2019, at 14:44, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
For the GC problem from https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391, there is no threading involved. Just invoking full GC in a large image.
Cheers, Doru
On Nov 11, 2019, at 1:47 PM, ducasse <stepharo@netcourrier.com> wrote:
Hi doru
Can you tell us more? How many threads are running? how many times did you save the image? Pablo is investigating a bug related to the GC that happens when we have many threads and saved the image multiple times. He was on it since a couple of days.
Now Pablo is attending Smalltalks and will be on vacation until end of the month. Guille is on father leave and coming back to work one day a week.
So we are concerned about this problem but we cannot really allocate people on it before beginning of december.
S.
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
The second one is that the sources contain wrong pointers, and this makes it hard to work with TFFI: https://github.com/pharo-project/pharo/issues/4967 This is already marked as important. We spent quite some time investigating this, especially in the context of TFFI, and we would be happy to pair to work through it.
Would anyone be available for a collaboration in the following days?
Cheers, Doru
-- feenk.com
"Not knowing how to do something is not an argument for how it cannot be done."
-- feenk.com
"Value is always contextual."
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
Doru To me this is not really a good bug report. How can we reproduce it? What is a big image? The first thing to do is to get a simple reproducible case. This is what pavel did with the crash on normal images. Can you see if you can do something like that? It will help us trying to fix it. S.
Hi, Yes, I know itâs not a good public bug report. That is why I asked for help. The problem is that this is not easily reproducible outside of specific cases. We encountered it in the context of GT which has double the code as compared to Pharo. So, to test it outside of GT, for the last couple of weeks we manipulated some data in plain Pharo, and we still got to the problem a couple of times (though it appears less often). We do not know yet how to reproduce it with anonymous data. That is why I said that some pairing might help. In parallel we are trying to reproduce it, like we do with all other bug reports. Cheers, Doru
On Nov 12, 2019, at 8:00 AM, ducasse <stepharo@netcourrier.com> wrote:
On 11 Nov 2019, at 12:57, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
During our work we encountered two issues we think should be considered critical.
The first is that the garbage collector is unreliable on larger images: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 We stumble on this when we manipulate larger pieces of data quite consistently in plain Pharo (7 or 8). We lack the knowledge of how to approach it, but we would happily pair and try to reproduce it with someone.
Doru
To me this is not really a good bug report. How can we reproduce it? What is a big image?
The first thing to do is to get a simple reproducible case. This is what pavel did with the crash on normal images. Can you see if you can do something like that? It will help us trying to fix it.
S.
-- feenk.com "What is more important: To be happy, or to make happy?"
Hi, we found a script that reproduces the GC segmentation fault. The report is here - https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/444 <https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/444> - which is likely the same issue as reported here: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391 <https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/391> Best, Juraj
On 12 Nov 2019, at 12:40, ducasse <stepharo@netcourrier.com> wrote:
ok It may be related to the bug pablo is working on. So it would be good to see how many times the image is saved.
Stef
On 12 Nov 2019, at 08:58, Tudor Girba <tudor@tudorgirba.com <mailto:tudor@tudorgirba.com>> wrote:
ith all other bug reports.
participants (4)
-
ducasse -
Juraj Kubelka -
Marcus Denker -
Tudor Girba