August 2020 - Pharo-users - lists.pharo.org

Re: [Pharo-users] CannotWriteData errors in P3 and Seaside

by Esteban Maringolo

Hi, I'm replying to the list as well... because the last two mails got replied to our personal addresses. On Thu, Aug 20, 2020 at 11:55 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 20 Aug 2020, at 15:31, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > Hi Sven, > > > > If a socketstream doesn't know the state of the connection, then what > > is the #socketIsConnected method for? In particular the > > #isOtherEndClosed test. > > > > ZdcAbstractSocketStream>>#socketIsConnected > > ^socket isConnected and: [ socket isOtherEndClosed not ] > > I don't know what is going on inside Socket, I just stated my opinion. Maybe there is something to investigate here? > With logging enabled, I can do the following: > > $ grep P3 server-2020-08-20.log | grep CONNECT | tail -n 20 > <snip> > 2020-08-20 14:43:06 [P3] 30513 DISCONNECTING psql://client-xyz:hiddenpassword@client-xyz-db:5432/client-xyz > 2020-08-20 14:44:06 [P3] 30516 CONNECTED psql://client-xyz:hiddenpassword@client-xyz-db:5432/client-xyz > 2020-08-20 14:44:06 [P3] 30516 DISCONNECTING psql://client-xyz:hiddenpassword@client-xyz-db:5432/client-xyz > 2020-08-20 14:44:06 [P3] 30517 CONNECTED psql://client-xyz:hiddenpassword@client-xyz-db:5432/client-xyz > 2020-08-20 14:44:06 [P3] 30517 DISCONNECTING psql://client-xyz:hiddenpassword@client-xyz-db:5432/client-xyz > > The number after [P3] is the session identifier (backend process id) of that connection. You should see each one being opened and closed in pairs. Yes, I noticed the pid, and compared it with what I had on the pg_stat_activity table. I don't get the CONNECTED log because there is no way to set the logging in the P3DatabaseDriver before it creates (and connects) the P3Client. Maybe there could be a setting on P3Client class to set verbosity globally? Or at the P3DatabaseDriver instead. Summarizing... I'm pretty confident that P3 works correctly and also the PG server. At this point I'm factoring out what might be causing this. It's an issue that only happens to me in production, and I don't have a better instrumentation in place to debug it. Again, thanks for the support. Regards.

Aug. 20, 2020

Re: [Pharo-users] Iceberg "adopt commit" behavior

by Esteban Maringolo

On Thu, Aug 20, 2020 at 10:48 AM Sean P. DeNigris <sean(a)clipperadams.com> wrote: > > The image knows at any moment the commit it has loaded (or it supposed > > to have loaded). > > For posterity, that info can be accessed via e.g. `anIceLibgitRepository > workingCopy referenceCommit shortId` This comes really handy, so I add how do I get the IceLibgitRepository instance? (given I provide the name). Regards! Esteban A. Maringolo

Aug. 20, 2020

Re: [Pharo-users] Iceberg "adopt commit" behavior

by Sean P. DeNigris

tesonep(a)gmail.com wrote > It sets the reference commit of the image to the given commit. It does > not affect the git repository...The adopt operation overrides the > reference commit with the selected > one, it does not affect the loaded code. Ah, good to know! I didn't know this was available and think it would've come in handy a few times :) tesonep(a)gmail.com wrote > The image knows at any moment the commit it has loaded (or it supposed > to have loaded). For posterity, that info can be accessed via e.g. `anIceLibgitRepository workingCopy referenceCommit shortId` ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Aug. 20, 2020

Re: [Pharo-users] Iceberg "adopt commit" behavior

by tesonep＠gmail.com

Yes, it should work. The same if you repair the repository doing a checkout loading the changes from the repository (there should not be changes) Tell us if you have any questions Thanks! On Thu, Aug 20, 2020 at 2:21 PM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi Pablo, > > Thanks for the answer. > > So my workflow of editing files outside of Pharo, committing them, and > then adopting the latest commit would be safe? (as long I don't modify > files modified by Pharo). > > Best regards! > > > Esteban A. Maringolo > > On Thu, Aug 20, 2020 at 6:46 AM tesonep(a)gmail.com <tesonep(a)gmail.com> wrote: > > > > Hi, > > the adopt commit operation has nothing to do with the rebase. It > > sets the reference commit of the image to the given commit. It does > > not affect the git repository. > > The image knows at any moment the commit it has loaded (or it supposed > > to have loaded). > > The adopt operation overrides the reference commit with the selected > > one, it does not affect the loaded code. > > There is risk of doing so, it changes the commit in the image but > > without changing the repository nor the loaded code. So, it can > > produce a detached working-copy, and also it affects the records of > > changes the image has. > > > > On Thu, Aug 20, 2020 at 5:16 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > > > Hi, > > > > > > What does the "Adopt commit" mean? > > > > > > It seems like a rebase, but I'm not sure. > > > > > > I often have ongoing changes in my image, and also changes in the > > > filesystem (css, js, Dockerfile, etc.). So what I do is to commit on > > > the filesystem, and then "adopt" the recently created commit, and then > > > commit in Iceberg (and probably push). > > > > > > Is this okay? Is there any risk in doing this? > > > > > > Regards! > > > > > > Esteban A. Maringolo > > > > > > > > > -- > > Pablo Tesone. > > tesonep(a)gmail.com > > > -- Pablo Tesone. tesonep(a)gmail.com

Aug. 20, 2020

Re: [Pharo-users] Iceberg "adopt commit" behavior

by Esteban Maringolo

Hi Pablo, Thanks for the answer. So my workflow of editing files outside of Pharo, committing them, and then adopting the latest commit would be safe? (as long I don't modify files modified by Pharo). Best regards! Esteban A. Maringolo On Thu, Aug 20, 2020 at 6:46 AM tesonep(a)gmail.com <tesonep(a)gmail.com> wrote: > > Hi, > the adopt commit operation has nothing to do with the rebase. It > sets the reference commit of the image to the given commit. It does > not affect the git repository. > The image knows at any moment the commit it has loaded (or it supposed > to have loaded). > The adopt operation overrides the reference commit with the selected > one, it does not affect the loaded code. > There is risk of doing so, it changes the commit in the image but > without changing the repository nor the loaded code. So, it can > produce a detached working-copy, and also it affects the records of > changes the image has. > > On Thu, Aug 20, 2020 at 5:16 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > Hi, > > > > What does the "Adopt commit" mean? > > > > It seems like a rebase, but I'm not sure. > > > > I often have ongoing changes in my image, and also changes in the > > filesystem (css, js, Dockerfile, etc.). So what I do is to commit on > > the filesystem, and then "adopt" the recently created commit, and then > > commit in Iceberg (and probably push). > > > > Is this okay? Is there any risk in doing this? > > > > Regards! > > > > Esteban A. Maringolo > > > > > -- > Pablo Tesone. > tesonep(a)gmail.com >

Aug. 20, 2020

Re: [Pharo-users] Iceberg "adopt commit" behavior

by tesonep＠gmail.com

Hi, the adopt commit operation has nothing to do with the rebase. It sets the reference commit of the image to the given commit. It does not affect the git repository. The image knows at any moment the commit it has loaded (or it supposed to have loaded). The adopt operation overrides the reference commit with the selected one, it does not affect the loaded code. There is risk of doing so, it changes the commit in the image but without changing the repository nor the loaded code. So, it can produce a detached working-copy, and also it affects the records of changes the image has. On Thu, Aug 20, 2020 at 5:16 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi, > > What does the "Adopt commit" mean? > > It seems like a rebase, but I'm not sure. > > I often have ongoing changes in my image, and also changes in the > filesystem (css, js, Dockerfile, etc.). So what I do is to commit on > the filesystem, and then "adopt" the recently created commit, and then > commit in Iceberg (and probably push). > > Is this okay? Is there any risk in doing this? > > Regards! > > Esteban A. Maringolo > -- Pablo Tesone. tesonep(a)gmail.com

Aug. 20, 2020

Iceberg "adopt commit" behavior

by Esteban Maringolo

Hi, What does the "Adopt commit" mean? It seems like a rebase, but I'm not sure. I often have ongoing changes in my image, and also changes in the filesystem (css, js, Dockerfile, etc.). So what I do is to commit on the filesystem, and then "adopt" the recently created commit, and then commit in Iceberg (and probably push). Is this okay? Is there any risk in doing this? Regards! Esteban A. Maringolo

Aug. 20, 2020

Re: [Pharo-users] CannotWriteData errors in P3 and Seaside

by Sven Van Caekenberghe

IIUC a socket stream does not automatically/automagically know that the state of the connection changed, unless/until it tries to use it (read or write to it, wait for data, ...). I would recommend using #isWorking to actually test if a connection is good, if you would need to do that. That does an actual query. Did you enable P3 logging as I suggested ? What did you learn ? It feels as if in your particular setup the server eagerly closes connections that are open for too long. P3 was written under the assumption that that does not happen (it does not for me). > On 19 Aug 2020, at 19:48, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > I kept looking into this, and still haven't found what might be causing it. > > However, I was trying to "salvage" that until I find a solution, and > run a "healthcheck" to be sure that the GlorpSession has an active > connection, and then I found that a P3Client reports as connected even > when it's not. > > I connected to a PostgreSQL database running in a Docker container, > stopped the container, and the driver continues to report as > connected, even way after the server was stopped (the timeout is using > the default 10 seconds). > > The ZdcSocketStream reports both ends of the socket as connected, when > the server side certainly isn't. > > I noticed also that in P3Client>>connect it calls #ensureOpen, and it > takes the socket as open when it is not, so as soon as it tries to > flush the data written to it, a `CannotWriteData` exception is > signaled. > > In my case I'm developing on Pharo 8 on Windows with PostgreSQL > running in Docker on WSL2 (ubuntu), but on the server it is a 100% > Linux deployment. > > Any ideas? > > Esteban A. Maringolo > > On Mon, Aug 10, 2020 at 5:44 PM Esteban Maringolo <emaringolo(a)gmail.com> wrote: >> >> Looking at the Postgres side of the log I find that the connection was >> reset from the other side (it is Pharo). >> >> The reason for that is yet unknown to me. Since I don't do anything >> (that I'm aware of). >> >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:02:35.939 UTC >> [132] LOG: could not receive data from client: Connection reset by >> peer >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:06:58.083 UTC >> [139] LOG: could not receive data from client: Connection reset by >> peer >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:06:58.083 UTC >> [137] LOG: could not receive data from client: Connection reset by >> peer >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:33:20.163 UTC >> [166] LOG: could not receive data from client: Connection reset by >> peer >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:35:22.019 UTC >> [168] LOG: could not receive data from client: Connection reset by >> peer >> golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:37:33.091 UTC >> [177] LOG: could not receive data from client: Connection reset by >> peer >> >> I'll have to keep looking. >> >> Best regards! >> >> Esteban A. Maringolo >> >> >> On Mon, Aug 10, 2020 at 5:35 PM Esteban Maringolo <emaringolo(a)gmail.com> wrote: >>> >>> My Seaside session isn't closing the connection, only when >>> unregistered, but this seems to be something else I don't know. >>> >>> I saw there is logging, and I need to set it up in general (including >>> Fuel serialized stacks). >>> Looking in the web apparently there is the need for a keepalive that >>> is not in place. >>> What disturbs me is that it doesn't happen in development. Making things harder. >>> >>> Regards! >>> >>> Esteban A. Maringolo >>> >>> On Mon, Aug 10, 2020 at 5:02 PM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hi Esteban, >>>> >>>> I have a web app with P3 under Seaside in production and it works fine. But that is without Glorp, nor any connection pooling. >>>> >>>> You say the connection seems closed, maybe the closing got triggered by your app somehow ? How do you clean up expired sessions ? How do you handle logouts ? >>>> >>>> P3 does normally reconnect automatically, IIRC. >>>> >>>> You could try to enable logging in P3Client, that is a recent addition. It should show you what happens to your connections. >>>> >>>> Sven >>>> >>>>> On 10 Aug 2020, at 21:15, Esteban Maringolo <emaringolo(a)gmail.com> wrote: >>>>> >>>>> Hi all, Sven ;-) >>>>> >>>>> I'm having erratic P3 errors in a recent application I wrote using >>>>> Pharo, Seaside and Glorp with P3 as driver. >>>>> >>>>> Each Seaside session has a GlorpSession, which in turn has a >>>>> P3Connection in its accessor. I don't know why, but sometimes the >>>>> P3Connection socket is closed, and then when trying to read from the >>>>> database, it cannot write the query to the P3 socket and exception is >>>>> raised, and it isn't handled by the P3DatabaseDriver (automatically >>>>> trying to reconnect?). >>>>> >>>>> I don't know if I'm doing something wrong, I plan to migrate the >>>>> GlorpPooledDatabaseAccessor and also use the P3ConnectionPool, but I >>>>> want to be sure that the current setup works of if maybe I'm exceeding >>>>> some limit or timeout that causes the connection to be closed. >>>>> >>>>> Regards! >>>>> >>>>> >>>>> Esteban A. Maringolo >>>>> >>>> >>>>

Aug. 19, 2020

Re: [Pharo-users] CannotWriteData errors in P3 and Seaside

by Esteban Maringolo

I kept looking into this, and still haven't found what might be causing it. However, I was trying to "salvage" that until I find a solution, and run a "healthcheck" to be sure that the GlorpSession has an active connection, and then I found that a P3Client reports as connected even when it's not. I connected to a PostgreSQL database running in a Docker container, stopped the container, and the driver continues to report as connected, even way after the server was stopped (the timeout is using the default 10 seconds). The ZdcSocketStream reports both ends of the socket as connected, when the server side certainly isn't. I noticed also that in P3Client>>connect it calls #ensureOpen, and it takes the socket as open when it is not, so as soon as it tries to flush the data written to it, a `CannotWriteData` exception is signaled. In my case I'm developing on Pharo 8 on Windows with PostgreSQL running in Docker on WSL2 (ubuntu), but on the server it is a 100% Linux deployment. Any ideas? Esteban A. Maringolo On Mon, Aug 10, 2020 at 5:44 PM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Looking at the Postgres side of the log I find that the connection was > reset from the other side (it is Pharo). > > The reason for that is yet unknown to me. Since I don't do anything > (that I'm aware of). > > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:02:35.939 UTC > [132] LOG: could not receive data from client: Connection reset by > peer > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:06:58.083 UTC > [139] LOG: could not receive data from client: Connection reset by > peer > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:06:58.083 UTC > [137] LOG: could not receive data from client: Connection reset by > peer > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:33:20.163 UTC > [166] LOG: could not receive data from client: Connection reset by > peer > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:35:22.019 UTC > [168] LOG: could not receive data from client: Connection reset by > peer > golfware_database.1.9sl4bt9j6cv5@gw | 2020-08-10 20:37:33.091 UTC > [177] LOG: could not receive data from client: Connection reset by > peer > > I'll have to keep looking. > > Best regards! > > Esteban A. Maringolo > > > On Mon, Aug 10, 2020 at 5:35 PM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > My Seaside session isn't closing the connection, only when > > unregistered, but this seems to be something else I don't know. > > > > I saw there is logging, and I need to set it up in general (including > > Fuel serialized stacks). > > Looking in the web apparently there is the need for a keepalive that > > is not in place. > > What disturbs me is that it doesn't happen in development. Making things harder. > > > > Regards! > > > > Esteban A. Maringolo > > > > On Mon, Aug 10, 2020 at 5:02 PM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > > > > Hi Esteban, > > > > > > I have a web app with P3 under Seaside in production and it works fine. But that is without Glorp, nor any connection pooling. > > > > > > You say the connection seems closed, maybe the closing got triggered by your app somehow ? How do you clean up expired sessions ? How do you handle logouts ? > > > > > > P3 does normally reconnect automatically, IIRC. > > > > > > You could try to enable logging in P3Client, that is a recent addition. It should show you what happens to your connections. > > > > > > Sven > > > > > > > On 10 Aug 2020, at 21:15, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > > > > > Hi all, Sven ;-) > > > > > > > > I'm having erratic P3 errors in a recent application I wrote using > > > > Pharo, Seaside and Glorp with P3 as driver. > > > > > > > > Each Seaside session has a GlorpSession, which in turn has a > > > > P3Connection in its accessor. I don't know why, but sometimes the > > > > P3Connection socket is closed, and then when trying to read from the > > > > database, it cannot write the query to the P3 socket and exception is > > > > raised, and it isn't handled by the P3DatabaseDriver (automatically > > > > trying to reconnect?). > > > > > > > > I don't know if I'm doing something wrong, I plan to migrate the > > > > GlorpPooledDatabaseAccessor and also use the P3ConnectionPool, but I > > > > want to be sure that the current setup works of if maybe I'm exceeding > > > > some limit or timeout that causes the connection to be closed. > > > > > > > > Regards! > > > > > > > > > > > > Esteban A. Maringolo > > > > > > > > > >

Aug. 19, 2020

PostgresV2 Driver available?

by Esteban Maringolo

Hi, Is anybody using the "old" PostgresV2 driver in Pharo 8? I just checked in the pharo-rdbms/garage repository and of course everything is very old there. But I'm having a few issues with P3 that I didn't have before (with the PgV2) and I want to be test whether there is something related with the connection or the problem lies somewhere else. Regards! Esteban A. Maringolo

Aug. 19, 2020