'VOMongoConnectionError' when dowloading mcz from smalltalkhub.com
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz. If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ... Here's a screenshot of a manual login and navigation to the mcz file that is failing to download: And when I press the `Download .mcz` button, I get the following "response" after a delay:
Hi Dale, Can't you use this? http://static.smalltalkhub.com/dkh/metacello/mc/Metacello-Base-dkh.109.mcz Available at: http://static.smalltalkhub.com/dkh/metacello/mc/ Maybe the requests to the "dynamic" SmalltalkHub could be rewritten to point to the static version of it. Regards! Esteban A. Maringolo On Tue, Aug 25, 2020 at 1:03 PM Dale Henrichs < dale.henrichs@gemtalksystems.com> wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
And when I press the `Download .mcz` button, I get the following "response" after a delay:
On 8/25/20 9:07 AM, Esteban Maringolo wrote:
Hi Dale,
Can't you use this? http://static.smalltalkhub.com/dkh/metacello/mc/Metacello-Base-dkh.109.mcz
I can hack things to make my single test case work, but there is code and configuration files at GemStone/GLASS/GsDevKit sites around the world that I cannot edit ... If I was told "sorry we are not going to make sure that the smalltalkhub servers are going to work any more", then I wouldn't have a choice, but to drop everything and do the conversion RIGHT NOW. Eventually there is going to be a (hopefully) more stable static site and IF/WHEN that happens I won't have to change anything, but this interim flakiness of the smalltalkhub servers is getting to be annoying ...
Available at: http://static.smalltalkhub.com/dkh/metacello/mc/
Maybe the requests to the "dynamic" SmalltalkHub could be rewritten to point to the static version of it.
If the dynamic servers are going to be unreliable, then this is probably a better solution, I'm sure that the folks who are still actively using the dynamic site aren't pleased with the flakiness either:) Dale
Regards!
Esteban A. Maringolo
On Tue, Aug 25, 2020 at 1:03 PM Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
And when I press the `Download .mcz` button, I get the following "response" after a delay:
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem! Dale On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
And when I press the `Download .mcz` button, I get the following "response" after a delay:
Hi Dale, Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable. Do not hesitate to ping if you have any problem. Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz <http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz> is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz <http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz>.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
On 27 Aug 2020, at 14:36, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub.
Are you sure ? For me, http://smalltalkhub.com is different from http://static.smalltalkhub.com And the download links http://www.smalltalkhub.com/mc/SvenVanCaekenberghe/ZincHTTPComponents/main/C... and http://static.smalltalkhub.com/SvenVanCaekenberghe/ZincHTTPComponents/mc/Con... are coming from a different server.
I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Again, I don't know what software is serving SmalltalkHub HTTP requests, but can't we make a redirect or rewrite rule in the HTTP request to transparently answer the proper resource when requested? It would be simply removing the `/mc` and changing the hostname. This way Baselines and other dependencies don't have to be rewritten to continue working. Regards! Esteban A. Maringolo On Thu, Aug 27, 2020 at 9:47 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 27 Aug 2020, at 14:36, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub.
Are you sure ?
For me, http://smalltalkhub.com is different from http://static.smalltalkhub.com
And the download links
http://www.smalltalkhub.com/mc/SvenVanCaekenberghe/ZincHTTPComponents/main/C...
and
http://static.smalltalkhub.com/SvenVanCaekenberghe/ZincHTTPComponents/mc/Con...
are coming from a different server.
I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Hi Esteban,
Le 27 août 2020 à 15:14, Esteban Maringolo <emaringolo@gmail.com> a écrit :
Again, I don't know what software is serving SmalltalkHub HTTP requests, but can't we make a redirect or rewrite rule in the HTTP request to transparently answer the proper resource when requested?
It would be simply removing the `/mc` and changing the hostname.
This way Baselines and other dependencies don't have to be rewritten to continue working.
I do not get what you mean. The static version rewrites old / used urls so that all URLs that are used in baselines or whatever are still served transparently. We took care of this to do not break the load of old piece of software. Regards, Christophe.
Hi Cristophe, I was referring to exactly what you mention you did to preserve the load of dependencies. But since it failed me, and also to Dale and others I know, I thought it was not the case that such consideration was in place. Thanks! Esteban A. Maringolo On Thu, Aug 27, 2020 at 11:02 AM Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi Esteban,
Le 27 août 2020 à 15:14, Esteban Maringolo <emaringolo@gmail.com> a écrit :
Again, I don't know what software is serving SmalltalkHub HTTP requests, but can't we make a redirect or rewrite rule in the HTTP request to transparently answer the proper resource when requested?
It would be simply removing the `/mc` and changing the hostname.
This way Baselines and other dependencies don't have to be rewritten to continue working.
I do not get what you mean. The static version rewrites old / used urls so that all URLs that are used in baselines or whatever are still served transparently. We took care of this to do not break the load of old piece of software.
Regards, Christophe.
Hi Sven,
Le 27 août 2020 à 14:47, Sven Van Caekenberghe <sven@stfx.eu> a écrit :
On 27 Aug 2020, at 14:36, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub.
Are you sure ?
For me, http://smalltalkhub.com is different from http://static.smalltalkhub.com
Yes, there are DNS caches probably stil referring to the non-static version. Maybe you can try this: https://coolestguidesontheplanet.com/clear-the-local-dns-cache-in-osx/ Regards, Christophe.
Christophe, There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation. In this case [2][3], the list of file shows up on the dynamic(?) site: and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem. Dale [1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411 On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:) I will continue digging ... Dale [1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
My guess is lies in the difference in the payload returned. http://www.squeaksource.com/MooseSQL/ produces a html page: and the static smalltalkhub site does not: I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:) Dale On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Hmm, this is going to be a hard one. SmalltalkHub got optimised in Pharo, consider MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names vs. MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ] In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case. BTW, there exists code to generate the listing in ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ] Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:) Dale [1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651 On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
The static version of StHub seems to always assume the client is Pharo, while the dynamic version used format=raw (which non-Pharo implementation did not include in their request for the listing) to send the Pharo specific response only then.
On 27 Aug 2020, at 23:34, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:)
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651
On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
That makes sense and confirms that the static site has a bug ... portions of my work are on hold until the SmalltalkHub issue is resolved and at least one other GemStone user is impacted, so far... Dale On 8/27/20 2:41 PM, Sven Van Caekenberghe wrote:
The static version of StHub seems to always assume the client is Pharo, while the dynamic version used format=raw (which non-Pharo implementation did not include in their request for the listing) to send the Pharo specific response only then.
On 27 Aug 2020, at 23:34, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:)
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651
On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Hi Dale, I would not call it a bug but an omission ;) It is hard to guess that a tool is scraping an html page to get data. Nevertheless, I will take a look today to find a solution for Glass to work again with smalltalkhub. Could you tell me what is the expected output? What is the html structure that is expected? I think I can find a solution to produce an html page through apache index listing and rewrite rules to catch URLs like http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main. Thanks, Christophe.
Le 27 août 2020 à 23:50, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
That makes sense and confirms that the static site has a bug ... portions of my work are on hold until the SmalltalkHub issue is resolved and at least one other GemStone user is impacted, so far...
Dale
On 8/27/20 2:41 PM, Sven Van Caekenberghe wrote:
The static version of StHub seems to always assume the client is Pharo, while the dynamic version used format=raw (which non-Pharo implementation did not include in their request for the listing) to send the Pharo specific response only then.
On 27 Aug 2020, at 23:34, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:)
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651
On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/ I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? Regards, Christophe
Le 28 août 2020 à 09:37, Christophe Demarey <christophe.demarey@inria.fr> a écrit :
Hi Dale,
I would not call it a bug but an omission ;) It is hard to guess that a tool is scraping an html page to get data.
Nevertheless, I will take a look today to find a solution for Glass to work again with smalltalkhub. Could you tell me what is the expected output? What is the html structure that is expected?
I think I can find a solution to produce an html page through apache index listing and rewrite rules to catch URLs like http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main.
Thanks, Christophe.
Le 27 août 2020 à 23:50, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
That makes sense and confirms that the static site has a bug ... portions of my work are on hold until the SmalltalkHub issue is resolved and at least one other GemStone user is impacted, so far...
Dale
On 8/27/20 2:41 PM, Sven Van Caekenberghe wrote:
The static version of StHub seems to always assume the client is Pharo, while the dynamic version used format=raw (which non-Pharo implementation did not include in their request for the listing) to send the Pharo specific response only then.
On 27 Aug 2020, at 23:34, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:)
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651
On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure?
See my earlier mail with code.
Regards, Christophe
Le 28 août 2020 à 09:37, Christophe Demarey <christophe.demarey@inria.fr> a écrit :
Hi Dale,
I would not call it a bug but an omission ;) It is hard to guess that a tool is scraping an html page to get data.
Nevertheless, I will take a look today to find a solution for Glass to work again with smalltalkhub. Could you tell me what is the expected output? What is the html structure that is expected?
I think I can find a solution to produce an html page through apache index listing and rewrite rules to catch URLs like http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main.
Thanks, Christophe.
Le 27 août 2020 à 23:50, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
That makes sense and confirms that the static site has a bug ... portions of my work are on hold until the SmalltalkHub issue is resolved and at least one other GemStone user is impacted, so far...
Dale
On 8/27/20 2:41 PM, Sven Van Caekenberghe wrote:
The static version of StHub seems to always assume the client is Pharo, while the dynamic version used format=raw (which non-Pharo implementation did not include in their request for the listing) to send the Pharo specific response only then.
On 27 Aug 2020, at 23:34, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
Depends upon how old that optimized code is ... as little as a 15 days ago, the last time my travis cron job ran successfully[1], the pharo code presumably was handling html page returns ... I'm pretty certain I haven't touched the Monticello HTTP handling code for nearly a decade:)
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/builds/717364651
On 8/27/20 1:44 PM, Sven Van Caekenberghe wrote:
Hmm, this is going to be a hard one.
SmalltalkHub got optimised in Pharo, consider
MCHttpRepository>>#parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [aStream atEnd] whileFalse: [[aStream upTo: $<. {$a. $A. nil} includes: aStream next] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName urlDecoded ]]. ^ names
vs.
MCSmalltalkHubRepository>>#parseFileNamesFromStream: aNewLineDelimitedString ^ aNewLineDelimitedString ifNil: [ ^ OrderedCollection new ] ifNotNil: [ aNewLineDelimitedString substrings: String crlf ]
In the old server code there was probably a way to detect what kind of client was making the request to determine how to respond. I am not sure a static server can do that (it is the format=raw query parameter, see MCSmalltalkHubRepository>>#loadAllFileNames). I also believe GZIP compressed files were returned in the optimised case.
BTW, there exists code to generate the listing in
ZnMonticelloRepository>>#repositoryListing ^ ZnHtmlOutputStream streamContents: [ :html | html page: 'Monticello Repository' do: [ html tag: #ul do: [ self mczEntries do: [ :each | html tag: #li do: [ html tag: #a attributes: { #href. each } with: each ] ] ] ] ]
Sven
On 27 Aug 2020, at 22:29, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
My guess is lies in the difference in the payload returned.
http://www.squeaksource.com/MooseSQL/ produces a html page:
<pnioijaecacnfagp.png>
and the static smalltalkhub site does not:
<lepkhlgeiolajoki.png>
I think that all of the monticello web sites return an html web page listing of packages and presumably the static site should produce html ... I'm sure that the dynamic version of smalltalkhub produced html pages as well and for now we are caught between a rock and a hard place ... the dynamic site is flakey and the static site breaks existing Monticello package list reading code:)
Dale
On 8/27/20 1:04 PM, Dale Henrichs wrote:
As I've started digging around, I have found that this url[1] does produce the correct list of mcz files in the browser, but is currently failing to produce any list at all in GLASS ... so there is a different mystery ... other than the fact that this url[1] was working prior(?) to the switchover (if in fact the DNS has propagated to all the right spots) and has been working for all of the other http Monticello repositories for over a decade:)
I will continue digging ...
Dale
[1] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main
On 8/27/20 12:48 PM, Dale Henrichs wrote:
Christophe,
There is a new(?) problem that we are having that has been reported in this thread on the GLASS list[1] where I am able to successfully download an mcz file [2], but get a `Not Found` error when I try to list the mcz files in a project[3]. The missing mcz list is consistent with the failed builds that I am now seeing on travis [4] and that are being reported by Brodbeck[1]. I have yet to get to a point where I can debug the problems directly and determine what is actually going on and of course I can't tell if these are the results of slow DNS propagation.
In this case [2][3], the list of file shows up on the dynamic(?) site:
<popmbnhcnehhanno.png>
and can be downloaded by pressing the download for the selected mcz file, but the missing list of packages[3] is likely to be the root cause of the problem.
Dale
[1] http://forum.world.st/SmalltalkHub-packages-not-accessible-tt5120932.html [2] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/Seaside-Swazoo-jf.19.m... [3] http://smalltalkhub.com/mc/Seaside/Seaside30LGPL [4] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523221#L2411
On 8/27/20 5:36 AM, Christophe Demarey wrote:
Hi Dale,
Sorry, I did not see your message before. Yesterday, I switched smalltalkhub to the static version (a bit earlier than announced) to avoid frequent downtimes we had with smalltalkhub. I did not measure but downloads should now be faster and reliable.
Do not hesitate to ping if you have any problem.
Cheers, Christophe
Le 26 août 2020 à 18:12, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Well, I haven't see any email response, but today (after two days of brokenness), http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz is now downloading successfully, so THANK YOU, to whoever fixed the problem!
Dale
On 8/25/20 9:02 AM, Dale Henrichs wrote:
SmalltalkHub mcz downloads are broken ... looks like a mongo server has gone down? .... I ran into this problem running production tests yesterday and today I find that while the smalltalkhub site is up, I cannot download an mcz file, using this url: http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz.
If you are not going to keep the current smalltalkhub site functional, why don't you switch to the static site and give those of us who DEPEND upon static access to mcz files a reliable site to connect to ... I have plans to move completely away from mcz files, but I didn't plan on doing that this week ... and frankly I don't have the cycles to do that ... right now ...
Here's a screenshot of a manual login and navigation to the mcz file that is failing to download:
<jchancldefkbdajd.png>
And when I press the `Download .mcz` button, I get the following "response" after a delay:
<lnonccpgaamnnobg.png>
Le 28 août 2020 à 10:15, Sven Van Caekenberghe <sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure?
See my earlier mail with code.
Thanks Sven
Christophe, I appreciate your effort, but I assume that you aren't finished yet? When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url): This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash. Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories). I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories. If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/) ... Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option. Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ... Dale On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe <sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? See my earlier mail with code. Thanks Sven
I understand that he new site is unusable if tools do not get the expected input. Sorry for the inconvenience. I was just not aware of this as Pharo uses the raw format. I had to set up a CGI to produce an html listing from the raw file. It should now be ok. Could you tell me if it works in GemStone now? Christophe
Le 28 août 2020 à 19:21, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Christophe,
I appreciate your effort, but I assume that you aren't finished yet?
When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url):
<dkbjiegjmlbelgio.png>
This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash.
Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories).
I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/ <http://www.squeaksource.com/MooseSQL/>) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories.
If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/ <http://www.squeaksource.com/MooseSQL/>) ...
Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option.
Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ...
Dale
On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe <sven@stfx.eu> <mailto:sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> <mailto:christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw <http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw> http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/ <http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/>
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? See my earlier mail with code. Thanks Sven
Thanks looks fine to me, Christophe, thank you for the great work. The non format=raw case results in a redirect but that should be acceptable, both URL works both and without trailing /
On 29 Aug 2020, at 00:05, Christophe Demarey <christophe.demarey@inria.fr> wrote:
I understand that he new site is unusable if tools do not get the expected input. Sorry for the inconvenience. I was just not aware of this as Pharo uses the raw format. I had to set up a CGI to produce an html listing from the raw file. It should now be ok. Could you tell me if it works in GemStone now?
Christophe
Le 28 août 2020 à 19:21, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Christophe,
I appreciate your effort, but I assume that you aren't finished yet?
When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url):
<dkbjiegjmlbelgio.png>
This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash.
Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories).
I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories.
If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/) ...
Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option.
Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ...
Dale
On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe <sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex:
http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw
http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure?
See my earlier mail with code.
Thanks Sven
Christophe, Again, I appreciate your effort, but you are not quite there. If you look at SqueakSource or GemSource html (attached), the html uses <br /> to create newlines on the page and in the Smalltalkhub page, you've inserted a newline as part of the href filename, while SqueakSource and GemSource (and the other monticello sites) do not include a newline in the href, so for better or worse, in GemStone the method MCReader>>canReadFileNamed: expects the filename to end with `.mcz` not a newline and all of the filenames parsed from the SmalltalkHub page are discarded as non-Monticello files. So if you eliminate the trailing newline in the href, the GemStone code should finally be happy. Thanks for your effort! Dale On 8/28/20 3:05 PM, Christophe Demarey wrote:
I understand that he new site is unusable if tools do not get the expected input. Sorry for the inconvenience. I was just not aware of this as Pharo uses the raw format. I had to set up a CGI to produce an html listing from the raw file. It should now be ok. Could you tell me if it works in GemStone now?
Christophe
Le 28 août 2020 à 19:21, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Christophe,
I appreciate your effort, but I assume that you aren't finished yet?
When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url):
<dkbjiegjmlbelgio.png>
This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash.
Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories).
I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories.
If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/) ...
Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option.
Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ...
Dale
On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe<sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey<christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? See my earlier mail with code. Thanks Sven
I hope this time it will be fine. I removed the newline characters. I will also check if I can keep the original url in the browser rather than the redirected one. Regards, Christophe.
Le 29 août 2020 à 18:41, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Christophe,
Again, I appreciate your effort, but you are not quite there.
If you look at SqueakSource or GemSource html (attached), the html uses <br /> to create newlines on the page and in the Smalltalkhub page, you've inserted a newline as part of the href filename, while SqueakSource and GemSource (and the other monticello sites) do not include a newline in the href, so for better or worse, in GemStone the method MCReader>>canReadFileNamed: expects the filename to end with `.mcz` not a newline and all of the filenames parsed from the SmalltalkHub page are discarded as non-Monticello files.
So if you eliminate the trailing newline in the href, the GemStone code should finally be happy.
Thanks for your effort!
Dale
On 8/28/20 3:05 PM, Christophe Demarey wrote:
I understand that he new site is unusable if tools do not get the expected input. Sorry for the inconvenience. I was just not aware of this as Pharo uses the raw format. I had to set up a CGI to produce an html listing from the raw file. It should now be ok. Could you tell me if it works in GemStone now?
Christophe
Le 28 août 2020 à 19:21, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Christophe,
I appreciate your effort, but I assume that you aren't finished yet?
When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url):
<dkbjiegjmlbelgio.png>
This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash.
Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories).
I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/ <http://www.squeaksource.com/MooseSQL/>) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories.
If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/ <http://www.squeaksource.com/MooseSQL/>) ...
Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option.
Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ...
Dale
On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe <sven@stfx.eu> <mailto:sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey <christophe.demarey@inria.fr> <mailto:christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw <http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw> http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/ <http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/>
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? See my earlier mail with code. Thanks Sven
<squeaksource.html><gemsource.html>
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main is being used (no trailing slash) and this form of the url still produces the `raw` listing, on SqueakSource (and other sites) the trailing slash is not required to get the html-based listing ... Again, I appreciate your effort, I hopeful that this is the last detail, Dale [1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220 On 8/29/20 10:12 AM, Christophe Demarey wrote:
I hope this time it will be fine. I removed the newline characters. I will also check if I can keep the original url in the browser rather than the redirected one.
Regards, Christophe.
Le 29 août 2020 à 18:41, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Christophe,
Again, I appreciate your effort, but you are not quite there.
If you look at SqueakSource or GemSource html (attached), the html uses <br /> to create newlines on the page and in the Smalltalkhub page, you've inserted a newline as part of the href filename, while SqueakSource and GemSource (and the other monticello sites) do not include a newline in the href, so for better or worse, in GemStone the method MCReader>>canReadFileNamed: expects the filename to end with `.mcz` not a newline and all of the filenames parsed from the SmalltalkHub page are discarded as non-Monticello files.
So if you eliminate the trailing newline in the href, the GemStone code should finally be happy.
Thanks for your effort!
Dale
On 8/28/20 3:05 PM, Christophe Demarey wrote:
I understand that he new site is unusable if tools do not get the expected input. Sorry for the inconvenience. I was just not aware of this as Pharo uses the raw format. I had to set up a CGI to produce an html listing from the raw file. It should now be ok. Could you tell me if it works in GemStone now?
Christophe
Le 28 août 2020 à 19:21, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Christophe,
I appreciate your effort, but I assume that you aren't finished yet?
When I use the url from your message I get a `Not Found` error (note that I explicitly included a trailing slash in the url):
<dkbjiegjmlbelgio.png>
This is a bug (or omission) as well, since other Monticello sites produce the expected HTML output with or without a trailing slash.
Without the trailing slash I get the same (non-html) response that was causing a problem before ... although something must have changed, because the GemStone command is now producing an error instead of an empty list ... Once you get the static site to produce the expected output, I'm confident that the GemStone errors will go away (this code has been running for about a decade on all Monticello html repositories).
I suggest that you look at what is produced on SqueakSource (http://www.squeaksource.com/MooseSQL/) as an example of expected output of the mcz file listing ... this html page format has been used since 2003 for ALL (valid) monticello repositories.
If it is not clear, the `?format=raw` option is a recent Pharo only option and when the dynamic site was running and the `?format-raw` option was OMITTED it produced output compatible with (http://www.squeaksource.com/MooseSQL/) ...
Restoring HTML is *REQUIRED* for the static site to be a faithful replacement of the dynamic site... the format that is produced today is only compatible with the Pharo only `?format=raw` option.
Bug or omission ... the static site is currently in worse shape than the dynamic site before the swap ...
Dale
On 8/28/20 2:12 AM, Christophe Demarey wrote:
Le 28 août 2020 à 10:15, Sven Van Caekenberghe<sven@stfx.eu> a écrit :
On 28 Aug 2020, at 10:13, Christophe Demarey<christophe.demarey@inria.fr> wrote:
Smalltalkhub is now to able to distinguish between raw and not raw mcz listing requests. Ex: http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main?format=raw http://smalltalkhub.com/mc/Seaside/Seaside30LGPL/main/
I already use the apache directory index for another page so I will not be able to modify this one. I guess the current index will « not work » because glass expect some HTML structure. Could you confirm and tell me what is the expected structure? See my earlier mail with code. Thanks Sven
<squeaksource.html><gemsource.html>
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main> is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main> produces the html listing
On 30 Aug 2020, at 12:07, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main produces the html listing
For me as well.
On 8/30/20 3:07 AM, Christophe Demarey wrote:
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main produces the html listing
I am sure that the travis build is still failing[1] after having passed 2 weeks ago, before the dynamic site was switched out. I am sure the a manual test (different code path than the travis site) that was failing with the new site is now passing. I did discovered that that the reason I _thought_, that the html page wasn't being displayed was because the old page for that url was cached in my web browser:( Sorry for the red herring. Tomorrow I will dig a little deeper and let you know what is causing the travis test to fail ... I've seen this particular error with the Swazoo repository show up in production tests that I run locally, so I should be able to identify the cause of the failures. Dale [1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220#L2354
Okay, I've finally sorted things out ... and it gets down to the difference between MCHttpRepository>>parseFileNamesFromStream: returning an empty list (the original problem) and returning a list of the wrong thing the current problem). When I reported that the "manual test" was passing, that was true, but instead of listing the names of the packages, the "manual test" was returning a list of urls, and not a list of packageNames and I didn't recognize the difference --- but the code that is executing in the travis test does recognize the difference. The GemStone code does not special case SmalltalkHub, while Pharo does handle SmalltalkHub as a special case and to actually see the difference between the two, you have to use the following expression (the MCHttpRepository class>>location:user:password: is intercepted and creates an MCSqueaksourceRepository instance, which has different implementation of parseFileNamesFromStream:): (MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main'; user: ''; password: ''; yourself) allFileNames Both Pharo and GemStone, use the pretty much same code for MCHttpRepository>>parseFileNamesFromStream: (this is the GemStone version): parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [ aStream atEnd ] whileFalse: [ [ aStream upTo: $<. aStream atEnd ifTrue: [ true ] ifFalse: [ {$a. $A. nil} includes: aStream next ] ] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName unescapePercents ] ]. ^ names and both return the same result when pointed at http://www.squeaksource.com/MooseSQL (you should recognize the EyeCollectionInspector from Pharo3.0:) and the other inspector is from tODE ... Note that both are producing a list of .mcz file names: And when the expression: (MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main'; user: ''; password: ''; yourself) allFileNames is used, both Pharo and GemStone produce the same (wrong) list of package names from SmalltalkHub: The travis test is trying to scan for ConfigurationOfSwazoo2 package names in the list and that algorithm is failing because it expects only a list of .mcz file names as produced like that produced for http://www.squeaksource.com/MooseSQL ... The difference boils down to how the `href` statements are formed. The Here is the relevant statement from SqueakSource: <a href="Moose-SQL-Importer-FabrizioPerin.22.mcz">Moose-SQL-Importer-FabrizioPerin.22.mcz</a> and here is the statement from SmalltalkHub with the full url included in the href field): <a href="http://smalltalkhub.com/Swazoo/Swazoo/mc/Swazoo-HTTP-avi.4.mcz">Swazoo-HTTP-avi.4.mcz</a> So only the .mcz filename should be included in the href field as the code in MCHttpRepository>>parseFileNamesFromStream: _is parsing the package name from the href field_. Dale On 8/30/20 7:31 PM, Dale Henrichs wrote:
On 8/30/20 3:07 AM, Christophe Demarey wrote:
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main produces the html listing
I am sure that the travis build is still failing[1] after having passed 2 weeks ago, before the dynamic site was switched out.
I am sure the a manual test (different code path than the travis site) that was failing with the new site is now passing.
I did discovered that that the reason I _thought_, that the html page wasn't being displayed was because the old page for that url was cached in my web browser:( Sorry for the red herring.
Tomorrow I will dig a little deeper and let you know what is causing the travis test to fail ... I've seen this particular error with the Swazoo repository show up in production tests that I run locally, so I should be able to identify the cause of the failures.
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220#L2354
Hi Dale, Thanks for the very detailed explanation. I updated the href link to only include the mcz file name. I also took the opportunity to display a nicer URL and set up a permanent redirection to the current mc repository folder (else links would be broken : we must keep only file names, paths prohibited). Tell me if it is ok. Also, I will not reinvent the past but I think It was a mistake to use an html listing as input for tools. A raw list of files contained in the Monticello repository would have been better.It is also strange to use an href link value as the name of the mcz file. If the displayed text of the link was used, it would have allow to store mcz files at any place on the web server. But anyway, if things work fine now, we are done! Cheers, Christophe
Le 31 août 2020 à 21:02, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Okay, I've finally sorted things out ... and it gets down to the difference between MCHttpRepository>>parseFileNamesFromStream: returning an empty list (the original problem) and returning a list of the wrong thing the current problem). When I reported that the "manual test" was passing, that was true, but instead of listing the names of the packages, the "manual test" was returning a list of urls, and not a list of packageNames and I didn't recognize the difference --- but the code that is executing in the travis test does recognize the difference.
The GemStone code does not special case SmalltalkHub, while Pharo does handle SmalltalkHub as a special case and to actually see the difference between the two, you have to use the following expression (the MCHttpRepository class>>location:user:password: is intercepted and creates an MCSqueaksourceRepository instance, which has different implementation of parseFileNamesFromStream:):
(MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main>'; user: ''; password: ''; yourself) allFileNames Both Pharo and GemStone, use the pretty much same code for MCHttpRepository>>parseFileNamesFromStream: (this is the GemStone version):
parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [ aStream atEnd ] whileFalse: [ [ aStream upTo: $<. aStream atEnd ifTrue: [ true ] ifFalse: [ {$a. $A. nil} includes: aStream next ] ] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName unescapePercents ] ]. ^ names and both return the same result when pointed at http://www.squeaksource.com/MooseSQL <http://www.squeaksource.com/MooseSQL> (you should recognize the EyeCollectionInspector from Pharo3.0:) and the other inspector is from tODE ... Note that both are producing a list of .mcz file names:
<lgkbcdpahkmomino.png>
And when the expression: (MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main>'; user: ''; password: ''; yourself) allFileNames
is used, both Pharo and GemStone produce the same (wrong) list of package names from SmalltalkHub:
<elmgecjolgkmfkgb.png>The travis test is trying to scan for ConfigurationOfSwazoo2 package names in the list and that algorithm is failing because it expects only a list of .mcz file names as produced like that produced for http://www.squeaksource.com/MooseSQL <http://www.squeaksource.com/MooseSQL> ...
The difference boils down to how the `href` statements are formed. The Here is the relevant statement from SqueakSource:
<a href="Moose-SQL-Importer-FabrizioPerin.22.mcz">Moose-SQL-Importer-FabrizioPerin.22.mcz</a>
and here is the statement from SmalltalkHub with the full url included in the href field):
<a href="http://smalltalkhub.com/Swazoo/Swazoo/mc/Swazoo-HTTP-avi.4.mcz" <http://smalltalkhub.com/Swazoo/Swazoo/mc/Swazoo-HTTP-avi.4.mcz>>Swazoo-HTTP-avi.4.mcz</a>
So only the .mcz filename should be included in the href field as the code in MCHttpRepository>>parseFileNamesFromStream: _is parsing the package name from the href field_.
Dale
On 8/30/20 7:31 PM, Dale Henrichs wrote:
On 8/30/20 3:07 AM, Christophe Demarey wrote:
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main> is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main <http://smalltalkhub.com/mc/Swazoo/Swazoo/main> produces the html listing I am sure that the travis build is still failing[1] after having passed 2 weeks ago, before the dynamic site was switched out.
I am sure the a manual test (different code path than the travis site) that was failing with the new site is now passing.
I did discovered that that the reason I _thought_, that the html page wasn't being displayed was because the old page for that url was cached in my web browser:( Sorry for the red herring.
Tomorrow I will dig a little deeper and let you know what is causing the travis test to fail ... I've seen this particular error with the Swazoo repository show up in production tests that I run locally, so I should be able to identify the cause of the failures.
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220#L2354 <https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220#L2354>
Hi everyone, not quite sure what's happening yet, but bootstrapping Metacello in Squeak will attempt to download http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz which currently returns a 302 to http://smalltalkhub.com/dkh/metacello/mc?format=raw-html which just returns the index page's HTML instead of the mcz file. Is this possibly a slip in the matching rules you added recently? Best, Tom On Tue, Sep 1, 2020 at 10:52 AM Christophe Demarey < christophe.demarey@inria.fr> wrote:
Hi Dale,
Thanks for the very detailed explanation. I updated the href link to only include the mcz file name. I also took the opportunity to display a nicer URL and set up a permanent redirection to the current mc repository folder (else links would be broken : we must keep only file names, paths prohibited). Tell me if it is ok.
Also, I will not reinvent the past but I think It was a mistake to use an html listing as input for tools. A raw list of files contained in the Monticello repository would have been better.It is also strange to use an href link value as the name of the mcz file. If the displayed text of the link was used, it would have allow to store mcz files at any place on the web server. But anyway, if things work fine now, we are done!
Cheers, Christophe
Le 31 août 2020 à 21:02, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Okay, I've finally sorted things out ... and it gets down to the difference between MCHttpRepository>>parseFileNamesFromStream: returning an empty list (the original problem) and returning a list of the wrong thing the current problem). When I reported that the "manual test" was passing, that was true, but instead of listing the names of the packages, the "manual test" was returning a list of urls, and not a list of packageNames and I didn't recognize the difference --- but the code that is executing in the travis test does recognize the difference.
The GemStone code does not special case SmalltalkHub, while Pharo does handle SmalltalkHub as a special case and to actually see the difference between the two, you have to use the following expression (the MCHttpRepository class>>location:user:password: is intercepted and creates an MCSqueaksourceRepository instance, which has different implementation of parseFileNamesFromStream:):
(MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main'; user: ''; password: ''; yourself) allFileNames
Both Pharo and GemStone, use the pretty much same code for MCHttpRepository>>parseFileNamesFromStream: (this is the GemStone version):
parseFileNamesFromStream: aStream | names fullName | names := OrderedCollection new. [ aStream atEnd ] whileFalse: [ [ aStream upTo: $<. aStream atEnd ifTrue: [ true ] ifFalse: [ {$a. $A. nil} includes: aStream next ] ] whileFalse. aStream upTo: $". aStream atEnd ifFalse: [ fullName := aStream upTo: $". names add: fullName unescapePercents ] ]. ^ names
and both return the same result when pointed at http://www.squeaksource.com/MooseSQL (you should recognize the EyeCollectionInspector from Pharo3.0:) and the other inspector is from tODE ... Note that both are producing a list of .mcz file names:
<lgkbcdpahkmomino.png> And when the expression:
(MCHttpRepository new location: 'http://smalltalkhub.com/mc/Swazoo/Swazoo/main'; user: ''; password: ''; yourself) allFileNames
is used, both Pharo and GemStone produce the same (wrong) list of package names from SmalltalkHub:
<elmgecjolgkmfkgb.png>The travis test is trying to scan for ConfigurationOfSwazoo2 package names in the list and that algorithm is failing because it expects only a list of .mcz file names as produced like that produced for http://www.squeaksource.com/MooseSQL ...
The difference boils down to how the `href` statements are formed. The Here is the relevant statement from SqueakSource:
<a href="Moose-SQL-Importer-FabrizioPerin.22.mcz">Moose-SQL-Importer-FabrizioPerin.22.mcz</a>
and here is the statement from SmalltalkHub with the full url included in the href field):
<a href="http://smalltalkhub.com/Swazoo/Swazoo/mc/Swazoo-HTTP-avi.4.mcz" <http://smalltalkhub.com/Swazoo/Swazoo/mc/Swazoo-HTTP-avi.4.mcz>
Swazoo-HTTP-avi.4.mcz</a>
So only the .mcz filename should be included in the href field as the code in MCHttpRepository>>parseFileNamesFromStream: _is parsing the package name from the href field_.
Dale On 8/30/20 7:31 PM, Dale Henrichs wrote:
On 8/30/20 3:07 AM, Christophe Demarey wrote:
Hi Dale,
Le 29 août 2020 à 19:40, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
Sooooo close, the interactive tests are passing, but there is a use case that is popping up in a travis job[1] where this url http://smalltalkhub.com/mc/Swazoo/Swazoo/main is being used (no trailing slash) and this form of the url still produces the `raw` listing,
Are you sure? For me, http://smalltalkhub.com/mc/Swazoo/Swazoo/main produces the html listing
I am sure that the travis build is still failing[1] after having passed 2 weeks ago, before the dynamic site was switched out.
I am sure the a manual test (different code path than the travis site) that was failing with the new site is now passing.
I did discovered that that the reason I _thought_, that the html page wasn't being displayed was because the old page for that url was cached in my web browser:( Sorry for the red herring.
Tomorrow I will dig a little deeper and let you know what is causing the travis test to fail ... I've seen this particular error with the Swazoo repository show up in production tests that I run locally, so I should be able to identify the cause of the failures.
Dale
[1] https://travis-ci.org/github/GsDevKit/GsDevKit_home/jobs/721523220#L2354
Hi Tom,
Le 1 sept. 2020 à 12:14, Tom Beckmann <tomjonabc@gmail.com> a écrit :
Hi everyone,
not quite sure what's happening yet, but bootstrapping Metacello in Squeak will attempt to download http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz <http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz> which currently returns a 302 to http://smalltalkhub.com/dkh/metacello/mc?format=raw-html <http://smalltalkhub.com/dkh/metacello/mc?format=raw-html> which just returns the index page's HTML instead of the mcz file.
Is this possibly a slip in the matching rules you added recently?-
I think so. I did not see it while testing in my browser (probably because of its cache) but I can see it with curl. I will fix that ASAP. Thanks for reporting. Christophe
Le 1 sept. 2020 à 12:21, Christophe Demarey <christophe.demarey@inria.fr> a écrit :
Hi Tom,
Le 1 sept. 2020 à 12:14, Tom Beckmann <tomjonabc@gmail.com <mailto:tomjonabc@gmail.com>> a écrit :
Hi everyone,
not quite sure what's happening yet, but bootstrapping Metacello in Squeak will attempt to download http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz <http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz> which currently returns a 302 to http://smalltalkhub.com/dkh/metacello/mc?format=raw-html <http://smalltalkhub.com/dkh/metacello/mc?format=raw-html> which just returns the index page's HTML instead of the mcz file.
Is this possibly a slip in the matching rules you added recently?-
I think so. I did not see it while testing in my browser (probably because of its cache) but I can see it with curl. I will fix that ASAP.
Should be ok now. Let me know.
Works perfectly, thank you for the rapid fix! Best, Tom On Tue, Sep 1, 2020 at 12:27 PM Christophe Demarey < christophe.demarey@inria.fr> wrote:
Le 1 sept. 2020 à 12:21, Christophe Demarey <christophe.demarey@inria.fr> a écrit :
Hi Tom,
Le 1 sept. 2020 à 12:14, Tom Beckmann <tomjonabc@gmail.com> a écrit :
Hi everyone,
not quite sure what's happening yet, but bootstrapping Metacello in Squeak will attempt to download http://smalltalkhub.com/mc/dkh/metacello/main/Metacello-Base-dkh.109.mcz which currently returns a 302 to http://smalltalkhub.com/dkh/metacello/mc?format=raw-html which just returns the index page's HTML instead of the mcz file.
Is this possibly a slip in the matching rules you added recently?-
I think so. I did not see it while testing in my browser (probably because of its cache) but I can see it with curl. I will fix that ASAP.
Should be ok now. Let me know.
On 9/1/20 1:51 AM, Christophe Demarey wrote:
Hi Dale,
Thanks for the very detailed explanation. I updated the href link to only include the mcz file name. I also took the opportunity to display a nicer URL and set up a permanent redirection to the current mc repository folder (else links would be broken : we must keep only file names, paths prohibited). Tell me if it is ok.
Christophe, Everything looks fine. The travis jobs that were failing because of the packageName issue are now failing because of test issues (you win some, you lose some:), so I think we are finally out of woods.
Also, I will not reinvent the past but I think It was a mistake to use an html listing as input for tools. A raw list of files contained in the Monticello repository would have been better.It is also strange to use an href link value as the name of the mcz file. If the displayed text of the link was used, it would have allow to store mcz files at any place on the web server. The decision to use html for list packages was a decision made almost 20 years ago, when Avi and Colin first invented Monticello and once code gets written and people start relying on that code, it gets REAL hard to change without creating pain for all of the users. This gets even more complicated when there are multiple, independent sites implemented using the old scheme. But anyway, if things work fine now, we are done!
I agree that it looks like we are done and I appreciate your patience and effort to resolve the issues. Dale
Le 1 sept. 2020 à 18:44, Dale Henrichs <dale.henrichs@gemtalksystems.com> a écrit :
On 9/1/20 1:51 AM, Christophe Demarey wrote:
Hi Dale,
Thanks for the very detailed explanation. I updated the href link to only include the mcz file name. I also took the opportunity to display a nicer URL and set up a permanent redirection to the current mc repository folder (else links would be broken : we must keep only file names, paths prohibited). Tell me if it is ok.
Christophe,
Everything looks fine. The travis jobs that were failing because of the packageName issue are now failing because of test issues (you win some, you lose some:), so I think we are finally out of woods.
Also, I will not reinvent the past but I think It was a mistake to use an html listing as input for tools. A raw list of files contained in the Monticello repository would have been better.It is also strange to use an href link value as the name of the mcz file. If the displayed text of the link was used, it would have allow to store mcz files at any place on the web server. The decision to use html for list packages was a decision made almost 20 years ago, when Avi and Colin first invented Monticello and once code gets written and people start relying on that code, it gets REAL hard to change without creating pain for all of the users. This gets even more complicated when there are multiple, independent sites implemented using the old scheme.
Yes, it is easy to say a decision was not so good afterwards. And, yes, when people use code in many places, it becomes harder to change this code.
But anyway, if things work fine now, we are done!
I agree that it looks like we are done and I appreciate your patience and effort to resolve the issues.
Sorry for the time GemStone tools were broken but I was not aware of te use of this HTML package listing this way. The goal of Smalltalk archive was / is to be a « smooth » substitution of Smalltalkhub, keeping existing URLs alive, even if redirected. This avoids a lot of changes in existing code. Regards, Christophe
participants (5)
-
Christophe Demarey -
Dale Henrichs -
Esteban Maringolo -
Sven Van Caekenberghe -
Tom Beckmann