Hi, I asked this before, but I still have trouble with this. What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config. Basically, $vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_ towards a private server. In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler): $ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux $ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui $ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)' $ ./pharo Pharo.image save build $ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp 3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline $ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z So although it gave a warning, it worked fine. The warning said, using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. so let's try that after cleaning everything: $ rm -rf pharo-local/ build.* $ ./pharo Pharo.image save build $ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer: WTF ? Why does this not work (I can perfectly git clone that URL) ? Of course, this would just be the first step, eventually I want to use my own private server with a URL like https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git How do you build/deploy non-public production code using the command line ? Help would be much appreciated. Sven
I am still at a loss with this issue. When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it. So obviously, Pharo 7 is capable of accessing it. However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ? This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command line ?
Help would be much appreciated.
Sven
Hi Sven, You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and metacello because the apis are not equivalent (specially having iceberg in the middle). If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier. If you execute first: ./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ Then your execution: ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp Will work without warnings. Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this: ./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ" Then you full script will be like this: ./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp I hope this solves your problem :) Esteban On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use
my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command
line ?
Help would be much appreciated.
Sven
Hi Esteban, Thanks for the reply, I will test all this as soon as I can and report back. However, in your final script, are you sure the host (or port) can be specified (so no defaulting to GitHub.com). Also, I was reading the implementation (hard), and it seemed as if the only real solution would be to make a new type (scheme), for this case, like githttps://user1:passw1@server.company.com/scm/bla/foo-pharo.git which would then preserve the username/password/host/port and do the proper thing directly. IIUC, https git URLs are described as the most common ones. It might be that there is then still a problem with branches/tags, but maybe they are not needed. Sven
On 29 Oct 2019, at 11:07, Esteban Lorenzano <estebanlm@gmail.com> wrote:
Hi Sven,
You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and metacello because the apis are not equivalent (specially having iceberg in the middle). If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier.
If you execute first:
./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ
Then your execution:
./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
Will work without warnings.
Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this:
./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ"
Then you full script will be like this:
./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
I hope this solves your problem :)
Esteban
On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command line ?
Help would be much appreciated.
Sven
On 29 Oct 2019, at 11:24, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Esteban,
Thanks for the reply, I will test all this as soon as I can and report back.
However, in your final script, are you sure the host (or port) can be specified (so no defaulting to GitHub.com).
Indeed it Is not needed, but I recommend to do it :)
Also, I was reading the implementation (hard), and it seemed as if the only real solution would be to make a new type (scheme), for this case, like
githttps://user1:passw1@server.company.com/scm/bla/foo-pharo.git
And thatâs why we use another solution more generic. Otherwise we will need to add GitHubHTTPS, GitLabHTTPS, etc., etc.
which would then preserve the username/password/host/port and do the proper thing directly. IIUC, https git URLs are described as the most common ones.
It might be that there is then still a problem with branches/tags, but maybe they are not needed.
Sven
On 29 Oct 2019, at 11:07, Esteban Lorenzano <estebanlm@gmail.com> wrote:
Hi Sven,
You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and metacello because the apis are not equivalent (specially having iceberg in the middle). If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier.
If you execute first:
./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ
Then your execution:
./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
Will work without warnings.
Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this:
./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ"
Then you full script will be like this:
./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
I hope this solves your problem :)
Esteban
On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command line ?
Help would be much appreciated.
Sven
On Tue, 29 Oct 2019 at 06:34, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 29 Oct 2019, at 11:24, Sven Van Caekenberghe <sven@stfx.eu> wrote: Also, I was reading the implementation (hard), and it seemed as if the only real solution would be to make a new type (scheme), for this case, like
githttps://user1:passw1@server.company.com/scm/bla/foo-pharo.git
And thatâs why we use another solution more generic. Otherwise we will need to add GitHubHTTPS, GitLabHTTPS, etc., etc.
I'm someone using 2FA on many systems (I use Authy for a key generator on my android phone) and I've been using it on GitHub now for years. I am forced to use the SSH approach since there is no viable Iceberg solution with HTTPS (there is actually a work-around with a GitHub application-defined key, but that is just a special password that isn't really 2FA IMO). If there's a redesign of the Iceberg API, it would be good to consider the 2FA dimension, too. Cris
which would then preserve the username/password/host/port and do the
proper thing directly. IIUC, https git URLs are described as the most common ones.
It might be that there is then still a problem with branches/tags, but
maybe they are not needed.
Sven
On 29 Oct 2019, at 11:07, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
Hi Sven,
You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and
metacello because the apis are not equivalent (specially having iceberg in the middle).
If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier.
If you execute first:
./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ
Then your execution:
./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
Will work without warnings.
Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this:
./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ"
Then you full script will be like this:
./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
I hope this solves your problem :)
Esteban
On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use
my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command
line ?
Help would be much appreciated.
Sven
-- Christopher Fuhrman, P.Eng., PhD *Professeur au Département de génie logiciel et des technologies de l'informationÃTS (Ãcole de technologie supérieure)*
Hi Esteban, I tried your snippets, see comments below.
On 29 Oct 2019, at 11:07, Esteban Lorenzano <estebanlm@gmail.com> wrote:
Hi Sven,
You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and metacello because the apis are not equivalent (specially having iceberg in the middle). If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier.
If you execute first:
./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ
Then your execution:
./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
Will work without warnings.
Yes, this now removes the warning (not that that was a real problem). I do not think I understand the implications of changing the remoteTypeSelector though.
Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this:
./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ"
Then you full script will be like this:
./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
I hope this solves your problem :)
This worked as well, but since ZTimestamp is a public GitHub.com project, I am not sure the authentication is really used. I tried with my GitHub.com credentials (username/password) and it loaded, but when I altered the password for a wrong one, it still worked ;-) Now, the problem is: how do I access the private, password protected repository of my company ? I tried to add a host to the github:// URL but I do not have the impression that this works. Since making this more concrete involves some sensitive information, I will send you a private email with the details of a test project/repository/user that I made. Sven PS: I want to stress that for me Iceberg works really well on my local development machine: I created the test project/code/baseline without encountering any issue (Pharo 7).
Esteban
On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command line ?
Help would be much appreciated.
Sven
Am 29.10.2019 um 17:16 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi Esteban,
I tried your snippets, see comments below.
On 29 Oct 2019, at 11:07, Esteban Lorenzano <estebanlm@gmail.com> wrote:
Hi Sven,
You are facing two different problems at the same time :) 1) a problem with Metacello api. There is a mismatch between git and metacello because the apis are not equivalent (specially having iceberg in the middle). If you want to use HTTPS in metacello you have to explicit you want to use HTTPS, but you still has to use the GitHub:// identifier.
If you execute first:
./pharo build.image eval --save "Iceberg remoteTypeSelector: #httpsUrlâ
Then your execution:
./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
Will work without warnings.
Yes, this now removes the warning (not that that was a real problem). I do not think I understand the implications of changing the remoteTypeSelector though.
Now, you have another problem: 2) You cannot use an url as this with iceberg/metacello: https://deploy:secret@.../scm/xyz/xyz-pharo.git And yes, we could do a parse of it and use it (also solving your metacello command problem), but this is not easily doable and it would take time. The usual way of doing this is not using HTTPS but distributing public/private keys (then you use the regular SSH protocol). But if you still want/need to use HTTPS, then you need to provide your plaintext credentials. For iceberg, your credentials are stored in your preferences directory, at credentials.fuel (so you could add them to your CI server, if you have them). To add them programatically you need to do something like this:
./pharo build.image eval --save "IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: âgithub.comâ"
Then you full script will be like this:
./pharo build.image eval --save â Iceberg remoteTypeSelector: #httpsUrlâ. IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'esteban'; password: 'secret'; yourself) forHostname: 'github.com' â ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
I hope this solves your problem :)
This worked as well, but since ZTimestamp is a public GitHub.com project, I am not sure the authentication is really used. I tried with my GitHub.com credentials (username/password) and it loaded, but when I altered the password for a wrong one, it still worked ;-)
Now, the problem is: how do I access the private, password protected repository of my company ?
I tried to add a host to the github:// URL but I do not have the impression that this works.
Since making this more concrete involves some sensitive information, I will send you a private email with the details of a test project/repository/user that I made.
Sven
PS: I want to stress that for me Iceberg works really well on my local development machine: I created the test project/code/baseline without encountering any issue (Pharo 7).
I like to second that. Working with iceberg is mostly ok. Where it is not it is often related more to git than to iceberg. But yes the deployment side has weaknesses. So my position on this that for production I want to reproducible load code which you cannot be sure when using iceberg with git and metacello (repos do not get updated automatically). This is the main reason I use https. Furthermore iceberg is no lightweight. So I want to have a process that I can use for small images as well where iceberg should be absent. Please read this as something to consider for future enhancement. Managing credentials is something that needs to be improved. In order to be able to load a dependent project from a private repo I recompile url methods to inject credentials. So yes a better way is highly welcome ;) Norbert
Esteban
On Mon, Oct 28, 2019 at 11:58 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: I am still at a loss with this issue.
When loading from github://svenvc/ztimestamp using Metacello (see original mail) the code in MCGitBasedNetworkRepository>>#createIcebergRepositoryWithFallbackFor:url: clearly switches to using https://github.com/svenvc/ztimestamp.git - I traced it.
So obviously, Pharo 7 is capable of accessing it.
However, starting directly with https://github.com/svenvc/ztimestamp.git I end up in a totally different place. Why ?
This code is so complex...
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I asked this before, but I still have trouble with this.
What I ultimately want is what I used for years now: use the zero config tools to point to my private repo and install my code and all its dependencies from a single Metacello config.
Basically,
$vm $image config _CONFIG_REPO_ _CONFIG_NAME_ --username=_CONFIG_USER_ --password=_CONFIG_PASS_ --install=_CONFIG_VERSION_ --group=_CONFIG_GROUP_
towards a private server.
In the new world of git(hub)/tonel/iceberg I am trying the following (not using the 'config' but the new 'metacello' command line handler):
$ uname -a Linux audio359 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ curl get.pharo.org/64/70+vm | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3054 100 3054 0 0 57622 0 --:--:-- --:--:-- --:--:-- 57622 Downloading the latest 70 Image: http://files.pharo.org/get-files/70/pharo64.zip Pharo.image Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui
$ ./pharo Pharo.image printVersion [version] 'Pharo7.0.4' 'Pharo-7.0.4+build.168.sha.ccd1f6489120f58ddeacb2cac77cd3a0f0dcfbe6 (64 Bit)'
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install github://svenvc/ztimestamp BaselineOfZTimestamp
3. using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant. Fetched -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> BaselineOfZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loading baseline of BaselineOfZTimestamp... Fetched -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- https://github.com/svenvc/ztimestamp.git[master] Loaded -> ZTimestamp-CompatibleUserName.1554735117 --- https://github.com/svenvc/ztimestamp.git[master] --- cache ...finished baseline
$ ./pharo build.image eval ZTimestamp now 2019-10-25T12:28:34Z
So although it gave a warning, it worked fine.
The warning said,
using HTTPS instead SSH (Just use an url in the form HTTPS://etc.git). I will try to clone the HTTPS variant.
so let's try that after cleaning everything:
$ rm -rf pharo-local/ build.*
$ ./pharo Pharo.image save build
$ ./pharo build.image metacello install https://github.com/svenvc/ztimestamp.git BaselineOfZTimestamp
...RETRY->BaselineOfZTimestamp ...RETRY->BaselineOfZTimestamp ...FAILED->BaselineOfZTimestampCould not resolve: BaselineOfZTimestamp [BaselineOfZTimestamp] in /home/sven/pharo7/pharo-local/package-cache https://github.com/svenvc/ztimestamp.git MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
WTF ? Why does this not work (I can perfectly git clone that URL) ?
Of course, this would just be the first step, eventually I want to use my own private server with a URL like
https://deploy:secret@bitbucket.myserver.be/scm/xyz/xyz-pharo.git
How do you build/deploy non-public production code using the command line ?
Help would be much appreciated.
Sven
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
How do you build/deploy non-public production code using the command line ?
I just learned about the following technique: Using SSH agent forwarding https://developer.github.com/v3/guides/using-ssh-agent-forwarding/ This certainly makes using certificates on production servers much easier since you do no longer have to manage or install special ones on the deployment servers, you can just use your standard developer certificates. Sven
Am 31.10.2019 um 12:12 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
How do you build/deploy non-public production code using the command line ?
I just learned about the following technique:
Using SSH agent forwarding
https://developer.github.com/v3/guides/using-ssh-agent-forwarding/
This certainly makes using certificates on production servers much easier since you do no longer have to manage or install special ones on the deployment servers, you can just use your standard developer certificates.
That is quite late :P I always use this as it also chains through all the machins you log in. We use jenkins for building our software there it does not work. We just have a deployment key for that. That is a key pair without passphrase that is registered in the git repository and installed on the jenkins server. Norbert
On 31 Oct 2019, at 12:18, Norbert Hartl <norbert@hartl.name> wrote:
Am 31.10.2019 um 12:12 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
How do you build/deploy non-public production code using the command line ?
I just learned about the following technique:
Using SSH agent forwarding
https://developer.github.com/v3/guides/using-ssh-agent-forwarding/
This certainly makes using certificates on production servers much easier since you do no longer have to manage or install special ones on the deployment servers, you can just use your standard developer certificates.
That is quite late :P I always use this as it also chains through all the machins you log in. We use jenkins for building our software there it does not work. We just have a deployment key for that. That is a key pair without passphrase that is registered in the git repository and installed on the jenkins server.
Norbert
Indeed, the SSH forwarding cannot be used for independent CI builds. I am curious though: how do you do the initial checkout (clone) from your private repository ? Using regular git command line tools or using Iceberg/Metacello ? If the latter, how do you specify your URL, exactly ? The first case works for me too, using a gitlocal:// URL, but that is cheating a bit, I feel like we should be able to do this directly in Pharo. Sven
Am 01.11.2019 um 17:44 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 31 Oct 2019, at 12:18, Norbert Hartl <norbert@hartl.name> wrote:
Am 31.10.2019 um 12:12 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
How do you build/deploy non-public production code using the command line ?
I just learned about the following technique:
Using SSH agent forwarding
https://developer.github.com/v3/guides/using-ssh-agent-forwarding/
This certainly makes using certificates on production servers much easier since you do no longer have to manage or install special ones on the deployment servers, you can just use your standard developer certificates.
That is quite late :P I always use this as it also chains through all the machins you log in. We use jenkins for building our software there it does not work. We just have a deployment key for that. That is a key pair without passphrase that is registered in the git repository and installed on the jenkins server.
Norbert
Indeed, the SSH forwarding cannot be used for independent CI builds.
I am curious though: how do you do the initial checkout (clone) from your private repository ? Using regular git command line tools or using Iceberg/Metacello ?
If the latter, how do you specify your URL, exactly ?
The first case works for me too, using a gitlocal:// URL, but that is cheating a bit, I feel like we should be able to do this directly in Pharo.
If checking out from private repo your problem is as deep as you reference private repos. My project (1) is in a private repository, it contains a baseline that references a project in a private repository (2) and this references a project in a private repo (3). * Project (1) is checked out by jenkins. I have a deployment key pair without passphrase where the public key is registered in the bitbucket project and the private key is deployed on jenkins. * I try to avoid iceberg as much as possible for deployment builds because later I want to have an image without iceberg. And it is not reliable to use iceberg because metacello won't update git repos automatically and the odds are high that you load old content. So I load the project with the eval commandline handler with Metacello new repository: 'filetree://source'; baseline ⦠⦠* In order to load the private projects in the the baseline of the main project I run a script before build that hacks the image in a way that MCBitbucketRepository compile: 'httpsUrl ^ ''https://USER:PASS@bitbucket.org/'', projectPath, ''.git'''. MCBitbucketRepository compile: 'projectZipUrlFor: projectPath versionString: versionString ^ ''https://USER:PASS@bitbucket.org/'' , projectPath , ''/get/'' , versionString , ''.zip'''. MCBitbucketRepository class compile: 'projectZipUrlFor: projectPath versionString: versionString ^ ''https://USER:PASS@bitbucket.org/'' , projectPath , ''/get/'' , versionString , ''.zip'''. This works only because all private projects are on bitbucket and no public one. * Project (3) I made public because I did not see how to make that work Hope this helps. I feel this to be far too complicated. Together with the confusion between pharo5/pharo6/pharo7 with iceberg, tonel and iceberg and/or tonel support killed two weeks of my time at least. So I would welcome if there would be a better way to manage credentials Norbert
I made a PR to allow âgenericâ git repositories to be referenced: https://github.com/pharo-vcs/iceberg/pull/1324 <https://github.com/pharo-vcs/iceberg/pull/1324> But⦠problem is more complicated than you may think: Metacello âvanillaâ relies on download a zip file with the sources, it cannot have a âgenericâ way to act. This means the PR will work just if you use iceberg. And maybe you do not want to declare your baselines relying on iceberg. In that case, you have just one workaround possible: 1) clone locally the remove private sources outside the image. 2) load them using plain metacello and filetree://. 3) âlockâ those repositories. Then you can run your installation. Esteban
On 4 Nov 2019, at 12:52, Norbert Hartl <norbert@hartl.name> wrote:
Am 01.11.2019 um 17:44 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 31 Oct 2019, at 12:18, Norbert Hartl <norbert@hartl.name> wrote:
Am 31.10.2019 um 12:12 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 25 Oct 2019, at 14:49, Sven Van Caekenberghe <sven@stfx.eu> wrote:
How do you build/deploy non-public production code using the command line ?
I just learned about the following technique:
Using SSH agent forwarding
https://developer.github.com/v3/guides/using-ssh-agent-forwarding/
This certainly makes using certificates on production servers much easier since you do no longer have to manage or install special ones on the deployment servers, you can just use your standard developer certificates.
That is quite late :P I always use this as it also chains through all the machins you log in. We use jenkins for building our software there it does not work. We just have a deployment key for that. That is a key pair without passphrase that is registered in the git repository and installed on the jenkins server.
Norbert
Indeed, the SSH forwarding cannot be used for independent CI builds.
I am curious though: how do you do the initial checkout (clone) from your private repository ? Using regular git command line tools or using Iceberg/Metacello ?
If the latter, how do you specify your URL, exactly ?
The first case works for me too, using a gitlocal:// URL, but that is cheating a bit, I feel like we should be able to do this directly in Pharo.
If checking out from private repo your problem is as deep as you reference private repos. My project (1) is in a private repository, it contains a baseline that references a project in a private repository (2) and this references a project in a private repo (3).
* Project (1) is checked out by jenkins. I have a deployment key pair without passphrase where the public key is registered in the bitbucket project and the private key is deployed on jenkins.
* I try to avoid iceberg as much as possible for deployment builds because later I want to have an image without iceberg. And it is not reliable to use iceberg because metacello won't update git repos automatically and the odds are high that you load old content. So I load the project with the eval commandline handler with
Metacello new repository: 'filetree://source <filetree://source>'; baseline ⦠â¦
* In order to load the private projects in the the baseline of the main project I run a script before build that hacks the image in a way that
MCBitbucketRepository compile: 'httpsUrl ^ ''https://USER:PASS@bitbucket.org/'', projectPath, ''.git'' <https://USER:PASS@bitbucket.org/'',%20projectPath,%20''.git''>'.
MCBitbucketRepository compile: 'projectZipUrlFor: projectPath versionString: versionString ^ ''https://USER:PASS@bitbucket.org/'' , projectPath , ''/get/'' , versionString , ''.zip'' <https://USER:PASS@bitbucket.org/''%20,%20projectPath%20,%20''/get/''%20,%20versionString%20,%20''.zip''>'.
MCBitbucketRepository class compile: 'projectZipUrlFor: projectPath versionString: versionString ^ ''https://USER:PASS@bitbucket.org/'' , projectPath , ''/get/'' , versionString , ''.zip'' <https://USER:PASS@bitbucket.org/''%20,%20projectPath%20,%20''/get/''%20,%20versionString%20,%20''.zip''>'.
This works only because all private projects are on bitbucket and no public one.
* Project (3) I made public because I did not see how to make that work
Hope this helps. I feel this to be far too complicated. Together with the confusion between pharo5/pharo6/pharo7 with iceberg, tonel and iceberg and/or tonel support killed two weeks of my time at least. So I would welcome if there would be a better way to manage credentials
Norbert
participants (4)
-
Christopher Fuhrman -
Esteban Lorenzano -
Norbert Hartl -
Sven Van Caekenberghe