Why doesn't Iceberg checkin other assets (scripts) but does check them out?
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use). One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer. It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later). As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?). It strikes me that this is the kind of thing that git integration should bring to us? I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do? Tim
On 13/06/2018 11:50, Tim Mackinnon wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
[SNIP]
It strikes me that this is the kind of thing that git integration should bring to us?
Of the mentioned benefits of using Git (or any other file based SCM) was exactly that of versioning "external" assets. You could do that with filetree/gitfiletree. But I guess that given then tight integration of Iceberg with git and the Pharo workflow, to "sync" ("commit") external files, these should be visible to Iceberg as well, and should be added for staging on each commit, which means that Iceberg should have a "file explorer" of some sort integrated into its GUI. Regards, -- Esteban A. Maringolo
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC. at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind: Image -> Disk -> Git blob (database) you need to have this other: Image \ Git blob (database) Disk / you will see as soon as you change the mental image, your problems are gone ;) cheers! Esteban ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob? Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit? Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?) OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit? I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.? Tim Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could. Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
Just to complement Esteban's answer: - Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :) - About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right? - Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image! Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...) So far we cover the bare minimum that allows us to *not lose* changes :) On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick). However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now? I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically? Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22. This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)? Does this make sense? As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either. Tim Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
Wait... so it is no longer possible to #addtoIndex: external files from Pharo? I thought that this functionality was supposed to be preserved. Peter On Thu, Jun 14, 2018 at 10:30 AM, Tim Mackinnon <tim@testit.works> wrote:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*
*Web:* *http://guillep.github.io* <http://guillep.github.io>
*Phone: *+33 06 52 70 66 13
Ooooh - I didnât know about #addToIndex: that sounds like a possible workaround to my problem (but not a generic solution though). BTW - I donât want this to be a criticism, Iâm loving the tools and their possibilities! Tim Sent from my iPhone
On 14 Jun 2018, at 09:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
Wait... so it is no longer possible to #addtoIndex: external files from Pharo? I thought that this functionality was supposed to be preserved.
Peter
On Thu, Jun 14, 2018 at 10:30 AM, Tim Mackinnon <tim@testit.works> wrote: Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
addFileToIndex: and addFilesToIndex: are still there... We of course have support for changing and committing arbitrary files (actually, pharo code at the end are just arbitrary files!). The difficult part is to bring a consistent UI to life for it... On Thu, Jun 14, 2018 at 10:43 AM Tim Mackinnon <tim@testit.works> wrote:
Ooooh - I didnât know about #addToIndex: that sounds like a possible workaround to my problem (but not a generic solution though).
BTW - I donât want this to be a criticism, Iâm loving the tools and their possibilities!
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
Wait... so it is no longer possible to #addtoIndex: external files from Pharo? I thought that this functionality was supposed to be preserved.
Peter
On Thu, Jun 14, 2018 at 10:30 AM, Tim Mackinnon <tim@testit.works> wrote:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*
*Web:* *http://guillep.github.io* <http://guillep.github.io>
*Phone: *+33 06 52 70 66 13
-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
BTW, addFilesToIndex: will not add files to the git index immediately, it will do it just before a commit only. On Thu, Jun 14, 2018 at 11:03 AM Guillermo Polito <guillermopolito@gmail.com> wrote:
addFileToIndex: and addFilesToIndex: are still there...
We of course have support for changing and committing arbitrary files (actually, pharo code at the end are just arbitrary files!). The difficult part is to bring a consistent UI to life for it...
On Thu, Jun 14, 2018 at 10:43 AM Tim Mackinnon <tim@testit.works> wrote:
Ooooh - I didnât know about #addToIndex: that sounds like a possible workaround to my problem (but not a generic solution though).
BTW - I donât want this to be a criticism, Iâm loving the tools and their possibilities!
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
Wait... so it is no longer possible to #addtoIndex: external files from Pharo? I thought that this functionality was supposed to be preserved.
Peter
On Thu, Jun 14, 2018 at 10:30 AM, Tim Mackinnon <tim@testit.works> wrote:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I
could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*
*Web:* *http://guillep.github.io* <http://guillep.github.io>
*Phone: *+33 06 52 70 66 13
--
Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*
*Web:* *http://guillep.github.io* <http://guillep.github.io>
*Phone: *+33 06 52 70 66 13
-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this? Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
Hi Norbert, Tim, 2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2. I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync. IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project. Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
Yes - Iâm a fan of small checkins (not always great at it, but I strive for it) - and yes, the issue Iâve had is the same as what Thierry mentions - my Smalltalk code is tied to an external file which has some rewrite rules that depend on the classes I have in my image. I could possibly generate them from the Smalltalk code (this is something Sean does) - but I can see all kinds of reasons to atomically checkin code together. It havenât tried it - but Iâm guessing if I stick non smalltalk things in my src directory, that might work (ugly and potentially confusing, but it might do the trick). OR Iâve just learned about #addFilesToIndex: which might help me in this case. But as a more general (and easier to understand) solution - working well in a polyglot arena and checking other stuff in would be very handy. Hence my âstagingâ proposal (if it would work?). Alternatively, I will try and see what happens if I commit locally (without push) and then use my external tool to commit other files and then do the final push t(talking about it here helped me spot what could be an easy workaround). Still, I do hope we can keep working on the tools to get to something a bit more sophisticated - as the full atomic commit is the obvious end goal. Tim
On 14 Jun 2018, at 12:12, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
2018-06-14 15:39 GMT+02:00 Tim Mackinnon <tim@testit.works>:
Yes - Iâm a fan of small checkins (not always great at it, but I strive for it) - and yes, the issue Iâve had is the same as what Thierry mentions - my Smalltalk code is tied to an external file which has some rewrite rules that depend on the classes I have in my image. I could possibly generate them from the Smalltalk code (this is something Sean does) - but I can see all kinds of reasons to atomically checkin code together.
And generating is not allways possible.
It havenât tried it - but Iâm guessing if I stick non smalltalk things in my src directory, that might work (ugly and potentially confusing, but it might do the trick). OR Iâve just learned about #addFilesToIndex: which might help me in this case.
I wouldn't put things in the src directory. FileTree, for one, is implemented by "I erase everything on-disk, then I write back my package", which has a non-negligeable chance of erasing non-Smalltalk data in there. The addFilesToIndex: should work, but the question becomes what is the hook to automatically call it when saving your project with Iceberg.
But as a more general (and easier to understand) solution - working well in a polyglot arena and checking other stuff in would be very handy. Hence my âstagingâ proposal (if it would work?).
Probably, yes.
Alternatively, I will try and see what happens if I commit locally (without push) and then use my external tool to commit other files and then do the final push t(talking about it here helped me spot what could be an easy workaround). Still, I do hope we can keep working on the tools to get to something a bit more sophisticated - as the full atomic commit is the obvious end goal.
Consider using git squash, then. And calling git squash could be automated. One run the risk of turning Iceberg in a complete language-agnostic git client, which could make it too big to be sustainable in the long term. Thierry
Tim
On 14 Jun 2018, at 12:12, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com>:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
On 15 Jun 2018, at 08:11, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
+1 git favours many small commits. Esteban
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
2018-06-15 8:19 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
On 15 Jun 2018, at 08:11, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com
:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
+1 git favours many small commits.
.. and suggests you use rebase, squash and etc... to make sense of those many small commits. Thierry
Esteban
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
2018-06-15 8:11 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com
:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
That may be. But I'm not used to consider co-working on the same branch with my HW engineer as "deployment"... Thierry
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com>
wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very
satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its
something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I
change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could
checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration
should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg
directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr/>*
*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit. Of course you can choose to work a different way - but our tools shouldnât force a very common practice to be awkward, should they? I donât understand why many of you guys are so anti such a common thing. In fact our community pioneered this way of working - a build machine where you loaded all your code and ran all your tests before committing. Also, a large number of agile teams will council to work on master (and avoid branches - a technique we also pioneered). In todayâs world where we live in a wider polyglot stack this is one of the reasons why git is so attractive as it should easily embraces that. I get that itâs not quite possible yet - but there are some decent ways to accommodate this (a staging operation, or perhaps a better way to make use of the #addToIndex: which I will experiment with). Anyway, Iâll shut up - and possibly will learn how to implement it myself. Thinking more, I may also be able run with commit and no push , which I will try - as itâs probably the easiest mechanism if the CI will recognise that (I think it will). Thanks for the enlightening conversation, and as always the hard work on iceberg that made this conversation even possible. Tim Sent from my iPhone
On 15 Jun 2018, at 07:11, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com>:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr
Web: http://guillep.github.io Phone: +33 06 52 70 66 13
Hi Tim, On Fri, Jun 15, 2018 at 10:31 AM Tim Mackinnon <tim@testit.works> wrote:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
Of course you can choose to work a different way - but our tools shouldnât force a very common practice to be awkward, should they?
I donât understand why many of you guys are so anti such a common thing. In fact our community pioneered this way of working - a build machine where you loaded all your code and ran all your tests before committing. Also, a large number of agile teams will council to work on master (and avoid branches - a technique we also pioneered). In todayâs world where we live in a wider polyglot stack this is one of the reasons why git is so attractive as it should easily embraces that.
I get that itâs not quite possible yet - but there are some decent ways to accommodate this (a staging operation, or perhaps a better way to make use of the #addToIndex: which I will experiment with).
Anyway, Iâll shut up - and possibly will learn how to implement it myself. Thinking more, I may also be able run with commit and no push , which I will try - as itâs probably the easiest mechanism if the CI will recognise that (I think it will).
Please don't shut up ^^. I really value the feedback. I see myself the value of both small and atomic/self-contained/complete commits. In general what happens is that we are few modifying iceberg. But on the other side we really had given a thought to many of these topics, and many decisions had a reason (even if it is that it is not implemented because it would be too much work :)). I try to explain them as better as I can, I can probably do better :)
On 15 Jun 2018, at 10:29, Tim Mackinnon <tim@testit.works> wrote:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
but then, you mean every *push* should be shippable :) thatâs something I agree :)
Of course you can choose to work a different way - but our tools shouldnât force a very common practice to be awkward, should they?
I donât understand why many of you guys are so anti such a common thing.
Iâm not anti. Being âantiâ requires a lot of effort I prefer to spend on something else :) But in general, I prefer many small commits of tasks. And yes, I prefer those commits to be atomic, but when is not possible I just make sure I do not push until something is finished.
In fact our community pioneered this way of working - a build machine where you loaded all your code and ran all your tests before committing. Also, a large number of agile teams will council to work on master (and avoid branches - a technique we also pioneered).
hah⦠we pioneered it as a technique basically because our tools were not capable of branching properly, but well, It can be a way to minimise integration problems, yes (depends on the team, of course).
In todayâs world where we live in a wider polyglot stack this is one of the reasons why git is so attractive as it should easily embraces that.
I get that itâs not quite possible yet - but there are some decent ways to accommodate this (a staging operation, or perhaps a better way to make use of the #addToIndex: which I will experiment with).
I think the easiest/fastest way to handle this is to add a âspecial commitâ plugin which could export sources and then delegate commit to an external tool. And it could be a plugin because then people wanting to work with it can load it independently.
Anyway, Iâll shut up - and possibly will learn how to implement it myself. Thinking more, I may also be able run with commit and no push , which I will try - as itâs probably the easiest mechanism if the CI will recognise that (I think it will).
do not shut up :) your opinions and considerations are good and a good way to see a different pov that may be better than ours.
Thanks for the enlightening conversation, and as always the hard work on iceberg that made this conversation even possible.
thank you Esteban
Tim
Sent from my iPhone
On 15 Jun 2018, at 07:11, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
On Jun 15, 2018, at 2:05 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 15 Jun 2018, at 10:29, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
but then, you mean every *push* should be shippable :) thatâs something I agree :)
The way we run projects - pushing to master is forbidden. You push your work to a feature specific branch, a CI server runs all the tests in the branch every push. We create pull requests from the github project page, people review them and when the tests are green and reviewers approve, we perform the merge on github. Never touch master (or whatever branch you pick for main trunk) outside of github. Always perform new work in new branch.
Tod/Norbert - I wonât shy away from answering your comments on this - but if its ok - lets move that conversation to a new thread. I donât want to distract away from how one actually does some form of atomic checkin (and Iâm talking a small one - one Smalltalk class, one associated text file). I earlier tried doing a change on a class - committing, but not pushing and then switching to IntelliJ to then commit and push my config file change - and it didnât seem to work - which surprised me, so Iâm gong to redo my test again with the the latest Iceberg 1.1 in a new image (and also make sure my IntelliJ is attached to the same local repo). This seems like the closest root to how I would like to work (albeit a slightly bit more complicated than I would like, but certainly doable). Tim
On 15 Jun 2018, at 18:26, Todd Blanchard <tblanchard@mac.com> wrote:
On Jun 15, 2018, at 2:05 AM, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 15 Jun 2018, at 10:29, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
but then, you mean every *push* should be shippable :) thatâs something I agree :)
The way we run projects - pushing to master is forbidden. You push your work to a feature specific branch, a CI server runs all the tests in the branch every push. We create pull requests from the github project page, people review them and when the tests are green and reviewers approve, we perform the merge on github.
Never touch master (or whatever branch you pick for main trunk) outside of github. Always perform new work in new branch.
Phew - I can confirm that (unsurprisingly I guess) - just committing in Iceberg (without ticking - push changes to origin/master) does then let you easily overlay the same checked out project (typically ./iceberg/ProjectName) in a tool like VSCode or IntelliJ where you can then also commit further changes to files that Iceberg is ignoring (essentially everything not in the src directory - typically candidates would be your Readme.md file and possibly a ./scripts or ./assets directory). In that other tool you can then issue a push which will then atomically push all commits to a pipeline. It can be a little more tricky to understand what the pipeline is building - in Gitlab it will show you the comment of the most recent commit in the pipeline, however if you then click on the branch the pipeline is running on, you will get a list of commits and can see all the commits that are between the running pipeline and the previously run pipeline. For many of you - this might be pleadingly obvious - but when you have a different workflow mindset in your head, it might be trickier to spot this subtlety. Tim p.s. Thanks for all the useful debate in this thread. I should add that when I wrote âIâll shut upâ - a meant more that I felt the conversation was going in a cycle and repeating itself, not that I was offended or frustrated. I can probably live with the above nuance for my desire for atomic commits. p.p.s It hadnât occurred to me that the most common use case where people will hit a similar problem to me - is that Readme.md file. Definitely that is one where it should possibly automatically added to the index, or catered for - as it is really just a text file right?
On 15 Jun 2018, at 18:26, Todd Blanchard <tblanchard@mac.com> wrote:
On Jun 15, 2018, at 2:05 AM, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 15 Jun 2018, at 10:29, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
but then, you mean every *push* should be shippable :) thatâs something I agree :)
The way we run projects - pushing to master is forbidden. You push your work to a feature specific branch, a CI server runs all the tests in the branch every push. We create pull requests from the github project page, people review them and when the tests are green and reviewers approve, we perform the merge on github.
Never touch master (or whatever branch you pick for main trunk) outside of github. Always perform new work in new branch.
On 15/06/2018 20:00, Tim Mackinnon wrote:
Phew - I can confirm that (unsurprisingly I guess) - just committing in Iceberg (without ticking - push changes to origin/master) does then let you easily overlay the same checked out project (typically ./iceberg/ProjectName) in a tool like VSCode or IntelliJ where you can then also commit further changes to files that Iceberg is ignoring (essentially everything not in the src directory - typically candidates would be your Readme.md file and possibly a ./scripts or ./assets directory).
That could work as a *workaround* to a design decision that made the image the "staging" of a git commit.
In that other tool you can then issue a push which will then atomically push all commits to a pipeline. It can be a little more tricky to understand what the pipeline is building - in Gitlab it will show you the comment of the most recent commit in the pipeline, however if you then click on the branch the pipeline is running on, you will get a list of commits and can see all the commits that are between the running pipeline and the previously run pipeline.
For many of you - this might be pleadingly obvious - but when you have a different workflow mindset in your head, it might be trickier to spot this subtlety.
But it's not what I'd expect when using git, in particular if you use commit commands and/or CI, because if you add a comment like like "Fixes Issue #2345" in your Iceberg commit, then your secondary commit (the one done "externally" will look detached, you should replicate the commit comment, so your CI will show the comment of the commit on which it's building, etc. Git favors small commits, but other tools expect atomic commits. They're not exclusive.
p.s. Thanks for all the useful debate in this thread. I should add that when I wrote âIâll shut upâ - a meant more that I felt the conversation was going in a cycle and repeating itself, not that I was offended or frustrated. I can probably live with the above nuance for my desire for atomic commits.
We're grown ups trying to improve Pharo, so unless stated otherwise or clear agressions are visible (ad-hominems, passive agressiveness, etc.), it's healthier to assume all comments are honest, constructive and with good intentions. Let's avoid the meta-disussion. :) Regards, -- Esteban A. Maringolo
Am 15.06.2018 um 10:29 schrieb Tim Mackinnon <tim@testit.works>:
In many parts of the dev world - every commit should be shippable, meaning atomic - particularly with the use of CI systems (Travis, Gitlab etc) that build on every commit.
To be honest I donât like such comments. Do you have a proof for âmany parts of the worldâ? I do not get why every commit should be shippable. To me it sounds like a misplaced constraint. And for every project that is not pretty simple I doubt it can work at all. Commits are commits and a shippable product is shippable product not much in common.
Of course you can choose to work a different way - but our tools shouldnât force a very common practice to be awkward, should they?
Again. Where is this common practice? You cannot just state such things without proof in order to back your argument. I can see that in some scenarios it comes in handy that such commits ease some things. But I do not agree to see that this is or should be the common case.
I donât understand why many of you guys are so anti such a common thing. In fact our community pioneered this way of working - a build machine where you loaded all your code and ran all your tests before committing. Also, a large number of agile teams will council to work on master (and avoid branches - a technique we also pioneered). In todayâs world where we live in a wider polyglot stack this is one of the reasons why git is so attractive as it should easily embraces that.
I donât think I am anti. I donât get your point making this a common thing. I think that wanting commits spawning multiple repos is much more common.
I get that itâs not quite possible yet - but there are some decent ways to accommodate this (a staging operation, or perhaps a better way to make use of the #addToIndex: which I will experiment with).
Anyway, Iâll shut up - and possibly will learn how to implement it myself. Thinking more, I may also be able run with commit and no push , which I will try - as itâs probably the easiest mechanism if the CI will recognise that (I think it will).
No, do no shut up. Just proof your point. We cannot support every possible use case. Norbert
Thanks for the enlightening conversation, and as always the hard work on iceberg that made this conversation even possible.
Tim
Sent from my iPhone
On 15 Jun 2018, at 07:11, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Hi Norbert, Tim,
2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>>:
Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>:
Hi - yes Iâm pleased you check out the entire tree, although currently itâs a bit confusing that you do (fortunately this does give the possibility that we can checkout images and other resources that an Pharo application might rely on - without having to resort to the Seaside FileLibrary trick).
However my concrete case was that I have a gitlab ci pipeline and next to my src directory in my project I have a config directory that has some Nginx config for my teapot app. If I add a teapot route, I also need to adjust that config and check both changes in together. I canât easily do that now?
I can modify /config/app.nginx either in another app (intellij) or even in the simple Pharo text editor, and the I can add my new route in my DemoApp>>createRoutes method but how do I check them in together so my pipeline will build atomically?
Iceberg hasnât written out the changes yet, so IntelliJ canât see them to do a commit, and iceberg ignores the parallel /config directory (that it checked out). So itâs a catch 22.
This is why I suggested maybe we could specify safer (textual) directories that iceberg might also checkin? OR we have a Stage command in iceberg that does everything that commit does up to the point of actually writing to the repo - then I could jump to IntelliJ and do the final commit there and use its tools to manage non Pharo stuff (until we can build more)?
Does this make sense?
I donât understand why you are so eager to have everything into one commit. Usually the tension is rather have small commits. What is the problem of having two commits for this?
A single commit allow one to make sure that both the smalltalk code and the external resource are well in sync, and that you may not ending up in a situation were at commit j has data format v1 and smalltalk code for format v2, because it is in commit j+1 that you rewrote the data in format v2.
Yes, sure but that is only a problem if you lost control over which commit goes into action. I think the problem is in the deployment then. Putting integrity constraints on commits is IMHO the wrong level deployment granularity
Norbert
I had that issue recently with a Pharo / FPGA project with a Pharo package generating state machines to be integrated in a Verilog FPGA design with C code for the drivers, the softcore in the FPGA, and test programs, and both Pharo and the C/FPGA working out of the same test data... And that whole thing getting regularly out of sync.
IMHO: I'd model a concept of "multi-lingual projet" for that sort of things, where one can list, in Pharo, Monticello packages and external files or directories. Whatever the technology you use (OSProcess, libgit, whatever...) it is easy then to commit all this in a single pass. On a per-package basis, I'd see the package manifest as a suitable place for listing external resources. On a per-project basis, a possible place could be the manifest for the BaselineOf the project.
Thierry
Norbert
As an aside - Iâd really like to checkin in the play-xxx directories (the .ph files) as there is often useful playground stuff Iâd like to access on my home computer. We canât do that easily at the moment either.
Tim
Sent from my iPhone
On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:
Just to complement Esteban's answer:
- Iceberg checks out in disk more than the src directory because you **may** want to edit files from the command line, and after long discussions we did not want to forbid that. Actually, just to put everybody in perspective, at first the idea was to not have a working copy in disk at all, but just hit to the blob. Imagine is nowadays we are a bit alien, that would have been worst :)
- About checking in files. I'd like to understand what you mean exactly. - Do you want to load them into memory? This would be the "more consistent" way to do it, following the "the image it its own working copy" metaphore. This would allow us to, for example, share an image and transparently share resources with it (without requiring to clone). But this would have some impact in memory consumption and add stress to the GC, right?
- Or do you mean to ask like any other Git client and show you the file differences between the working copy and the git index? The problem with this approach is that we will have some treatment for pharo code and some different treatment for non-code... If I do a change to a class, the change is kept in the image. But if I do a change to a file, that change is not kept in the image!
Also, as Esteban says, having an IDE with support for files would mean that we would need good tools to edit in-memory files (not only text files, right? but also any kind of binary file...)
So far we cover the bare minimum that allows us to *not lose* changes :)
On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Fair enough - its pretty cool weâve got this far, and I guess the onus is on the rest of us to learn more about how its done and see if we can contribute more somehow. I really appreciate the love youâve already put into this - it works far better than I think we even realised it could.
Tim
On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Esteban - so I don't then understand why iceberg (usefully in my view) checks out more than the src directory, if itâs only focusing on the Pharo blob?
Iâm guessing that by knowing where the src is, you are just committing that part of the tree with libgit?
Perhaps from a pragmatic first step you might consider letting us add a second safe resources directory that you could check in atomically as well (on the understanding all bets are off if it goes wrong?)
OR could we have a check in mode does all the add/remove operations and writes to disk but then letâs you drop to the command line/other tool to add any other files and do the final commit?
I just feel like you/we are so close to something that works a bit more broadly and embrace the wider world.?
yeah⦠but is a lot of work and no time just right now. long term, it would be cool to manage everything from iceberg. but reality check, is a huge amount of work so it has to come step by step.
Tim
Sent from my iPhone
On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
hi,
On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:
Hi - my second attempt at using Pharo with Git has proven very satisfying (I saw the potential in phase 1, but it was often difficult to understand what was happening and the workflow to use).
One thing that has come up a few times for me however - and its something that using git nicely highlights, there are many non-smalltalk assets in my project that donât need to live in the image (like Seaside FileLibraries were trying to do) but do need to be versioned and be part of my project. Common examples are server config files, images and even the playground history files that are useful to pull up when on another computer.
It seems that while Iceberg does check out a full project, if I change any of the files outside of the src directory (like edit a .txt file using the crude Pharo file editor), those changes donât get committed when I do a checkin? Is this on purpose? It makes the workflow a bit trickier to do an atomic commit of a piece of work - and Iâm not clear whether this is a conscious thing, or an MVP thing (and it will come later).
workflow is tricker because you are expecting iceberg to talk with the local working copy and to handle that WC. what happens in fact is different: iceberg treats the image as a working copy itself (it has its own âstageâ area) and what you have in disk is like a separated WC.
at least, this is the metaphor we are using now, because we cannot realistically handle/control what is in disk since it can be anything. So, instead having this picture in mind:
Image -> Disk -> Git blob (database)
you need to have this other:
Image \ Git blob (database) Disk /
you will see as soon as you change the mental image, your problems are gone ;)
cheers! Esteban
ps: diagram before is not exactly as it is since the image actually writes into disk first, but this is an implementation detail we would like to remove in the future, even.
As mentioned above, I was also thinking it would be nice if I could checkin some of the play-xxxx/*.sh files to essentially keep some of that history synced between environments (or team members?).
It strikes me that this is the kind of thing that git integration should bring to us?
I can overlay my copy of IntelliJ on top of my local iceberg directory and then use it for checkins - but then I still have the atomic problem, as its only when I commit that tonel files are written out onto the file system for me to checkin along with any other assets Iâve changed. Does anyone else have a good workflow for this? What do you guys do?
Tim
--
Guille Polito Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>
Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13
participants (8)
-
Esteban A. Maringolo -
Esteban Lorenzano -
Guillermo Polito -
Norbert Hartl -
Peter Uhnák -
Thierry Goubier -
Tim Mackinnon -
Todd Blanchard