[Pharo-users] Why doesn't Iceberg checkin other assets (scripts) but does check them out?

Thierry Goubier thierry.goubier at gmail.com
Thu Jun 14 09:54:36 EDT 2018


2018-06-14 15:39 GMT+02:00 Tim Mackinnon <tim at testit.works>:

> Yes - I’m a fan of small checkins (not always great at it, but I strive
> for it) - and yes, the issue I’ve had is the same as what Thierry mentions
> - my Smalltalk code is tied to an external file which has some rewrite
> rules that depend on the classes I have in my image. I could possibly
> generate them from the Smalltalk code (this is something Sean does) - but I
> can see all kinds of reasons to atomically checkin code together.
>
>
And generating is not allways possible.


> It haven’t tried it - but I’m guessing if I stick non smalltalk things in
> my src directory, that might work (ugly and potentially confusing, but it
> might do the trick). OR I’ve just learned about #addFilesToIndex: which
> might help me in this case.
>

I wouldn't put things in the src directory. FileTree, for one, is
implemented by "I erase everything on-disk, then I write back my package",
which has a non-negligeable chance of erasing non-Smalltalk data in there.

The addFilesToIndex: should work, but the question becomes what is the hook
to automatically call it when saving your project with Iceberg.


>
> But as a more general (and easier to understand) solution - working well
> in a polyglot arena and checking other stuff in would be very handy. Hence
> my “staging” proposal (if it would work?).
>

Probably, yes.


>
> Alternatively, I will try and see what happens if I commit locally
> (without push) and then use my external tool to commit other files and then
> do the final push t(talking about it here helped me spot what could be an
> easy workaround). Still, I do hope we can keep working on the tools to get
> to something a bit more sophisticated - as the full atomic commit is the
> obvious end goal.
>

Consider using git squash, then. And calling git squash could be automated.

One run the risk of turning Iceberg in a complete language-agnostic git
client, which could make it too big to be sustainable in the long term.

Thierry


>
> Tim
>
>
> On 14 Jun 2018, at 12:12, Thierry Goubier <thierry.goubier at gmail.com>
> wrote:
>
> Hi Norbert, Tim,
>
> 2018-06-14 11:33 GMT+02:00 Norbert Hartl <norbert at hartl.name>:
>
>>
>>
>> Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim at testit.works>:
>>
>> Hi - yes I’m pleased you check out the entire tree, although currently
>> it’s a bit confusing that you do (fortunately this does give the
>> possibility that we can checkout images and other resources that an Pharo
>> application might rely on - without having to resort to the Seaside
>> FileLibrary trick).
>>
>> However my concrete case was that I have a gitlab ci pipeline and next to
>> my src directory in my project I have a config directory that has some
>> Nginx config for my teapot app. If I add a teapot route, I also need to
>> adjust that config and check both changes in together. I can’t easily do
>> that now?
>>
>> I can modify /config/app.nginx either in another app (intellij) or even
>> in the simple Pharo text editor, and the I can add my new route in my
>> DemoApp>>createRoutes method but how do I check them in together so my
>> pipeline will build atomically?
>>
>> Iceberg hasn’t written out the changes yet, so IntelliJ can’t see them to
>> do a commit, and iceberg ignores the parallel /config directory (that it
>> checked out). So it’s a catch 22.
>>
>> This is why I suggested maybe we could specify safer (textual)
>> directories that iceberg might also checkin? OR we have a Stage command in
>> iceberg that does everything that commit does up to the point of actually
>> writing to the repo - then I could jump to IntelliJ and do the final commit
>> there and use its tools to manage non Pharo stuff (until we can build more)?
>>
>> Does this make sense?
>>
>> I don’t understand why you are so eager to have everything into one
>> commit. Usually the tension is rather have small commits. What is the
>> problem of having two commits for this?
>>
>
> A single commit allow one to make sure that both the smalltalk code and
> the external resource are well in sync, and that you may not ending up in a
> situation were at commit j has data format v1 and smalltalk code for format
> v2, because it is in commit j+1 that you rewrote the data in format v2.
>
> I had that issue recently with a Pharo / FPGA project with a Pharo package
> generating state machines to be integrated in a Verilog FPGA design with C
> code for the drivers, the softcore in the FPGA, and test programs, and both
> Pharo and the C/FPGA working out of the same test data... And that whole
> thing getting regularly out of sync.
>
> IMHO: I'd model a concept of "multi-lingual projet" for that sort of
> things, where one can list, in Pharo, Monticello packages and external
> files or directories. Whatever the technology you use (OSProcess, libgit,
> whatever...) it is easy then to commit all this in a single pass. On a
> per-package basis, I'd see the package manifest as a suitable place for
> listing external resources. On a per-project basis, a possible place could
> be the manifest for the BaselineOf the project.
>
> Thierry
>
>
>>
>> Norbert
>>
>> As an aside - I’d really like to checkin in the play-xxx directories (the
>> .ph files) as there is often useful playground stuff I’d like to access on
>> my home computer. We can’t do that easily at the moment either.
>>
>> Tim
>>
>> Sent from my iPhone
>>
>> On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopolito at gmail.com>
>> wrote:
>>
>> Just to complement Esteban's answer:
>>
>> - Iceberg checks out in disk more than the src directory because you
>> **may** want to edit files from the command line, and after long
>> discussions we did not want to forbid that.
>> Actually, just to put everybody in perspective, at first the idea was to
>> not have a working copy in disk at all, but just hit to the blob.
>> Imagine is nowadays we are a bit alien, that would have been worst :)
>>
>> - About checking in files. I'd like to understand what you mean exactly.
>>   - Do you want to load them into memory?
>>     This would be the "more consistent" way to do it, following the "the
>> image it its own working copy" metaphore.
>>     This would allow us to, for example, share an image and
>> transparently share resources with it (without requiring to clone).
>>     But this would have some impact in memory consumption and add stress
>> to the GC, right?
>>
>>   - Or do you mean to ask like any other Git client and show you the
>> file differences between the working copy and the git index?
>>     The problem with this approach is that we will have some treatment
>> for pharo code and some different treatment for non-code...
>>     If I do a change to a class, the change is kept in the image. But if
>> I do a change to a file, that change is not kept in the image!
>>
>> Also, as Esteban says, having an IDE with support for files would mean
>> that we would need good tools to edit in-memory files (not only text files,
>> right? but also any kind of binary file...)
>>
>> So far we cover the bare minimum that allows us to *not lose* changes :)
>>
>> On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim at testit.works> wrote:
>>
>>>
>>> > yeah… but is a lot of work and no time just right now.
>>> > long term, it would be cool to manage everything from iceberg.
>>> > but reality check, is a huge amount of work so it has to come step by
>>> step.
>>>
>>> Fair enough - its pretty cool we’ve got this far, and I guess the onus
>>> is on the rest of us to learn more about how its done and see if we can
>>> contribute more somehow. I really appreciate the love you’ve already put
>>> into this - it works far better than I think we even realised it could.
>>>
>>> Tim
>>>
>>>
>>>
>>> > On 13 Jun 2018, at 21:55, Esteban Lorenzano <estebanlm at gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> >> On 13 Jun 2018, at 22:44, Tim Mackinnon <tim at testit.works> wrote:
>>> >>
>>> >> Esteban - so I don't then understand why iceberg (usefully in my
>>> view) checks out more than the src directory, if it’s only focusing on the
>>> Pharo blob?
>>> >>
>>> >> I’m guessing that by knowing where the src is, you are just
>>> committing that part of the tree with libgit?
>>> >>
>>> >> Perhaps from a pragmatic first step you might consider letting us add
>>> a second safe resources directory that you could check in atomically as
>>> well (on the understanding all bets are off if it goes wrong?)
>>> >>
>>> >> OR could we have a check in mode does all the add/remove operations
>>> and writes to disk but then let’s you drop to the command line/other tool
>>> to add any other files and do the final commit?
>>> >>
>>> >> I just feel like you/we are so close to something that works a bit
>>> more broadly and embrace the wider world.?
>>> >
>>> > yeah… but is a lot of work and no time just right now.
>>> > long term, it would be cool to manage everything from iceberg.
>>> > but reality check, is a huge amount of work so it has to come step by
>>> step.
>>> >
>>> >>
>>> >> Tim
>>> >>
>>> >> Sent from my iPhone
>>> >>
>>> >>> On 13 Jun 2018, at 21:28, Esteban Lorenzano <estebanlm at gmail.com>
>>> wrote:
>>> >>>
>>> >>> hi,
>>> >>>
>>> >>>
>>> >>>> On 13 Jun 2018, at 16:50, Tim Mackinnon <tim at testit.works> wrote:
>>> >>>>
>>> >>>> Hi - my second attempt at using Pharo with Git has proven very
>>> satisfying (I saw the potential in phase 1, but it was often difficult to
>>> understand what was happening and the workflow to use).
>>> >>>>
>>> >>>> One thing that has come up a few times for me however - and its
>>> something that using git nicely highlights, there are many non-smalltalk
>>> assets in my project that don’t need to live in the image (like Seaside
>>> FileLibraries were trying to do) but do need to be versioned and be part of
>>> my project. Common examples are server config files, images and even the
>>> playground history files that are useful to pull up when on another
>>> computer.
>>> >>>>
>>> >>>> It seems that while Iceberg does check out a full project, if I
>>> change any of the files outside of the src directory (like edit a .txt file
>>> using the crude Pharo file editor), those changes don’t get committed when
>>> I do a checkin? Is this on purpose? It makes the workflow a bit trickier to
>>> do an atomic commit of a piece of work - and I’m not clear whether this is
>>> a conscious thing, or an MVP thing (and it will come later).
>>> >>>
>>> >>> workflow is tricker because you are expecting iceberg to talk with
>>> the local working copy and to handle that WC.
>>> >>> what happens in fact is different: iceberg treats the image as a
>>> working copy itself (it has its own “stage” area) and what you have in disk
>>> is like a separated WC.
>>> >>>
>>> >>> at least, this is the metaphor we are using now, because we cannot
>>> realistically handle/control what is in disk since it can be anything.
>>> >>> So, instead having this picture in mind:
>>> >>>
>>> >>> Image -> Disk -> Git blob (database)
>>> >>>
>>> >>> you need to have this other:
>>> >>>
>>> >>> Image \
>>> >>>      Git blob (database)
>>> >>> Disk    /
>>> >>>
>>> >>>
>>> >>> you will see as soon as you change the mental image, your problems
>>> are gone ;)
>>> >>>
>>> >>> cheers!
>>> >>> Esteban
>>> >>>
>>> >>> ps: diagram before is not exactly as it is since the image actually
>>> writes into disk first, but this is an implementation detail we would like
>>> to remove in the future, even.
>>> >>>
>>> >>>>
>>> >>>> As mentioned above, I was also thinking it would be nice if I could
>>> checkin some of the play-xxxx/*.sh files to essentially keep some of that
>>> history synced between environments (or team members?).
>>> >>>>
>>> >>>> It strikes me that this is the kind of thing that git integration
>>> should bring to us?
>>> >>>>
>>> >>>> I can overlay my copy of IntelliJ on top of my local iceberg
>>> directory and then use it for checkins - but then I still have the atomic
>>> problem, as its only when I commit that tonel files are written out onto
>>> the file system for me to checkin along with any other assets I’ve changed.
>>> Does anyone else have a good workflow for this? What do you guys do?
>>> >>>>
>>> >>>> Tim
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >
>>> >
>>>
>>>
>>>
>>
>> --
>>
>> Guille Polito
>> Research Engineer
>>
>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>> CRIStAL - UMR 9189
>> French National Center for Scientific Research - *http://www.cnrs.fr
>> <http://www.cnrs.fr/>*
>>
>> *Web:* *http://guillep.github.io* <http://guillep.github.io/>
>> *Phone: *+33 06 52 70 66 13
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-users_lists.pharo.org/attachments/20180614/177105e2/attachment-0001.html>


More information about the Pharo-users mailing list