- Pharo-dev - lists.pharo.org

Re: [Ann] Soil v2

by Norbert Hartl

> Am 03.09.2024 um 03:04 schrieb sean(a)clipperadams.com: > > Exciting news! > > Two questions: > > How is the stability? I am finally outgrowing SimplePersistence, but some of my systems are too critical to risk beta testingâ¦ > I think the only way finding out is to try. In ApptiveGrid we have 5000 instances and it is in production over one year. I wouldn't say there weren't problems in that time. But for me it is quite some time since I've found a conceptual problem that caused errors. This doesn't mean that it works for you and you won't bump into something the first time you use it. Stability is use case dependent. > I remember hearing (or reading?) that Fuel wasnât used underneath because it wasnât flexible or fast enough. Is that correct? If so, is there anywhere I can read about this analysis? Or would you touch on it here? It seemed like a lot of community resources were invested in Fuel so itâs disappointing that a rewrite felt necessary and Iâm wondering if any of the lessons learned in Soil could help Fuel. > The new implementation is only questionable from a bird's eye view. Meaning when you put them under the coarse grained category serializers. That is not appropriate for many things because the way you use it is important. There is no written detailled analysis but let me give you a few points: Fuel is a whole graph serializer. Meaning you give it one object and then it produces an efficient serialized blob out of it. There are not many interception points. Soil supports graph partitioning, meaning that your graph will be sliced into smaller parts in order to have efficient control over the amount of data needs to be read/written. Adding that to fuel was hard and in the end did not give enough control to tweak Fuel has its own way of managing class names which needs pre-processing which is not suited if you want to use streaming objects to disk Fuel only supports management of class names because it assumes them to be there in the image. Soil stores class descriptions in the database. Adding that to Fuel wouldn't be rather difficult. Soil has class versions because as it is a database you can store data in one shape and then change the code. So we need proper migration support which in Fuel is only doable with a central registry and an external version managment Soil separates content and structure completely, meaning sub-graphs are stored together but everything referencing and addressing is done on the database level, meaning you can deal with the structure without having the code at hand etc. Soil uses byte type codes for special types to map it efficiently as type-length-value format onto disk. Fuel has no support for how you store objects on disk. ... Hope this gives a bit more insight than last time. I did not "just wanted to do my own", I considered using Fuel but it didn't work. So as usual there is no one-fits-all solution. And this is a good example. Norbert > Anyway thanks for creating and sharing this :) > > Sean >

Sept. 3, 2024

Re: Metacello Dependency Version Best Practice

by Guillermo Polito

Hi Sean, This is my take on this issue, do not take it as ground truth is far from that :) TL;DR; forcing everybody to use major versions is not a solution: why do we have minor versions if we can only use major versions? > El 4 ago 2024, a las 5:22â¯a. m., sean(a)clipperadams.com escribiÃ³: > > I recently experienced some CI problems for several of my projects because of an (IMO unnecessarily tight) version specification of XML-XPATHâs dependency on XMLParser: `github://pharo-contributions/XML-XMLParser:v3.6.x/src`. At minimum, it seems the dependency should be on the major version (e.g. v3), not the minor (unless Iâm missing a needed feature that was added in 3.6). > Iâll start with a fact here. The developer chose a dependency 3.6. Two options here: it could have been a good justified choice or not. 1) if justified, then there is no way to use the major version. The thing is we donât have proper support to manage version conflicts Letâs say we have projects A, B, C, and D forming a diamond. .----depends on v1 of ---> B ----depends on v1.x of â. / \ A -> D \ / .----depends on v1 of ---> C ----depends on v1.1 of â' >From a semantic versioning point of view, 1.1 is compatible with 1.x So a good version resolution should load D v1.1 when loading A, right? However, if you do this right now with the current state of Metacello, dependency management is delegated to git or the user. Git will know nothing about semantic versioning and tell that 1.1 and 1.x are different refs, and there is a conflict. => My conclusion here is that semantic versioning is a tool - we need good semantic version resolution if we embrace it - if we forbid people to use semantic versioning, letâs stop using semantic versioning at all An even worst example. What if B and C depend on incompatible versions of D? How would you resolve that issue? .----depends on v1 of ---> B ----depends on v2.x of â. / \ A -> D \ / .----depends on v1 of ---> C ----depends on v1.1 of ââ Here itâs easy to blame either the developer of C (for not upgrading) or A (for getting a wrong configuration, but maybe it was his only choice!). Maybe why not Bâs developer, because of âhasty upgradingâ? Thing is, - maybe Câs maintainer is not there anymore, - B was developed when D v2.x was available so people did not even think of developing it against older versions - A did what he could with what was availableâ¦ => My conclusion here is that life is complicated :) - besides good tooling and good criteria, complex cases will arise, what do we propose as a resolution? 2) Letâs say that the developerâs choice was not justified. Then why not engage with the developer to understand the reason for the decision? Can the dependency be fixed without breaking anything? There is no magic :) > More importantly though, it raises questions about what the best practice is here. In summary, it seems to me that in general itâs better to only pin versions in reaction to CI failures because pinning has significant downsides and uncertainties. Defensive pinning smells like premature optimization IMO. > Why are you assuming somebody did a âdefensive pinningâ? I mean, maybe itâs the case, but why are you assuming this was not justified? > In more detail, partially pinning (to major or minor version) by default doesn't seem right to me for several reasons unless we're talking about a tagged release that should be reproducible because: > In general, I agree with the rule :) But what if this was a conscious decision and not just a default thing? I understand this does not suit your current needs, but I see a lot of implicit assumptions in my interpretation of your view. > We have limited community resources to manage these pinnings, which create cascading conflict problems with all other dependent projects. > True > Pinnings seem somewhat arbitrary because often community maintainers are not familiar enough with both projects or have enough time to do a thorough investigation. How does the maintainer know whether a particular dependency version really âworksâ with the project? > You test? > Unless they are intimately familiar with both projects (and even then I wouldnât have confidence in a manual review), the best way I can think of is to rely on passing CI, and in that caseâ¦ > I donât see what is the solution you propose here... > It seems easier to just react to CI failures; defensive pinning smells like premature optimization IMO > What do you mean to react to CI failures? > Partial pinning (minor or major version) will not lead to reproducible builds because the patch is floating > True > What does reproducibility even mean from an untagged baseline? > True > Looking through a bunch of repos, it doesnât seem that there is consensus either way. Some specify baselines and some specific versions. > Yes. The thing is also that Metacello is both at the same time used to - load development versions of projects with a more floating schema... - try to load a working âfixedâ version > p.s. for tracking the major, I like ba-stâs naming convention of v{integer} instead of adding â.xââs of unclear value > This is aesthetics only, right? Anyways, Iâd like to have a more âstandardisedâ way :)

Sept. 3, 2024

Re: [Ann] Soil v2

by sean＠clipperadams.com

Exciting news! Two questions: * How is the stability? I am finally outgrowing SimplePersistence, but some of my systems are too critical to risk beta testingâ¦ * I remember hearing (or reading?) that Fuel wasnât used underneath because it wasnât flexible or fast enough. Is that correct? If so, is there anywhere I can read about this analysis? Or would you touch on it here? It seemed like a lot of community resources were invested in Fuel so itâs disappointing that a rewrite felt necessary and Iâm wondering if any of the lessons learned in Soil could help Fuel. Anyway thanks for creating and sharing this :) Sean

Sept. 3, 2024

Re: Metacello Dependency Version Best Practice

by sean＠clipperadams.com

Anyone? It would be good to have some consensus, at least for community-contribution projectsâ¦

Sept. 3, 2024

This week (35/2024) on the Pharo Issue Tracker

by Marcus Denker

# Pharo 12 - Avoid rewriting methods that are not installed #17005 https://github.com/pharo-project/pharo/pull/17005 # Virtual Machine - Remove hostname lookup on network initialization #845 https://github.com/pharo-project/pharo-vm/pull/845 - small change in copyWithoutReturn to handle CCoerce #836 https://github.com/pharo-project/pharo-vm/pull/836 - Improvements in stack management #710 https://github.com/pharo-project/pharo-vm/pull/710 - Frame unification #602 https://github.com/pharo-project/pharo-vm/pull/602 # Features - 1583 Allow to browse a method when selection is selector #1584 https://github.com/pharo-spec/Spec/pull/1584/files - Add an Apply button for setting options #824 https://github.com/pharo-spec/NewTools/pull/824 - Add header title to filtering list #1591 https://github.com/pharo-spec/Spec/pull/1591 # Launcher - Add âfileServiceâ for PharoLauncherApplication allowing â.imageâ files to be opened through PhLLaunchImageFromDiskCommand #696 https://github.com/pharo-project/pharo-launcher/pull/696 # Cleanup Morphic - Unhook Settings Browser dialog: method from Morphic #17026 https://github.com/pharo-project/pharo/pull/17026 - Remove usage of Calypso browser search dialog made in Morphic #16994 https://github.com/pharo-project/pharo/pull/16994 # Cleanup Misc - Rename some common AST nodes to remove RB prefix #17008 https://github.com/pharo-project/pharo/pull/17008

Aug. 30, 2024

Re: [Ann] Soil v2

by Norbert Hartl

> Am 29.08.2024 um 14:46 schrieb Esteban Maringolo <emaringolo(a)gmail.com>: > > Hi Norbert, > > Excellent news. > > What's the problem that's stopping it from working in Windows? I couldn't find a Github issue for that. Because you assume windows support is something we want. I personally do not care about windows because I never used it and I think it is a really bad operating system that kills lots of things. The technical view is: - soil uses file range locks to lock portions of a file (think of row-level locking in relational databases) - soil uses a system call fsync() on POSIX systems that flushes bytes from OS to disk. This is needed to meet transaction requirements The architecture view is: - I've seen the implementation of file opening with additional flags in OmniBase and how the design got really bad in order to support all of this. This might be non-issue today but needs to prove me wrong first - windows might have comparable support for the requirements above but I'm not easily convinced to ruin the design in order to support it. Might also not be true... The maintainer view is: - I won't add windows support ever and Pablo was the only one offering his help so far (why should we implement something nobody needs?) - even if there would be someone doing it initially I still need to be convinced that there is always someone maintaining it because I won't. And I surely do not want to block progress of soil just because things cannot move. They will be removed then. So there are basically two options: - windows supports the requirements (current and future ones) comparable to unix systems so that the design does not need to be changed a lot in order to support windows. Then adding support is feasible - in the meantime I made the file locking and fsync configurable. So by switching them off it might just work out of the box without any change. On a desktop machine it should(!) work reliably. Locking and range locks are mostly needed for concurrency. But noone would be insane enough to use windows as a server or in a distributed environment, haha, no that would be really foolish...hahaha But we also try to make soil the de-facto database for pharo so in the end we have no choice then to add windows support. Hope this helps, Norbert > > Regards! > > > Esteban A. Maringolo > > > On Wed, Aug 28, 2024 at 9:00â¯AM Norbert Hartl <norbert(a)hartl.name <mailto:norbert@hartl.name>> wrote: >> We are proud to announce the availability of Soil version 2. >> >> This release is dated back to July because we made some bigger changes after that and all the bug fixes before that deserve a release. You can access the new release via >> >> <Soil.png> >> ApptiveGrid/Soil at v2 >> github.com >> <https://github.com/ApptiveGrid/Soil/tree/v2>ApptiveGrid/Soil at v2 <https://github.com/ApptiveGrid/Soil/tree/v2> >> github.com <https://github.com/ApptiveGrid/Soil/tree/v2> >> >> The changelog is available at >> >> >> Soil/docs/changelog.md at v2 Â· ApptiveGrid/Soil >> github.com >> <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md>Soil/docs/changelog.md at v2 Â· ApptiveGrid/Soil <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> >> github.com <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> >> The new features need to mature a bit but we expect to have another release in a month or two. >> >> Enjoy, >> >> Norbert + Marcus > <Soil.png>

Aug. 29, 2024

Re: [Ann] Soil v2

by Esteban Maringolo

Hi Norbert, Excellent news. What's the problem that's stopping it from working in Windows? I couldn't find a Github issue for that. Regards! Esteban A. Maringolo On Wed, Aug 28, 2024 at 9:00â¯AM Norbert Hartl <norbert(a)hartl.name> wrote: > We are proud to announce the availability of Soil version 2. > > This release is dated back to July because we made some bigger changes > after that and all the bug fixes before that deserve a release. You can > access the new release via > > [image: Soil.png] > > ApptiveGrid/Soil at v2 <https://github.com/ApptiveGrid/Soil/tree/v2> > github.com <https://github.com/ApptiveGrid/Soil/tree/v2> > <https://github.com/ApptiveGrid/Soil/tree/v2> > > The changelog is available at > > [image: Soil.png] > > Soil/docs/changelog.md at v2 Â· ApptiveGrid/Soil > <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > github.com <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > > The new features need to mature a bit but we expect to have another > release in a month or two. > > Enjoy, > > Norbert + Marcus >

Aug. 29, 2024

Re: [Pharo-users] [Ann] Soil v2

by stephane ducasse

Excellent news! S > On 28 Aug 2024, at 14:00, Norbert Hartl <norbert(a)hartl.name> wrote: > > We are proud to announce the availability of Soil version 2. > > This release is dated back to July because we made some bigger changes after that and all the bug fixes before that deserve a release. You can access the new release via > > https://github.com/ApptiveGrid/Soil/tree/v2 > > The changelog is available at > > https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md > The new features need to mature a bit but we expect to have another release in a month or two. > > Enjoy, > > Norbert + Marcus StÃ©phane Ducasse http://stephane.ducasse.free.fr 06 30 93 66 73 "If you knew today was your last day on earth, what would you do differently? ....ESPECIALLY if, by doing something different, today might not be your last day on earth.â Calvin & Hobbes

Aug. 29, 2024

This week (34/2024) on the Pharo Issue Tracker

by Marcus Denker

This is Aug 16 to Aug 22 (last week) # Features - Enable Calypso to use a custom navigation history #17014 https://github.com/pharo-project/pharo/pull/17014 - Add scroll to index to Lists #1586 https://github.com/pharo-spec/Spec/pull/1586 # Virtual Machine - small change in dead code elimination to considers a method with only comments empty #843 https://github.com/pharo-project/pharo-vm/pull/843 - add a comment explaining why declareCVarsIn: is empty in some subclasses #842 https://github.com/pharo-project/pharo-vm/pull/842 - DruidJIT: Adding super-instructions for jump #831 https://github.com/pharo-project/pharo-vm/pull/831 # Fixes - Fix setting of the position of the TransferMorph in #startDrag: on FTTableMorph to take into account that the eventâs position is in local coordinates #17015 https://github.com/pharo-project/pharo/pull/17015 - Fix senders broken in Calypso editors #17011 https://github.com/pharo-project/pharo/pull/17011 - Adapt test because CI uses another display scale configuration #1587 https://github.com/pharo-spec/Spec/pull/1587 # Refactoring Engine - Speed up more refactoring tests by constricting the refactoring model #17010 https://github.com/pharo-project/pharo/pull/17010

Aug. 29, 2024

Re: [Ann] Soil v2

by Martin Dias

Awesome, thanks for sharing! MartÃn El miÃ©, 28 ago 2024 a la(s) 8:00â¯a.m., Norbert Hartl (norbert(a)hartl.name) escribiÃ³: > We are proud to announce the availability of Soil version 2. > > This release is dated back to July because we made some bigger changes > after that and all the bug fixes before that deserve a release. You can > access the new release via > > [image: Soil.png] > > ApptiveGrid/Soil at v2 <https://github.com/ApptiveGrid/Soil/tree/v2> > github.com <https://github.com/ApptiveGrid/Soil/tree/v2> > <https://github.com/ApptiveGrid/Soil/tree/v2> > > The changelog is available at > > [image: Soil.png] > > Soil/docs/changelog.md at v2 Â· ApptiveGrid/Soil > <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > github.com <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > <https://github.com/ApptiveGrid/Soil/blob/v2/docs/changelog.md> > > The new features need to mature a bit but we expect to have another > release in a month or two. > > Enjoy, > > Norbert + Marcus >

Aug. 28, 2024