September 2024 - Pharo-dev - lists.pharo.org

Re: [Ann] Soil v2

by Norbert Hartl

> Am 03.09.2024 um 03:04 schrieb sean(a)clipperadams.com: > > Exciting news! > > Two questions: > > How is the stability? I am finally outgrowing SimplePersistence, but some of my systems are too critical to risk beta testingâ¦ > I think the only way finding out is to try. In ApptiveGrid we have 5000 instances and it is in production over one year. I wouldn't say there weren't problems in that time. But for me it is quite some time since I've found a conceptual problem that caused errors. This doesn't mean that it works for you and you won't bump into something the first time you use it. Stability is use case dependent. > I remember hearing (or reading?) that Fuel wasnât used underneath because it wasnât flexible or fast enough. Is that correct? If so, is there anywhere I can read about this analysis? Or would you touch on it here? It seemed like a lot of community resources were invested in Fuel so itâs disappointing that a rewrite felt necessary and Iâm wondering if any of the lessons learned in Soil could help Fuel. > The new implementation is only questionable from a bird's eye view. Meaning when you put them under the coarse grained category serializers. That is not appropriate for many things because the way you use it is important. There is no written detailled analysis but let me give you a few points: Fuel is a whole graph serializer. Meaning you give it one object and then it produces an efficient serialized blob out of it. There are not many interception points. Soil supports graph partitioning, meaning that your graph will be sliced into smaller parts in order to have efficient control over the amount of data needs to be read/written. Adding that to fuel was hard and in the end did not give enough control to tweak Fuel has its own way of managing class names which needs pre-processing which is not suited if you want to use streaming objects to disk Fuel only supports management of class names because it assumes them to be there in the image. Soil stores class descriptions in the database. Adding that to Fuel wouldn't be rather difficult. Soil has class versions because as it is a database you can store data in one shape and then change the code. So we need proper migration support which in Fuel is only doable with a central registry and an external version managment Soil separates content and structure completely, meaning sub-graphs are stored together but everything referencing and addressing is done on the database level, meaning you can deal with the structure without having the code at hand etc. Soil uses byte type codes for special types to map it efficiently as type-length-value format onto disk. Fuel has no support for how you store objects on disk. ... Hope this gives a bit more insight than last time. I did not "just wanted to do my own", I considered using Fuel but it didn't work. So as usual there is no one-fits-all solution. And this is a good example. Norbert > Anyway thanks for creating and sharing this :) > > Sean >

Sept. 3, 2024

Re: Metacello Dependency Version Best Practice

by Guillermo Polito

Hi Sean, This is my take on this issue, do not take it as ground truth is far from that :) TL;DR; forcing everybody to use major versions is not a solution: why do we have minor versions if we can only use major versions? > El 4 ago 2024, a las 5:22â¯a. m., sean(a)clipperadams.com escribiÃ³: > > I recently experienced some CI problems for several of my projects because of an (IMO unnecessarily tight) version specification of XML-XPATHâs dependency on XMLParser: `github://pharo-contributions/XML-XMLParser:v3.6.x/src`. At minimum, it seems the dependency should be on the major version (e.g. v3), not the minor (unless Iâm missing a needed feature that was added in 3.6). > Iâll start with a fact here. The developer chose a dependency 3.6. Two options here: it could have been a good justified choice or not. 1) if justified, then there is no way to use the major version. The thing is we donât have proper support to manage version conflicts Letâs say we have projects A, B, C, and D forming a diamond. .----depends on v1 of ---> B ----depends on v1.x of â. / \ A -> D \ / .----depends on v1 of ---> C ----depends on v1.1 of â' >From a semantic versioning point of view, 1.1 is compatible with 1.x So a good version resolution should load D v1.1 when loading A, right? However, if you do this right now with the current state of Metacello, dependency management is delegated to git or the user. Git will know nothing about semantic versioning and tell that 1.1 and 1.x are different refs, and there is a conflict. => My conclusion here is that semantic versioning is a tool - we need good semantic version resolution if we embrace it - if we forbid people to use semantic versioning, letâs stop using semantic versioning at all An even worst example. What if B and C depend on incompatible versions of D? How would you resolve that issue? .----depends on v1 of ---> B ----depends on v2.x of â. / \ A -> D \ / .----depends on v1 of ---> C ----depends on v1.1 of ââ Here itâs easy to blame either the developer of C (for not upgrading) or A (for getting a wrong configuration, but maybe it was his only choice!). Maybe why not Bâs developer, because of âhasty upgradingâ? Thing is, - maybe Câs maintainer is not there anymore, - B was developed when D v2.x was available so people did not even think of developing it against older versions - A did what he could with what was availableâ¦ => My conclusion here is that life is complicated :) - besides good tooling and good criteria, complex cases will arise, what do we propose as a resolution? 2) Letâs say that the developerâs choice was not justified. Then why not engage with the developer to understand the reason for the decision? Can the dependency be fixed without breaking anything? There is no magic :) > More importantly though, it raises questions about what the best practice is here. In summary, it seems to me that in general itâs better to only pin versions in reaction to CI failures because pinning has significant downsides and uncertainties. Defensive pinning smells like premature optimization IMO. > Why are you assuming somebody did a âdefensive pinningâ? I mean, maybe itâs the case, but why are you assuming this was not justified? > In more detail, partially pinning (to major or minor version) by default doesn't seem right to me for several reasons unless we're talking about a tagged release that should be reproducible because: > In general, I agree with the rule :) But what if this was a conscious decision and not just a default thing? I understand this does not suit your current needs, but I see a lot of implicit assumptions in my interpretation of your view. > We have limited community resources to manage these pinnings, which create cascading conflict problems with all other dependent projects. > True > Pinnings seem somewhat arbitrary because often community maintainers are not familiar enough with both projects or have enough time to do a thorough investigation. How does the maintainer know whether a particular dependency version really âworksâ with the project? > You test? > Unless they are intimately familiar with both projects (and even then I wouldnât have confidence in a manual review), the best way I can think of is to rely on passing CI, and in that caseâ¦ > I donât see what is the solution you propose here... > It seems easier to just react to CI failures; defensive pinning smells like premature optimization IMO > What do you mean to react to CI failures? > Partial pinning (minor or major version) will not lead to reproducible builds because the patch is floating > True > What does reproducibility even mean from an untagged baseline? > True > Looking through a bunch of repos, it doesnât seem that there is consensus either way. Some specify baselines and some specific versions. > Yes. The thing is also that Metacello is both at the same time used to - load development versions of projects with a more floating schema... - try to load a working âfixedâ version > p.s. for tracking the major, I like ba-stâs naming convention of v{integer} instead of adding â.xââs of unclear value > This is aesthetics only, right? Anyways, Iâd like to have a more âstandardisedâ way :)

Sept. 3, 2024

Re: [Ann] Soil v2

by sean＠clipperadams.com

Exciting news! Two questions: * How is the stability? I am finally outgrowing SimplePersistence, but some of my systems are too critical to risk beta testingâ¦ * I remember hearing (or reading?) that Fuel wasnât used underneath because it wasnât flexible or fast enough. Is that correct? If so, is there anywhere I can read about this analysis? Or would you touch on it here? It seemed like a lot of community resources were invested in Fuel so itâs disappointing that a rewrite felt necessary and Iâm wondering if any of the lessons learned in Soil could help Fuel. Anyway thanks for creating and sharing this :) Sean

Sept. 3, 2024

Re: Metacello Dependency Version Best Practice

by sean＠clipperadams.com

Anyone? It would be good to have some consensus, at least for community-contribution projectsâ¦

Sept. 3, 2024