Intended behaviour of flatten and flatCollect:
Hi everyone, I am getting surprising results trying to flatten nested arrays. So I read the docstrings. And now I understand even less how flatten and flatCollect are supposed to work. flatten says: "Recursively collect each non-collection element of the receiver and its descendant collections. Please note, this implementation assumes that strings are to be treated as objects rather than as collection." That suggests that #(1 2 3) flatten should yield #(1 2 3). But it raises an exception, trying to send do: to an integer. A quick look at the code shows that flatten is just flatCollect: [ :each | each ]. flatCollect: says: "Evaluate aBlock for each of the receiver's elements and answer the list of all resulting values flatten one level. Assumes that aBlock returns some kind of collection for each element. Equivalent to the lisp's mapcan" "Flatten one level" is different from flatten's "recursively". The assumption that aBlock returns a collection is also incompatible with what flatten is supposed to do (and explains my exception). I don't find the reference to a Lisp function very helpful. Assuming "Lisp" means "Common Lisp", this confirms the docstring of flatCollect:, this confirms the assumption of aBlock returning a collection, and that only one level of nesting is flattened. Since flatCollect does what it says, the problem seems to be in flatten. Either its docstring or its implementation are wrong. Or is it me misunderstanding something? Cheers, Konrad
Hi Konrad Sorry, but which version of Pharo are we talking about. I just tried with Pharo 10, which has no flatten method for arrays at all, somewhere along the version I believe the method was renamed to flattened. #(1 #(2 #(a b) 3) 4) flattened >>> #(1 2 #a #b 3 4) as one would expect. flatCollect: does not flatten the collection it is called on, but the result produced by the block #(1 3) flatCollect: [ :e | {e. e * 10} ] >>> #(1 10 3 30), and #(1 #(2 3 #(4 5)) 6) flatCollect: [ :e | {e. e * 10} ] >>> #(1 10 #(2 3 #(4 5)) #(20 30 #(40 50)) 6 60) I am not going to comment on naming of such core api methods - they tend to have a 40 year history of forgotten arguments and random âclean upsâ. Best, Kasper
On 12 Apr 2022, at 11.51, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
Hi everyone,
I am getting surprising results trying to flatten nested arrays. So I read the docstrings. And now I understand even less how flatten and flatCollect are supposed to work.
flatten says:
"Recursively collect each non-collection element of the receiver and its descendant collections. Please note, this implementation assumes that strings are to be treated as objects rather than as collection."
That suggests that
#(1 2 3) flatten
should yield #(1 2 3). But it raises an exception, trying to send do: to an integer.
A quick look at the code shows that flatten is just flatCollect: [ :each | each ]. flatCollect: says:
"Evaluate aBlock for each of the receiver's elements and answer the list of all resulting values flatten one level. Assumes that aBlock returns some kind of collection for each element. Equivalent to the lisp's mapcan"
"Flatten one level" is different from flatten's "recursively". The assumption that aBlock returns a collection is also incompatible with what flatten is supposed to do (and explains my exception).
I don't find the reference to a Lisp function very helpful. Assuming "Lisp" means "Common Lisp", this confirms the docstring of flatCollect:, this confirms the assumption of aBlock returning a collection, and that only one level of nesting is flattened.
Since flatCollect does what it says, the problem seems to be in flatten. Either its docstring or its implementation are wrong.
Or is it me misunderstanding something?
Cheers, Konrad
Hi Kasper, Thanks for your reply!
Sorry, but which version of Pharo are we talking about.
Sorry, I forgot to say that I am still using Pharo 9.
I just tried with Pharo 10, which has no flatten method for arrays at
Interesting. So I double-checked - and found that the "flatten" in my image actually comes from an add-on package called CollectionExtensions (https://github.com/pharo-contributions/CollectionExtensions/blob/master/src/...). So I will complain there :-)
I am not going to comment on naming of such core api methods - they tend to have a 40 year history of forgotten arguments and random âclean upsâ.
I agree, I didn't expect any level of homogeneous naming. Just an agreement between code and documentation. Cheers, Konrad
"Flattening" is problematic in every programming language I know that has something like it in its library. It was born problematic in Lisp and it has continued problematic ever since. (I am aware of several Smalltalk systems offering it. With the exception of Squeak and Pharo, no two of them agree.) The fundamental problem is that which containers (if any) should be treated as leaves and which should not is not something that the language can define. It is problem-specific. Your best bet is to figure out what YOU need for YOUR problem and implement exactly that. I was about to include some code I wrote while trying to make sense of "flattening", but no. There is no substitute for writing something that works for precisely the objects you want it to work and for no others.
It would help to know which version of Pharo you are using. #(1 2 3) flatten raises an exception in Pharo 9 because there is no #flatten in Array's protocol. Collection has #flatCollect: -- #gather: was not a terribly helpful name, -- but #flatCollect: is horribly misleading. #flatCollect:as: -- A misplaced method, should be -- Collection class >> withAll: aCollection gather: aBlock #flatCollectAsSet: -- a trivial special case of #flatCollect:as:, -- not clear why this special case exists. #flattenOn: -- aStream nextPutAll: self flattened -- but without creating the intermediate data structure #flattened -- a curiously twisted "flatten" that does not work -- at all for non-collections. #(1 2 3) flattened => #(1 2 3) 1 flattened => BOOM! By the way, we cannot blame the cruftiness of #flattened on 40 years of history. #flatten/#flattened only entered Squeak about 7 years ago and it was BORN crufty. I mistakenly thought Squeak and Pharo agreed on what it meant. They do not. Squeak: puts it on SequenceableCollection and Stream. Pharo : puts it on Collection (more general) and not Stream (less so). Squeak: treats stream elements like nested collections. Pharo : doesn't. So my earlier claim bears repeating: of the Smalltalk systems that have some kind of flattening, no two agree on what it means. The one use I have ever had for flattening is to build up a large string by accumulating characters, strings, symbols, and sequences of the same in a sequence and then flattening the tree into a single string, which is something that Squeak and Pharo flatly refuse to do. One improvement I would like to see in Pharo 11 is for these methods to be deleted.
PS: *Neither* #flattened nor #flatCollect: can be blamed on 40 years of history. #flatten(ed) arrived in Squeak back in 2015, at which time #flatCollect: did not exist. In fact there is *still* no #flatCollect: in Squeak, although Squeak does have #gather:, since 2002. #flatCollect: was added to Pharo more recently still, as a new name for the tolerably well established #gather:, apparently in the belief that it was a better name. Given #flattened, it is not. #gather: is not all that helpful, but #flatCollect: is just wrong. Even if #flattened did not exist, the result of #flatCollect: is generally not in any sense flat. I repeat: NONE of this can be blamed on Smalltalk history. #flattened was born crufty in 2015 and #flatCollect: is a recent addition to Pharo. On Thu, 14 Apr 2022 at 02:03, Richard O'Keefe <raoknz@gmail.com> wrote:
It would help to know which version of Pharo you are using. #(1 2 3) flatten raises an exception in Pharo 9 because there is no #flatten in Array's protocol. Collection has #flatCollect: -- #gather: was not a terribly helpful name, -- but #flatCollect: is horribly misleading. #flatCollect:as: -- A misplaced method, should be -- Collection class >> withAll: aCollection gather: aBlock #flatCollectAsSet: -- a trivial special case of #flatCollect:as:, -- not clear why this special case exists. #flattenOn: -- aStream nextPutAll: self flattened -- but without creating the intermediate data structure #flattened -- a curiously twisted "flatten" that does not work -- at all for non-collections.
#(1 2 3) flattened => #(1 2 3) 1 flattened => BOOM!
By the way, we cannot blame the cruftiness of #flattened on 40 years of history. #flatten/#flattened only entered Squeak about 7 years ago and it was BORN crufty. I mistakenly thought Squeak and Pharo agreed on what it meant. They do not. Squeak: puts it on SequenceableCollection and Stream. Pharo : puts it on Collection (more general) and not Stream (less so). Squeak: treats stream elements like nested collections. Pharo : doesn't. So my earlier claim bears repeating: of the Smalltalk systems that have some kind of flattening, no two agree on what it means.
The one use I have ever had for flattening is to build up a large string by accumulating characters, strings, symbols, and sequences of the same in a sequence and then flattening the tree into a single string, which is something that Squeak and Pharo flatly refuse to do.
One improvement I would like to see in Pharo 11 is for these methods to be deleted.
participants (3)
-
Kasper Osterbye -
Konrad Hinsen -
Richard O'Keefe