March 2019 - Pharo-users - lists.pharo.org

Parsing text to discover general data of interest (phone, email, address, ...)

by CÃ©drick BÃ©ler

Hi all, Iâve often got the need to analyse some random unstructured text to discover (structured) information (in email for instance), to extract : - emails - telephone numbers - addresses - events - person names (according to a list of known persons), - etcâ¦ Apple do it in email for instance (strangely, this is not generalized). So my questions are : - do we have something equivalent in Smalltalk/Pharo ? (I didnât find) - if not, what strategy would you use ? => I do really stupid text analysis (substrings, finding @, â¦, parsing according to the text structure when there isâ¦ kind of Soup parsingâ¦) => I feel this is a job for PetitParser ? And would be a nice feet to the new GToolkit. All ideas or suggestions are welcome ;-) TIA, CÃ©drick

March 7, 2019

Re: [Pharo-users] IMAP in Pharo

by CÃ©drick BÃ©ler

> > >> On 6 Mar 2019, at 16:08, Sean P. DeNigris <sean(a)clipperadams.com <mailto:sean@clipperadams.com>> wrote: >> >> cedreek wrote >>> Still havent get my head around this FORK/PR cycle but I will for sure one >>> day :) >> >> It took me quite a while to grok. Fork = 'personal remote clone' and: [ PR = >> 'UI support for discussing, testing, and accepting commitsâ ] I got the basic idea but just need to dive in this âtroublingâ water ^^ Quite complex but I know this is powerful. > > It took me even longer to realise that when you load your master branch - you still need to pull changes from the original unforked repository (into your local repository) and then push them back up to your fork, AND then branch to make a change. For ages I was was getting unrelated diff changes even though I thought I had started on âmasterâ. > > Iâm hoping there will be a talk about all of this at Pharo Days. When you get it right its awesome - but when you take a mis-step its quite confusing until you realise what the pieces mean. (And Iâm sure I still have the model wrong, but its fun learning more each day) Yes this kind of small tips are really important to not being confused. And that is whatâs confusing. I think maybe what we miss would be a tutorial and a sandbox project so people can play with (quite intimidating to do his first PR the wrong way). Cheers, CÃ©drick > > Tim >

March 7, 2019

Re: [Pharo-users] Fail on condense when image name is drgeo.image

by Hilaire

Thanks Alistair. Great! I can now build DrGeo with the P7.0.1 image. I will check with the P7.0.2, but should be fine. Now the OS X DrGeo build behave much more gently, but I need to test a bit further. Hilaire Le 06/03/2019 Ã 08:53, Alistair Grant a Ã©critÂ : > For future reference: > > Issue: https://github.com/pharo-project/pharo/issues/2753 > PR: https://github.com/pharo-project/pharo/pull/2754 -- Dr. Geo http://drgeo.eu

March 7, 2019

Re: [Pharo-users] Google Protobuf and usage of Slots

by Holger Freyther

> On 4. Mar 2019, at 16:12, Marcus Denker <marcus.denker(a)inria.fr> wrote: > Thanks for the help! [ToC] I try to keep it brief so please tell me when to elaborate more. (1) Handling of default values. (2) Fast lazy initialization of OrderedCollection for repeated fields (3) Installation of selectors based on the type (4) Caching a descriptor reverse table somewhere (5) Mandatory field handling (6) Verbosity/Tooling (1) In Protobuf a field can have a default value associated. I am torn if to use the option of a Slot to write these fields or have a PBTypeMessage>>#initialize that iterates over the slots and writes the default values. (2) PBRepeatedType>>#decodeFrom:instVarIndex:to: lazily initializes the collection for a repeated field. I am not sure if I can do (much) better and what the trade-offs are (e.g. use the option to write instructions to make loads lazy initialize). (3) I started to use AccessorInstanceVariableSlot as base-class and it is quite neat. I wondered if I want/should/can protect overwriting a user-generated selector. (4) For decoding I am building a field_number -> Slot map. I am currently building this whenever a message needs to be parse. I would like to (lazily) initialize it and store it somewhere. I am considering to use a hidden slot but maybe there is something obvious? (5) After materialization I should check if all mandatory fields were written. The most lazy implementation seems to select all slots that are not repeated/optional and verify that the fields are not nil.. (6) My slot syntax is (too) complex and writing it by hand is difficult. I wondered if Class>>#addSlot: could be exposed nicely in Calypso. And in GTInspector my class is shown without slots. :) The following works now $ protoc -odescriptor.pb src/google/protobuf/descriptor.proto GBPFileDescriptorSet materializeFrom: 'descriptor.pb' asFileReference binaryReadStream. > Hello! > > Nice, I will have a look. > > I am interested to see how class layouts / slots can be used here. > (but it might take some days to find timeâ¦) > > Marcus > >> On 26 Feb 2019, at 07:54, Holger Freyther <holger(a)freyther.de> wrote: >> >> Hi, >> >> from my point of view the pillars of future services are Google protobuf[1] and gRPC. Protobuf can be used for configuration files[2], tracing/logging[3] and storage[4]. gRPC is built on top-of HTTP2 and is using the protobuf IDL and marshaling. Many projects (etcd, envoy, Google Cloud, ...) provide a gRPC based API (and some automatically map this from REST to gRPC) already and I would like to build clients and servers in Pharo. >> >> >> My plan of action is to start with a protobuf compiler/model and then look into binding the gRPC C implementation to Pharo. >> >> >> I have started with gokr's Protobuf code on Smalltalkhub and defined base types and a Slot and could use some help to structure this more nicely. >> >> The code is at: https://github.com/zecke/pharo-protobuf and I could use some comments/review of how to use Slots in a significant way (add "type" validation on write, constraint checks, builder pattern?). >> >> Given a definition like: >> >> syntax = "proto2"; >> package foo; >> enum Color { >> RED = 0; >> GREEN = 1; >> BLUE = 2; >> } >> message MyMessage { >> optional Color color = 1; >> } >> >> And an encoded binary message of #[8 2] the following can: >> >> PBTestMessage materializeFrom: #[8 2] readStream >> >> >> decode and set the color field. Proper handling of mandatory and repeated fields are missing and decoding the other values, nested messages... :) >> >> >> >> Looking forward to have some eyes on the code. >> >> holger >> >> >> >> >> >> [1] A simple IDL to define structs/enums with marshaller/materializer for a binary protocol, JSON (and yaml). >> >> [2] Without having to write a XML/JSON schema and getting validation from the IDL. >> >> [3] Many of us log structured to text, then have something like logstash to re-parse, regexp it and dump it into ElasticSearch. With protobuf we can have plain objects end to end for logging. >> >> [4] We have FUEL so this is less interesting for us than others but quite interesting if you are mixing Pharo with other implementation languages. > >

March 7, 2019

Re: [Pharo-users] Why can't a Bag answer as a dictionary?

by Steffen MÃ¤rcker

This is exactly how I think and feel about bags. Thanks Richard. Am .03.2019, 01:35 Uhr, schrieb Richard O'Keefe <raoknz(a)gmail.com>: > To me, a bag is first and foremost a multiset. > Yes, it's a collection, but it's not *just* a collection. > It's a very specific kind of collection with a very > salient "characteristic function". > > In my own Smalltalk library, #asDictionary isn't even *defined* > on things that are not dictionaries or some kind of mapping, > because let's face it, a dictionary *isn't* a collection of > Associations, and trying to make it look like one made > Smalltalk-80 rather inconsistent. For example, if a dictionary > were a collection of associations, you would expect > (Dictionary with: #a -> 1 with: #b -> 2) includes: #a -> 1 > to be true. No, it's false. You would expect > (Dictionary with: #a -> 1 with: #b -> 2) asArray > to be (a->1 b->2). No, it's (1 2). In order to avoid major > confusion, I had to learn never to think of Associations in > connection with Dictionaries. So I expect #asDictionary to > have something to do with #keysAndValuesDo:. > or something like that. > > For {1->2. 3->4} asDictionary I would have to write > Dictionary withAllAssociations: {1->2. 3->4} > except that I never actually found a use for it. > > #valuesAndCounts is a method that violates good OO practice, > because it exposes (what you'd expect to be) private state. > For example, > b := 'abracadabra' asBag. > b valuesAndCounts at: $a put: 'BOOM!'. > b > select, Print It, BOOM!. It should *copy* the dictionary. > > > > On Thu, 7 Mar 2019 at 04:50, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> I was just explaining how it is now, what I think the rationale is >> behind >> it. >> >> I understand #asDictionary as working on a collection of >> pairs/associations (because it basically goes to #withAll:). >> >> A bag is just a collection that is optimised for many duplicates, the >> fact >> that you have values and counts is more an implementation detail than an >> intrinsic property. >> >> The conversion that you want, and that already exists in >> #valuesAndCounts >> is one interpretation of what a bag is, not the only one. A bag is >> foremost >> a collection of things. >> >> I am not immediately convinced that #valuesAndCounts should be the >> default >> #asDictionary interpretation. >> >> What about the inverse for example ? >> >> { #foo->2. #bar->3 } asDictionary asBag. >> >> But this is certainly an interesting discussion. >> >> > On 6 Mar 2019, at 16:23, Tim Mackinnon <tim(a)testit.works> wrote: >> > >> > As Richard said - as a bag is relationship between keys and >> frequencies, >> I would expect it to be able to convert to a dictionary. >> > >> > It displays in an inspector just like a Dictionary - which is why I >> figured I could convert to pass it back to the exercise that was written >> with Dictionaries in mind. >> > >> > <PastedGraphic-3.png> >> > >> > >> > The code to fix it is quite simple, but on these kinds of things - I >> thought it worth checking before submitting a PR. >> > >> > Tim >> > >> > >> >> On 6 Mar 2019, at 13:53, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> >> >> Why would that work ? What would you expect the output to be ? >> >> >> >> Try: >> >> >> >> #(1 2 3) asDictionary >> >> >> >> it fails in exactly the same way. You need key/value pairs >> (Associations). >> >> >> >> These do work >> >> >> >> Bag new add: #foo->100; asDictionary. >> >> >> >> Bag new addAll: 'ABABABAAAA'; valuesAndCounts. >> >> >> >>> On 6 Mar 2019, at 14:25, Tim Mackinnon <tim(a)testit.works> wrote: >> >>> >> >>> I was surprised to find that a Bag canât convert to a dictionary - >> e.g. >> >>> >> >>> Bag new >> >>> addAll: 'aabbbbcdddâ; >> >>> asDictionary >> >>> >> >>> Gives an error - Dnu #key >> >>> >> >>> >> >>> It looks to me like Bag is inheriting a bad version of >> #associationsDo: and instead could simply forward it to >> #doWithOccurences: >> instead? >> >>> >> >>> I know Bag is not used that much - but it comes up a lot in >> programming exercises. >> >>> >> >>> Tim >> >> >> >> >> > >> >> >>

March 7, 2019

Re: [Pharo-users] Google Protobuf and usage of Slots

by Holger Freyther

> On 5. Mar 2019, at 19:06, HenrikNergaard5 <h_n91(a)hotmail.com> wrote: > > Hi Holger, > >> My plan of action is to start with a protobuf compiler/model and then look >> into binding the gRPC C implementation to Pharo. > > I have an almost complete parser and compiler (class generation) > implementation of protobuf in Pharo around somewhere... > I cant quite remember the state of it, but I think everything is implemented > parsing wise expect the gRPC part. > > Instead of using slots the compiler generates accessor methods for each > variable, initialization, custom encoding and decoding based on its message > definition. That would be good timing. I have hand created enough descriptions to be able to materialize the descriptor.proto (the model definition in itself) and class generation is next. I have experimented (by using the right super class) with automatic selector creation and I think it will work if tweaked to handle collections/repeated fields correctly. A class/type definition currently reads as: PBTypeMessage subclass: #GBPFieldDescriptorProto slots: { #name => (PBTypeString asSlot fieldName: 'name'; fieldNumber: 1; beOptional). #extendee => (PBTypeString asSlot fieldName: 'extendee'; fieldNumber: 2; beOptional). #number => (PBTypeInt32 asSlot fieldName: 'number'; fieldNumber: 3; beOptional). #label => (GBPFieldDescriptorProto_Label asSlot fieldName: 'label'; fieldNumber: 4; beOptional). #type => (GBPFieldDescriptorProto_Type asSlot fieldName: 'type'; fieldNumber: 5; beOptional). #type_name => (PBTypeString asSlot fieldName: 'type_name'; fieldNumber: 6; beOptional). #default_value => (PBTypeString asSlot fieldName: 'default_value'; fieldNumber: 7; beOptional). #options => (GBPFieldOptions asSlot fieldName: 'options'; fieldNumber: 8; beOptional). #json_name => (PBTypeString asSlot fieldName: 'json_name'; fieldNumber: 10; beOptional) } classVariables: { } package: 'Protobuf-Compiler-Definitions' > i can probably dig it up and make it publicly available if it is of > interest. That would be helpful! > > Best regards, > Henrik > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >

March 7, 2019

Re: [Pharo-users] Why can't a Bag answer as a dictionary?

by Richard O'Keefe

Handling <some bag> as: Dictionary is really quite straightforward. Add a one-line method to Bag: Bag>> associationsDo: aBlock contents associationsDo: aBlock. This is a good idea anyway, because currently there is a bug where aBag associationsDo: aBlock fails to pass Associations to aBlock. Once you have done that, 'abracadabra' asBag as: Dictionary just works. This one simple addition - makes aBag associationsDo: aBlock work when it didn't before - makes aBag asDictionary work - makes aBag as: Dictionary work - makes Dictionary newFrom: aBag work On Thu, 7 Mar 2019 at 07:35, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > I am still not convinced. > > > On 6 Mar 2019, at 19:24, Tim Mackinnon <tim(a)testit.works> wrote: > > > > I still am tempted to make asDictionary work - because the the moment > you get a #key , DNU error - which is definitely not right. > > The error might not be super clear, but it is not wrong, the elements of > your collection do not respond to #key, a necessity to create a dictionary > from a collection of associations/pairs. > > Also, if you modify #asDictionary like that, then > > <some bag> as: #Dictionary > > will still fail. > > This is all not so simple as it seems. > > > So should Bag asDictionary give you a reasonable dictionary, or should > it throw a proper exception - something like: > > > > DomainError signal: â#asDictionary not supported, use #valueAndCountsâ > > > > Is that better - or do we accept that it inherits it from Collection and > so can do something useful? > > > > Tim > > > >> On 6 Mar 2019, at 18:11, phil(a)highoctane.be wrote: > >> > >> And when asking the object: > >> > >> #(1 2 4 5 6 7 2 1 4 3) asBag isDictionary > >> > >> we get false. So, not, not a dictionary. > >> > >> I still have been using #valuesAndCounts to get the dictionary out of a > bag on several occasions. > >> > >> And is useful indeed. > >> > >> #(1 2 4 5 6 7 2 1 4 3) asBag occurrencesOf: 2 > >> > >> and > >> > >> #(1 2 4 5 6 7 2 1 4 3) asBag valuesAndCounts at: 2 > >> > >> seem pretty much the same. > >> > >> But in this age of streams/reactive streams and lazy evaluations, what > do we have in Pharo on that front? > >> > >> Phil > >> > >> > >> > >> > >> > >> On Wed, Mar 6, 2019 at 4:49 PM Sven Van Caekenberghe <sven(a)stfx.eu> > wrote: > >> I was just explaining how it is now, what I think the rationale is > behind it. > >> > >> I understand #asDictionary as working on a collection of > pairs/associations (because it basically goes to #withAll:). > >> > >> A bag is just a collection that is optimised for many duplicates, the > fact that you have values and counts is more an implementation detail than > an intrinsic property. > >> > >> The conversion that you want, and that already exists in > #valuesAndCounts is one interpretation of what a bag is, not the only one. > A bag is foremost a collection of things. > >> > >> I am not immediately convinced that #valuesAndCounts should be the > default #asDictionary interpretation. > >> > >> What about the inverse for example ? > >> > >> { #foo->2. #bar->3 } asDictionary asBag. > >> > >> But this is certainly an interesting discussion. > >> > >> > On 6 Mar 2019, at 16:23, Tim Mackinnon <tim(a)testit.works> wrote: > >> > > >> > As Richard said - as a bag is relationship between keys and > frequencies, I would expect it to be able to convert to a dictionary. > >> > > >> > It displays in an inspector just like a Dictionary - which is why I > figured I could convert to pass it back to the exercise that was written > with Dictionaries in mind. > >> > > >> > <PastedGraphic-3.png> > >> > > >> > > >> > The code to fix it is quite simple, but on these kinds of things - I > thought it worth checking before submitting a PR. > >> > > >> > Tim > >> > > >> > > >> >> On 6 Mar 2019, at 13:53, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> >> > >> >> Why would that work ? What would you expect the output to be ? > >> >> > >> >> Try: > >> >> > >> >> #(1 2 3) asDictionary > >> >> > >> >> it fails in exactly the same way. You need key/value pairs > (Associations). > >> >> > >> >> These do work > >> >> > >> >> Bag new add: #foo->100; asDictionary. > >> >> > >> >> Bag new addAll: 'ABABABAAAA'; valuesAndCounts. > >> >> > >> >>> On 6 Mar 2019, at 14:25, Tim Mackinnon <tim(a)testit.works> wrote: > >> >>> > >> >>> I was surprised to find that a Bag canât convert to a dictionary - > e.g. > >> >>> > >> >>> Bag new > >> >>> addAll: 'aabbbbcdddâ; > >> >>> asDictionary > >> >>> > >> >>> Gives an error - Dnu #key > >> >>> > >> >>> > >> >>> It looks to me like Bag is inheriting a bad version of > #associationsDo: and instead could simply forward it to #doWithOccurences: > instead? > >> >>> > >> >>> I know Bag is not used that much - but it comes up a lot in > programming exercises. > >> >>> > >> >>> Tim > >> >> > >> >> > >> > > >> > >> > >> > > > > >

March 7, 2019

Re: [Pharo-users] Why can't a Bag answer as a dictionary?

by Richard O'Keefe

To me, a bag is first and foremost a multiset. Yes, it's a collection, but it's not *just* a collection. It's a very specific kind of collection with a very salient "characteristic function". In my own Smalltalk library, #asDictionary isn't even *defined* on things that are not dictionaries or some kind of mapping, because let's face it, a dictionary *isn't* a collection of Associations, and trying to make it look like one made Smalltalk-80 rather inconsistent. For example, if a dictionary were a collection of associations, you would expect (Dictionary with: #a -> 1 with: #b -> 2) includes: #a -> 1 to be true. No, it's false. You would expect (Dictionary with: #a -> 1 with: #b -> 2) asArray to be (a->1 b->2). No, it's (1 2). In order to avoid major confusion, I had to learn never to think of Associations in connection with Dictionaries. So I expect #asDictionary to have something to do with #keysAndValuesDo:. or something like that. For {1->2. 3->4} asDictionary I would have to write Dictionary withAllAssociations: {1->2. 3->4} except that I never actually found a use for it. #valuesAndCounts is a method that violates good OO practice, because it exposes (what you'd expect to be) private state. For example, b := 'abracadabra' asBag. b valuesAndCounts at: $a put: 'BOOM!'. b select, Print It, BOOM!. It should *copy* the dictionary. On Thu, 7 Mar 2019 at 04:50, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > I was just explaining how it is now, what I think the rationale is behind > it. > > I understand #asDictionary as working on a collection of > pairs/associations (because it basically goes to #withAll:). > > A bag is just a collection that is optimised for many duplicates, the fact > that you have values and counts is more an implementation detail than an > intrinsic property. > > The conversion that you want, and that already exists in #valuesAndCounts > is one interpretation of what a bag is, not the only one. A bag is foremost > a collection of things. > > I am not immediately convinced that #valuesAndCounts should be the default > #asDictionary interpretation. > > What about the inverse for example ? > > { #foo->2. #bar->3 } asDictionary asBag. > > But this is certainly an interesting discussion. > > > On 6 Mar 2019, at 16:23, Tim Mackinnon <tim(a)testit.works> wrote: > > > > As Richard said - as a bag is relationship between keys and frequencies, > I would expect it to be able to convert to a dictionary. > > > > It displays in an inspector just like a Dictionary - which is why I > figured I could convert to pass it back to the exercise that was written > with Dictionaries in mind. > > > > <PastedGraphic-3.png> > > > > > > The code to fix it is quite simple, but on these kinds of things - I > thought it worth checking before submitting a PR. > > > > Tim > > > > > >> On 6 Mar 2019, at 13:53, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> > >> Why would that work ? What would you expect the output to be ? > >> > >> Try: > >> > >> #(1 2 3) asDictionary > >> > >> it fails in exactly the same way. You need key/value pairs > (Associations). > >> > >> These do work > >> > >> Bag new add: #foo->100; asDictionary. > >> > >> Bag new addAll: 'ABABABAAAA'; valuesAndCounts. > >> > >>> On 6 Mar 2019, at 14:25, Tim Mackinnon <tim(a)testit.works> wrote: > >>> > >>> I was surprised to find that a Bag canât convert to a dictionary - e.g. > >>> > >>> Bag new > >>> addAll: 'aabbbbcdddâ; > >>> asDictionary > >>> > >>> Gives an error - Dnu #key > >>> > >>> > >>> It looks to me like Bag is inheriting a bad version of > #associationsDo: and instead could simply forward it to #doWithOccurences: > instead? > >>> > >>> I know Bag is not used that much - but it comes up a lot in > programming exercises. > >>> > >>> Tim > >> > >> > > > > >

March 7, 2019

Re: [Pharo-users] How exactly is "share repositories between images" supposed to work without tripping each other up?

by Ben Coman

On Thu, 7 Mar 2019 at 02:16, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > I couldn't find a "simple enough" solution to deal with that, so I > moved to emulated Linux [1]. > > I don't know if the long file path is a limitation of Windows itself or > libgit2. > Its a combination, but for our purposes the limitation lies with libgit2. Issue open since April 2015... "Support `core.longpaths` on Windows " https://github.com/libgit2/libgit2/issues/3053#issuecomment-324363336 cheers -ben

March 6, 2019

Re: [Pharo-users] How exactly is "share repositories between images" supposed to work without tripping each other up?

by Dale Henrichs

On 3/6/19 8:18 AM, Sean P. DeNigris wrote: > Tim Mackinnon wrote >> how is this shared repository supposed to work? > While I initially liked the space efficiency of the shared approach, I > eventually gave up because it created too many (often obscure) problems. It > just doesn't seem to be a good match for git, although you can get away with > it in the simplest cases (read-only, no branch switching). The use case that I have used for years (tODE and GemStone) is that I am sharing 60+ git repositories amongst 60+ images and I want all of the images to be sharing the same versions of all of the git projects. When I want to do private work I create a clone that is local to the image that I'm working in and do my work on a topic branch ... when I'm done with the private work, I merge the topic branch back into my commonly shared branch and then update the shared git repository so that when I do activate one of the images. I can quickly tell (via tools) that i have not loaded the latest shared code into my image ... if I care (like preparing to do development), I update to the latest shared code, so that my changes can be shared... if I don't care, then I remember not to make any code changes that I want to keep:) The images that I am using are a mix different versions of GemStone, so I _do_ make it a practice to ensure that projects I work with are all loadable in the different versions of GemStone that I care about via baseline changes (I use packages for version-specific code and avoid putting version-specific code on different branches or tags) ... So, when you say it's not "a good match for git", I don't think that it is "git" that is the problem, but the use cases themselves are the culprit... different tool features are very likely needed to support different use cases ... Dale

March 6, 2019