Cédrick

In principle, what you are asking for is to identify 'islands' of structured information in a 'sea' of otherwise unstructured material, which is now a standard pattern in PetitParser.

Exactly :)

You could imagine a parser spec of the form:

(sea optional, (email/phone/address/....), sea optional) plus

Where email etc are parsers for the individual structures. As a parser this would probably lead to lots of backtracking and be hideously inefficient, but for a short text like an e-mail it could be usable.

Yes this is only for shot text like email or say a text selection + shortcut.

This also assumes that the items of interest are really structured; there could be many ways of writing phone numbers, for instance.

Phone numbers are actually not easy… I see them as a limited sequence of number (if not well structure) + eventually the +contrycode). I’d like fuzzy structuration actually, but would be perfectly ok with an initial crisp one. I find this is a nice pet project to dive into PetitParser. When you say "unstructured material ... is now a standard pattern in PetitParser », how could I begin exploring that ? Any tutorials ? Thanks Peter, Cédrick

HTH

Peter Kenny

-----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Cédrick Béler Sent: 07 March 2019 09:52 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Cc: Tudor Girba <tudor@tudorgirba.com> Subject: [Pharo-users] Parsing text to discover general data of interest (phone, email, address, ...)

Hi all,

I’ve often got the need to analyse some random unstructured text to discover (structured) information (in email for instance), to extract : - emails - telephone numbers - addresses - events - person names (according to a list of known persons), - etc…

Apple do it in email for instance (strangely, this is not generalized).

So my questions are : - do we have something equivalent in Smalltalk/Pharo ? (I didn’t find) - if not, what strategy would you use ? => I do really stupid text analysis (substrings, finding @, …, parsing according to the text structure when there is… kind of Soup parsing…) => I feel this is a job for PetitParser ? And would be a nice feet to the new GToolkit.

All ideas or suggestions are welcome ;-)

TIA,

Cédrick

When you say "unstructured material ... is now a standard pattern in PetitParser », how could I begin exploring that ? Any tutorials ?

I’ll load it and play around. https://github.com/kursjan/petitparser2/blob/master/README.md

Cedrick

Sorry for not answering sooner.

Do not worry, this is because I haden’t properly searched before answering.

You have probably found what you want.

Not yet. Loading.

I meant that ‘island + sea’ is now a standard pattern. Follow the link to the tutorial from the readme you referenced – the example of javascript in HTML might carry over well enough to your situation.

Perfect, Thanks again, Cédrick

Peter

From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Cédrick Béler Sent: 07 March 2019 11:20 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Parsing text to discover general data of interest (phone, email, address, ...)

...

When you say "unstructured material ... is now a standard pattern in PetitParser », how could I begin exploring that ? Any tutorials ?

I’ll load it and play around. https://github.com/kursjan/petitparser2/blob/master/README.md <https://github.com/kursjan/petitparser2/blob/master/README.md>

Hi Hernan, Really nice. I try it today. It might be what I need. I come back if installation pb. Cheers, Cédrick

Le 8 mars 2019 à 03:34, Hernán Morales Durand <hernan.morales@gmail.com> a écrit :

Hi Cédrick,

I wrote some years ago an interface to a named-entity recognizer: https://80738163270632.blogspot.com/2015/02/stner-interface-to-stanford-name...

I think that was Pharo 5, so you may want to check if there are load problems in current Pharo.

The blogger post didn't parsed correctly the output but for the input:

StSocketNERClient new tagText: 'Argentina President Kirchner has been asked to testify in court on the death of Alberto Nisman the crusading prosecutor who had accused her of conspiring to cover up involvement of Iran'

output would be:

'<location>Argentina</LOCATION> President <person>Kirchner</PERSON> has been asked to testify in court on the death of <person>Alberto Nisman</PERSON> the crusading prosecutor who had accused her of conspiring to cover up involvement of <location>Iran</LOCATION>'

Cheers,

Hernán

El jue., 7 mar. 2019 a las 6:53, Cédrick Béler (<cdrick65@gmail.com>) escribió:

...

Hi all,

I’ve often got the need to analyse some random unstructured text to discover (structured) information (in email for instance), to extract : - emails - telephone numbers - addresses - events - person names (according to a list of known persons), - etc…

Apple do it in email for instance (strangely, this is not generalized).

So my questions are : - do we have something equivalent in Smalltalk/Pharo ? (I didn’t find) - if not, what strategy would you use ? => I do really stupid text analysis (substrings, finding @, …, parsing according to the text structure when there is… kind of Soup parsing…) => I feel this is a job for PetitParser ? And would be a nice feet to the new GToolkit.

All ideas or suggestions are welcome ;-)

TIA,

Cédrick

You say that named entity recognition is not generalised beyond Mail, but the support library is there for anyone to use. See for example https://developer.apple.com/documentation/foundation/nslinguistictagger/iden... <https://developer.apple.com/documentation/foundation/nslinguistictagger/iden...>

Yes true.

In Python, you can use NLTK to do roughly the same.

There's no real point in reimplementing this stuff in Pharo. Just set up a separate process, send text to it, and receive results back.

I agree, that is an excellent option. "NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources <http://nltk.org/nltk_data/> such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum <http://groups.google.com/group/nltk-users>." Thanks for pointing NLTK. Great tool for sure. I agree that there no point in reimplementing but in this case of NLP, this might worth it as I think we have already all good foundations (Basic string manipulation, PP, Zn, XML, etc…). The hard stuff is probably the integration of 100 corpora and lexical ressources (http://www.nltk.org/nltk_data/)... The thing is I need a lighter version, based on my own (growing) corpora (my experience). So Hernan solution, PP, or straight string processing would do the job. But thanks, this is something I will explore too (especially the luges ressources). Cheers, Cédrick

On Thu, 7 Mar 2019 at 22:53, Cédrick Béler <cdrick65@gmail.com <mailto:cdrick65@gmail.com>> wrote: Hi all,

I’ve often got the need to analyse some random unstructured text to discover (structured) information (in email for instance), to extract : - emails - telephone numbers - addresses - events - person names (according to a list of known persons), - etc…

Apple do it in email for instance (strangely, this is not generalized).

So my questions are : - do we have something equivalent in Smalltalk/Pharo ? (I didn’t find) - if not, what strategy would you use ? => I do really stupid text analysis (substrings, finding @, …, parsing according to the text structure when there is… kind of Soup parsing…) => I feel this is a job for PetitParser ? And would be a nice feet to the new GToolkit.

All ideas or suggestions are welcome ;-)

TIA,

Cédrick