May 2017 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Fwd: Re: 11/05/17 - Tabular Data Structures for Data Analysis - Oleksandr Zaytsev

by Oleksandr Zaytsev

I would love to, but to go to Lille from my country I would need a visa. Which is not that easy to acquire. So maybe I will come to PharoDays 2018. And I will definitely try to come to ESUG Conference in September. Oleks On Tue, May 16, 2017 at 7:26 PM, <serge.stinckwich(a)gmail.com> wrote: > > > EnvoyÃ© de mon iPhone > > Le 11 mai 2017 Ã 11:43, "phil(a)highoctane.be" <phil(a)highoctane.be> a > Ã©crit : > > ---------- Message transfÃ©rÃ© ---------- > De : "phil(a)highoctane.be" <phil(a)highoctane.be> > Date : 11 mai 2017 10:54 > Objet : Re: 11/05/17 - Tabular Data Structures for Data Analysis - > Oleksandr Zaytsev > Ã : "Nick Papoylias" <npapoylias(a)gmail.com> > Cc : > > > > On Thu, May 11, 2017 at 10:20 AM, Nick Papoylias <npapoylias(a)gmail.com> > wrote: > >> >> >> On Thu, May 11, 2017 at 5:24 AM, Oleksandr Zaytsev <olk.zaytsev(a)gmail.com >> > wrote: >> >>> >>> *A. Work done* >>> >>> - Downloaded the threaded VM as suggested by Esteban Lorenzano to >>> make Iceberg work. And it does! I have successfully pushed my NeuralNetwork >>> code to GitHub: https://github.com/olekscode/MLNeuralNetwork >>> - Joined the PolyMath organization on GitHub >>> - Created a repository for the TabularDataset project >>> https://github.com/PolyMathOrg/TabularDataset >>> <https://github.com/PolyMathOrg/TabularDataset> as a part of >>> PolyMath organization on GitHub >>> - Fixed a PolyMath issue #25 and made a PR >>> - Read an article from Wolfram Mathematica documentation regarding >>> Dataset. It was one of the reading suggestions sent to me by Nick Papoylias >>> >>> >>> *B. Next steps* >>> >>> - Fix more issues of PolyMath, using Iceberg. I have to get used to >>> it by the time the coding phase starts >>> - Read the rest of Nick Papoylias's suggestions >>> >>> >>> *C. Help needed* >>> >>> - The Dataset in Wolfram, as well as Pandas in Python, has a very >>> advanced indexing system. Smalltalk has its own special conventions for >>> indexing, so I think that it would be great if I got familiar with them. >>> Could you suggest me some reading on this topic (what are the indexing >>> conventions in Smalltalk?). >>> For example, in Wolfram, I can write *dataset[[-1]]* to extract the >>> last row. But in Pharo indexes can not be negative. In Pharo I would say *dataset >>> last*. But how about *dataset[[-5]]*? >>> >>> This would be a good exercise for you ;) In Pharo you can easily add >> negative indexing yourself. >> >> *Hint:* You know the index of the last element, since this is the size >> of the collection, so... ;) >> >> No need for changes, this exists already. > > Use atWrap: index put: value and atWrap: with negative indexes. > 'hello' atWrap: -2 > > There is a specific version for Array using a primitive. > #[ 10 20 30 40 ] atWrap: -1 > > atWrap:0 gives you the last item. > atWrap: -1 gives 30 > > This is different from 0 based index languages. > > The interesing thing about atWrap: is that it uses modulo interally so you > do not need to care about that. > > ($/ split: 'abc/def/ghi/jkl') atWrap: -1 > --> 'ghi' > > The Matrix class has a bunch of things API wise but the class is highly > inefficient, doing copies all the time etc. It would be nice to have some > kind of futures/copy on write style things in there. > > I miss cbind and rbind. These are useful. I have some half baked super > inefficient implementations of these things for Matrix. > > https://stat.ethz.ch/R-manual/R-devel/library/base/html/cbind.html > > The ability to name columns is also nice to have. > > In R one does: > > df <- dataframe() > cbind(df, c(1,2,3)) > cbind(df, c(4,5,6)) > names(df)<-("C1", "C2", "C3") > names can be found back with: > > names(df) > > A Smalltalkish style would be welcome. > > > > > Interesting ! Are you coming to PharoDays ? We can talk about that if we > found time. > > Maybe looking at the Voyage queries can be helpful. > > Phil > > > >> Try adding an extention method to Ordrered or SequenceableCollection. >> >> If the Pharo by example chapter is not enough or the MOOC, read the source >> itself in the core, to see how basic methods are implemented (it is less >> scary, >> than it sounds). >> >> You can also try Chapters 9, 10, 11 of the blue book (some API changes >> may apply): >> >> <http://goog_1902892863> >> http://sdmeta.gforge.inria.fr/FreeBooks/BlueBook/Bluebook.pdf >> >> >>> - Or what is the best way of implementing this index: >>> *dataset[["name"]]* (extracts a named row), *dataset[[1]*] (extracts >>> the first row)? Should I create two separate messages: *dataset >>> rowNamed: 'name'* and *dataset rowAt: 1*? >>> >>> rowNamed: > rowAt: > > yes, look like it. > > But if we want to model things like R dataframes for example, this has to > be seen as a vectorized operation, so you can to use row slices, column > slices, and logical indexes. > > Check this out: > > http://www.r-tutor.com/r-introduction/data-frame/data-frame-row-slice > https://www.r-bloggers.com/working-with-data-frames/ > > > >> The internal representation of your data-structure can be anything at the >> moment, *as long as you encapsulate it.* >> >> (ie it can be nested OrderedCollections with meta-data for column-names >> to indexes, or dictionary of collections etc). >> >> *If you don't expose it to the user* (ie return it from the public api, >> or expect knowledge of it in argument passing), >> we can easily change it later. So *first make it work, and we optimize >> later ;)* >> >> For your case it will be a little bit trickier because *you also have >> the notions of a) rows and b) columns*, which >> are exposed to the user. So *you would need to create abstractions* for >> these too. >> >> Cheers, >> >> Nick >> >>> >>> - >>> >>> >>> If someone else is having problems with Iceberg on Linux, try >>> downloading the threaded VM: >>> >>> wget -O- get.pharo.org/vmT60 | bash >>> >>> And use SSH (not HTTPS) remote URL. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Pharo Google Summer of Code" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to pharo-gsoc+unsubscribe(a)googlegroups.com. >>> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/pharo-gsoc/CAEp0Uzu-8fw3dA6ezVoj-QptvLcB8cWPHvZ1tfLg1Ce8 >>> qkTqfQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/pharo-gsoc/CAEp0Uzu-8fw3dA6ezVoj-QptvLcB8…> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Pharo Google Summer of Code" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pharo-gsoc+unsubscribe(a)googlegroups.com. >> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/pharo-gsoc/CACEStOgLC6HbYJ8HBLHWfs5%2BwqN3ib_kdVGuVizx7G >> h1c0sM%3DA%40mail.gmail.com >> <https://groups.google.com/d/msgid/pharo-gsoc/CACEStOgLC6HbYJ8HBLHWfs5%2BwqN…> >> . >> For more options, visit https://groups.google.com/d/optout. >> > >

May 16, 2017

Re: [Pharo-users] HiDPI

by Steven R. Baker

I would be happy to help, I just don't know where to get started. -Steven On 16/05/17 18:01, Tim Mackinnon wrote: > I raised a similar thing with a new MacBook Pro a few months ago - currently Pharo is blurry and I also find it quite jolting moving from apps like Chrome or IntelliJ to a slightly fuzzier Pharo. > > Unfortunately - this isnât an easy fix as it seems like the solution is tied to Bloc. I believe progress is moving forward with Bloc, but Iâm keen to hear at Pharo days if there is some way to move higher res screen implications forward as well. > > I know many current Pharo developers either have lower res screens, or have just got used to it (Iâm not finding Iâm getting used to it) - but more importantly it makes it very difficult for me to show new developers how cool Pharo is as they all comment that it looks funny compare to their other tools. > > Of course, its open source, and we can all chip in - unfortunately this one seems like it needs quite specialised knowledge - but I think the desire is there. > > Tim > >> On 15 May 2017, at 17:52, Steven R. Baker <steven(a)stevenrbaker.com> wrote: >> >> Heya folks, >> >> I have a laptop with a 3K screen. Is it possible to tell Pharo to render >> everything at a double resolution or something? I feel like this ought >> to have been a solved problem already, sorry if it's really obvious and >> I just haven't found it. >> >> I use Debian if that matters; I suspect this just works out of the box >> on Macs. >> >> Cheers! >> >> -Steven >> >> >

May 16, 2017

Re: [Pharo-users] Fwd: Re: 11/05/17 - Tabular Data Structures for Data Analysis - Oleksandr Zaytsev

by serge.stinckwich＠gmail.com

EnvoyÃ© de mon iPhone > Le 11 mai 2017 Ã 11:43, "phil(a)highoctane.be" <phil(a)highoctane.be> a Ã©crit : > > ---------- Message transfÃ©rÃ© ---------- > De : "phil(a)highoctane.be" <phil(a)highoctane.be> > Date : 11 mai 2017 10:54 > Objet : Re: 11/05/17 - Tabular Data Structures for Data Analysis - Oleksandr Zaytsev > Ã : "Nick Papoylias" <npapoylias(a)gmail.com> > Cc : > > > >> On Thu, May 11, 2017 at 10:20 AM, Nick Papoylias <npapoylias(a)gmail.com> wrote: >> >> >>> On Thu, May 11, 2017 at 5:24 AM, Oleksandr Zaytsev <olk.zaytsev(a)gmail.com> wrote: >>> A. Work done >>> Downloaded the threaded VM as suggested by Esteban Lorenzano to make Iceberg work. And it does! I have successfully pushed my NeuralNetwork code to GitHub: https://github.com/olekscode/MLNeuralNetwork >>> Joined the PolyMath organization on GitHub >>> Created a repository for the TabularDataset project https://github.com/PolyMathOrg/TabularDataset as a part of PolyMath organization on GitHub >>> Fixed a PolyMath issue #25 and made a PR >>> Read an article from Wolfram Mathematica documentation regarding Dataset. It was one of the reading suggestions sent to me by Nick Papoylias >>> B. Next steps >>> Fix more issues of PolyMath, using Iceberg. I have to get used to it by the time the coding phase starts >>> Read the rest of Nick Papoylias's suggestions >>> C. Help needed >>> The Dataset in Wolfram, as well as Pandas in Python, has a very advanced indexing system. Smalltalk has its own special conventions for indexing, so I think that it would be great if I got familiar with them. Could you suggest me some reading on this topic (what are the indexing conventions in Smalltalk?). >>> For example, in Wolfram, I can write dataset[[-1]] to extract the last row. But in Pharo indexes can not be negative. In Pharo I would say dataset last. But how about dataset[[-5]]? >> This would be a good exercise for you ;) In Pharo you can easily add negative indexing yourself. >> >> Hint: You know the index of the last element, since this is the size of the collection, so... ;) >> > No need for changes, this exists already. > > Use atWrap: index put: value and atWrap: with negative indexes. > 'hello' atWrap: -2 > > There is a specific version for Array using a primitive. > #[ 10 20 30 40 ] atWrap: -1 > > atWrap:0 gives you the last item. > atWrap: -1 gives 30 > > This is different from 0 based index languages. > > The interesing thing about atWrap: is that it uses modulo interally so you do not need to care about that. > > ($/ split: 'abc/def/ghi/jkl') atWrap: -1 > --> 'ghi' > > The Matrix class has a bunch of things API wise but the class is highly inefficient, doing copies all the time etc. It would be nice to have some kind of futures/copy on write style things in there. > > I miss cbind and rbind. These are useful. I have some half baked super inefficient implementations of these things for Matrix. > > https://stat.ethz.ch/R-manual/R-devel/library/base/html/cbind.html > > The ability to name columns is also nice to have. > > In R one does: > > df <- dataframe() > cbind(df, c(1,2,3)) > cbind(df, c(4,5,6)) > names(df)<-("C1", "C2", "C3") > names can be found back with: > > names(df) > > A Smalltalkish style would be welcome. > Interesting ! Are you coming to PharoDays ? We can talk about that if we found time. > Maybe looking at the Voyage queries can be helpful. > > Phil > > >> Try adding an extention method to Ordrered or SequenceableCollection. >> >> If the Pharo by example chapter is not enough or the MOOC, read the source >> itself in the core, to see how basic methods are implemented (it is less scary, >> than it sounds). >> >> You can also try Chapters 9, 10, 11 of the blue book (some API changes may apply): >> >> http://sdmeta.gforge.inria.fr/FreeBooks/BlueBook/Bluebook.pdf >> >>> Or what is the best way of implementing this index: dataset[["name"]] (extracts a named row), dataset[[1]] (extracts the first row)? Should I create two separate messages: dataset rowNamed: 'name' and dataset rowAt: 1? > rowNamed: > rowAt: > > yes, look like it. > > But if we want to model things like R dataframes for example, this has to be seen as a vectorized operation, so you can to use row slices, column slices, and logical indexes. > > Check this out: > > http://www.r-tutor.com/r-introduction/data-frame/data-frame-row-slice > https://www.r-bloggers.com/working-with-data-frames/ > > >> The internal representation of your data-structure can be anything at the moment, as long as you encapsulate it. >> >> (ie it can be nested OrderedCollections with meta-data for column-names to indexes, or dictionary of collections etc). >> >> If you don't expose it to the user (ie return it from the public api, or expect knowledge of it in argument passing), >> we can easily change it later. So first make it work, and we optimize later ;) >> >> For your case it will be a little bit trickier because you also have the notions of a) rows and b) columns, which >> are exposed to the user. So you would need to create abstractions for these too. >> >> Cheers, >> >> Nick >>> >>> >>> If someone else is having problems with Iceberg on Linux, try downloading the threaded VM: >>> wget -O- get.pharo.org/vmT60 | bash >>> And use SSH (not HTTPS) remote URL. >>> -- >>> You received this message because you are subscribed to the Google Groups "Pharo Google Summer of Code" group. >>> To unsubscribe from this group and stop receiving emails from it, send an email to pharo-gsoc+unsubscribe(a)googlegroups.com. >>> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >>> To view this discussion on the web visit https://groups.google.com/d/msgid/pharo-gsoc/CAEp0Uzu-8fw3dA6ezVoj-QptvLcB8…. >>> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups "Pharo Google Summer of Code" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to pharo-gsoc+unsubscribe(a)googlegroups.com. >> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/msgid/pharo-gsoc/CACEStOgLC6HbYJ8HBLHWfs5%2BwqN…. >> For more options, visit https://groups.google.com/d/optout. >

May 16, 2017

Re: [Pharo-users] HiDPI

by Tim Mackinnon

I raised a similar thing with a new MacBook Pro a few months ago - currently Pharo is blurry and I also find it quite jolting moving from apps like Chrome or IntelliJ to a slightly fuzzier Pharo. Unfortunately - this isnât an easy fix as it seems like the solution is tied to Bloc. I believe progress is moving forward with Bloc, but Iâm keen to hear at Pharo days if there is some way to move higher res screen implications forward as well. I know many current Pharo developers either have lower res screens, or have just got used to it (Iâm not finding Iâm getting used to it) - but more importantly it makes it very difficult for me to show new developers how cool Pharo is as they all comment that it looks funny compare to their other tools. Of course, its open source, and we can all chip in - unfortunately this one seems like it needs quite specialised knowledge - but I think the desire is there. Tim > On 15 May 2017, at 17:52, Steven R. Baker <steven(a)stevenrbaker.com> wrote: > > Heya folks, > > I have a laptop with a 3K screen. Is it possible to tell Pharo to render > everything at a double resolution or something? I feel like this ought > to have been a solved problem already, sorry if it's really obvious and > I just haven't found it. > > I use Debian if that matters; I suspect this just works out of the box > on Macs. > > Cheers! > > -Steven > >

May 16, 2017

Loading Glorp+PostgreSQL

by Sven Van Caekenberghe

I am bringing my Reddit example back up to date (https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b53…) for an upcoming Pharo Booklet. I used to depend on and load Glorp+PostgreSQL in my Configuration as follows: spec project: 'Glorp' with: [ spec className: 'ConfigurationOfGlorp'; versionString: #stable; repository: 'http://www.smalltalkhub.com/mc/DBXTalk/Configurations/main' ]. spec project: 'PostgresV2' with: [ spec className: 'ConfigurationOfPostgresV2'; versionString: #bleedingEdge; repository: 'http://www.smalltalkhub.com/mc/PharoExtras/PostgresV2/main' ]. spec package: 'GlorpDriverPostgreSQL' with: [ spec repository: 'http://www.smalltalkhub.com/mc/PharoExtras/PostgresV2/main'; postLoadDoIt: #postLoadGlorpPostgresNativeDriverDBXTalkPharo; requires: #( 'Glorp' 'PostgresV2' ) ]. This works fine in Pharo 6 now. My first question is, is this the 'correct' way of loading Glorp+PostgreSQL ? Do I get the latest, maintained code this way ? My second question is, why is #stable not available for Pharo 6 for PostgresV2, what should be done to fix this and could someone add it ? Thx, Sven

May 16, 2017

Re: [Pharo-users] Problems loading XML System ( was [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding)

by monty

Something went wrong during your upgrade with class initialization. Installing the latest versions of these projects into a clean image would work, and so would installing the latest XMLParserHTML and XMLParserStAX into the newest Moose-6.1 image (which has the latest XMLParser and XPath). But if you insist on upgrading your old image, try the latest ConfigurationOfXMLParser (.303.mcz) and ConfigurationOfXPath (.149.mcz) from their PharoExtras repos and install their latest project versions, and do the same with XMLParserHTML and XMLParserStAX (the older versions aren't compatible with newer XMLParser versions). Then open the test runner and run all "XML|XPath" tests. If you get any failures, evaluate this: #('XML-Parser' 'XPath-Core') do: [:package | (SystemNavigation default allClassesInPackageNamed: package) do: [:class | class initialize]] and try running the tests again. > Sent: Monday, May 15, 2017 at 6:50 PM > From: PBKResearch <peter(a)pbkresearch.co.uk> > To: "'Any question about pharo is welcome'" <pharo-users(a)lists.pharo.org> > Subject: [Pharo-users] Problems loading XML System ( was Re: [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding) > > Monty > > As an update, I have rebuilt from the Moose 6.0 download. The version of XML-Parser in that was dated 18 July 2016 (configuration monty.233), so I installed versions of XML-Parser-HTML and XML-Parser-StAX contemporary with that. (The respective configurations are monty.48 and monty.39). With these versions all my previous XMLHTMLParser operations work as before, and I have been able to use the StAX parser in a simple way. So I can start exploring as I intended. > > I have made repeated attempts to update this rebuilt image to more recent versions of the HTML and StAX parsers, and every time I run into the same error reported below. I started from the latest version and worked backwards, but gave up quickly; it takes about 6 minutes on my machine to load and compile a version, and it soon gets tedious. If I feel more enthusiastic tomorrow, I might start working forwards from my current versions. > > Anyway, I now have a working system with the StaX and HTML parsers, so I can continue to explore. > > Best wishes > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of PBKResearch > Sent: 15 May 2017 20:44 > To: 'Any question about pharo is welcome' <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding > > Monty > > I have just started trying to use the StAX parsers, and I have found that the update has introduced a problem, which means that XMLHTMLParser no longer works on examples I have used before. I updated to ConfigurationOfXMLParser(monty.302), which is the latest version on the smalltalkhub repository, and then used the load version in the class comment, which loads the stable default. Similarly, I loaded ConfigurationOfXMLParserHTML(monty.62) and ConfigurationOfXMLParserStAX(monty.51), again using stable and default. When I try to run the XMLHTMLParser example I quoted below, I get an error message 'MessageNotunderstood: receiver of "critical:" is nil'. The same message comes up with anything else I try with XMLHTMLParser or with StAXHTMLParser. > > I am not really up to using the debugger on someone else's code, but the one thing I can see is that the problem lies in XMLKeyValueCache>>critical:, which has the code: > ^ self mutex critical: aBlock > The problem being that mutex is nil. > > In my enthusiasm, I saved the updated image with the same name as the old image, which is now therefore overwritten. If I cannot solve this problem, my only way out is to rebuild my image from the Moose 6.0 download. Any suggestions gratefully received. > > Thanks in advance > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of PBKResearch > Sent: 15 May 2017 19:16 > To: 'Any question about pharo is welcome' <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding > > Monty > > Many thanks for this. My original purpose was just to answer Paul deBruicker's query, namely to parse an html file and stop reading at the end of the <head> section. I solved this by trial and error using the code shown below ( which actually stops at the opening tag of the body). This was not my problem at all, but Paul's; I just tackled it for fun. > > However, you note has prompted me to update my version of the whole XML system - I was using the version I downloaded with Moose 6.0, which was dated August 2016. I am looking at the StAX parsers as a possible way of simplifying what I currently do, which involves downloading an entire web page as a DOM and then manipulating it with XPath to extract the bits I am interested in. I may be able to use StAX to do some of the selection and manipulation as I am reading. > > It's all a new topic to me, so I foresee a lot of experimentation. It all helps to keep the grey matter active. > > Thanks again > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty > Sent: 15 May 2017 12:15 > To: pharo-users(a)lists.pharo.org > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding > > For that kind of incremental parsing, you could also use XMLParserStAX, a pull-parser that parses a document as a stream of event objects you control with #next, #peek, and #atEnd. It also supports pull-DOM parsing with messages like #nextNode, #nextElement, and #nextElementNamed:, which return the next event object(s) as DOM subtrees (searchable with XPath). See the StAXParser class comment for an example. (The StAXHTMLParser class requires XMLParserHTML be installed to work.) > > > > >

May 16, 2017

Problems loading XML System ( was Re: [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding)

by PBKResearch

Monty As an update, I have rebuilt from the Moose 6.0 download. The version of XML-Parser in that was dated 18 July 2016 (configuration monty.233), so I installed versions of XML-Parser-HTML and XML-Parser-StAX contemporary with that. (The respective configurations are monty.48 and monty.39). With these versions all my previous XMLHTMLParser operations work as before, and I have been able to use the StAX parser in a simple way. So I can start exploring as I intended. I have made repeated attempts to update this rebuilt image to more recent versions of the HTML and StAX parsers, and every time I run into the same error reported below. I started from the latest version and worked backwards, but gave up quickly; it takes about 6 minutes on my machine to load and compile a version, and it soon gets tedious. If I feel more enthusiastic tomorrow, I might start working forwards from my current versions. Anyway, I now have a working system with the StaX and HTML parsers, so I can continue to explore. Best wishes Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of PBKResearch Sent: 15 May 2017 20:44 To: 'Any question about pharo is welcome' <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding Monty I have just started trying to use the StAX parsers, and I have found that the update has introduced a problem, which means that XMLHTMLParser no longer works on examples I have used before. I updated to ConfigurationOfXMLParser(monty.302), which is the latest version on the smalltalkhub repository, and then used the load version in the class comment, which loads the stable default. Similarly, I loaded ConfigurationOfXMLParserHTML(monty.62) and ConfigurationOfXMLParserStAX(monty.51), again using stable and default. When I try to run the XMLHTMLParser example I quoted below, I get an error message 'MessageNotunderstood: receiver of "critical:" is nil'. The same message comes up with anything else I try with XMLHTMLParser or with StAXHTMLParser. I am not really up to using the debugger on someone else's code, but the one thing I can see is that the problem lies in XMLKeyValueCache>>critical:, which has the code: ^ self mutex critical: aBlock The problem being that mutex is nil. In my enthusiasm, I saved the updated image with the same name as the old image, which is now therefore overwritten. If I cannot solve this problem, my only way out is to rebuild my image from the Moose 6.0 download. Any suggestions gratefully received. Thanks in advance Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of PBKResearch Sent: 15 May 2017 19:16 To: 'Any question about pharo is welcome' <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding Monty Many thanks for this. My original purpose was just to answer Paul deBruicker's query, namely to parse an html file and stop reading at the end of the <head> section. I solved this by trial and error using the code shown below ( which actually stops at the opening tag of the body). This was not my problem at all, but Paul's; I just tackled it for fun. However, you note has prompted me to update my version of the whole XML system - I was using the version I downloaded with Moose 6.0, which was dated August 2016. I am looking at the StAX parsers as a possible way of simplifying what I currently do, which involves downloading an entire web page as a DOM and then manipulating it with XPath to extract the bits I am interested in. I may be able to use StAX to do some of the selection and manipulation as I am reading. It's all a new topic to me, so I foresee a lot of experimentation. It all helps to keep the grey matter active. Thanks again Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty Sent: 15 May 2017 12:15 To: pharo-users(a)lists.pharo.org Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding For that kind of incremental parsing, you could also use XMLParserStAX, a pull-parser that parses a document as a stream of event objects you control with #next, #peek, and #atEnd. It also supports pull-DOM parsing with messages like #nextNode, #nextElement, and #nextElementNamed:, which return the next event object(s) as DOM subtrees (searchable with XPath). See the StAXParser class comment for an example. (The StAXHTMLParser class requires XMLParserHTML be installed to work.)

May 15, 2017

Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding

by PBKResearch

Monty I have just started trying to use the StAX parsers, and I have found that the update has introduced a problem, which means that XMLHTMLParser no longer works on examples I have used before. I updated to ConfigurationOfXMLParser(monty.302), which is the latest version on the smalltalkhub repository, and then used the load version in the class comment, which loads the stable default. Similarly, I loaded ConfigurationOfXMLParserHTML(monty.62) and ConfigurationOfXMLParserStAX(monty.51), again using stable and default. When I try to run the XMLHTMLParser example I quoted below, I get an error message 'MessageNotunderstood: receiver of "critical:" is nil'. The same message comes up with anything else I try with XMLHTMLParser or with StAXHTMLParser. I am not really up to using the debugger on someone else's code, but the one thing I can see is that the problem lies in XMLKeyValueCache>>critical:, which has the code: ^ self mutex critical: aBlock The problem being that mutex is nil. In my enthusiasm, I saved the updated image with the same name as the old image, which is now therefore overwritten. If I cannot solve this problem, my only way out is to rebuild my image from the Moose 6.0 download. Any suggestions gratefully received. Thanks in advance Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of PBKResearch Sent: 15 May 2017 19:16 To: 'Any question about pharo is welcome' <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding Monty Many thanks for this. My original purpose was just to answer Paul deBruicker's query, namely to parse an html file and stop reading at the end of the <head> section. I solved this by trial and error using the code shown below ( which actually stops at the opening tag of the body). This was not my problem at all, but Paul's; I just tackled it for fun. However, you note has prompted me to update my version of the whole XML system - I was using the version I downloaded with Moose 6.0, which was dated August 2016. I am looking at the StAX parsers as a possible way of simplifying what I currently do, which involves downloading an entire web page as a DOM and then manipulating it with XPath to extract the bits I am interested in. I may be able to use StAX to do some of the selection and manipulation as I am reading. It's all a new topic to me, so I foresee a lot of experimentation. It all helps to keep the grey matter active. Thanks again Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty Sent: 15 May 2017 12:15 To: pharo-users(a)lists.pharo.org Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding For that kind of incremental parsing, you could also use XMLParserStAX, a pull-parser that parses a document as a stream of event objects you control with #next, #peek, and #atEnd. It also supports pull-DOM parsing with messages like #nextNode, #nextElement, and #nextElementNamed:, which return the next event object(s) as DOM subtrees (searchable with XPath). See the StAXParser class comment for an example. (The StAXHTMLParser class requires XMLParserHTML be installed to work.) > Sent: Friday, May 12, 2017 at 5:30 AM > From: PBKResearch <peter(a)pbkresearch.co.uk> > To: "'Any question about pharo is welcome'" > <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte > for utf-8 encoding > > With reference to Norbert's comment, there /may/ be an ambiguity about > the word 'header' in Udo's reply. It could refer to the http HEAD > section, in which case Norbert is of course right. It could also refer > to the <head> section of the html file, which is part of the content of the http response. > If it is the latter, this is similar to a question that Paul > deBruicker posted last November ("[Pharo-users] ZnClient GET, but just > the content of the <head> tag?"). I tried the method I devised for > Paul's case on Udo's problem website, and read the html header with no > problem. Incidentally, the header includes 'charset=iso-8859-1', which > does not agree with Sven's findings. > > In case it is of interest, I used XMLHTMLParser to read and parse the > header. Try the following in a Playground: > > par := XMLHTMLParser onURL: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'. > par parseDocumentUntil: [|top|(top := par topNode) notNil and: [ top > isElement and:[ top isNamed: 'body']]]. > par parsingResult findElementNamed: 'head'. > > If you 'Do it and go', the full header appears. The way I get it to > stop after the header may not be quite correct, because it uses > XMLHTMLParser>>topNode, which is a private method. On the other hand, > XMLHTMLParser>>I > can't see how to make the stop condition for > XMLHTMLParser>>parseDocumentUntil: depend on the parsed results > XMLHTMLParser>>without > using a private method. > > Hope this is helpful > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On > Behalf Of Norbert Hartl > Sent: 12 May 2017 08:04 > To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte > for > utf-8 encoding > > Just to mention. If you are not interested in the content body you > could do a HEAD request instead of GET. > > Norbert > > > Am 11.05.2017 um 22:44 schrieb Udo Schneider > <udo.schneider(a)homeaddress.de>: > > > > Hi Sven, > > > > that's perfect. To be honest I don't care about the content - I'm > > just > parsing the header. And even if there is a wrong decoding in there... > I can live with that. > > > > Thank you very very much! For your help but also your stuff in general. > > > > CU, > > > > Udo > > > > > >> Am 11/05/17 um 22:35 schrieb Sven Van Caekenberghe: > >> Hi Udo, > >>> On 11 May 2017, at 21:37, Udo Schneider > >>> <udo.schneider(a)homeaddress.de> > wrote: > >>> > >>> All, > >>> > >>> I'm hitting an error where fetching web content fails. The website > >>> does > indeed use invalid characters. > >>> > >>> The easiest way to reproduce: > >>> > >>> ZnEasy get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723' > >>> > >>> Is there any way to tell Zinc to simply ignore that error and to > continue? > >>> > >>> CU, > >>> > >>> Udo > >> That server/page has a mime-type text/plain with no explicit > >> encoding > (charset) setting, so we have to guess. Like utf-8, pure > latin1/iso88591 does not work. The following does work, but you can't > be sure everything went well (beLenient takes some bytes as they are). > >> ZnDefaultCharacterEncoder > >> value: ZnCharacterEncoder latin1 beLenient > >> during: [ > >> ZnClient new > >> get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'; > >> yourself ]. > >> I added some API earlier today, so that the following should also > >> work > (you need to load Zn #bleedingEdge first). > >> ZnClient new > >> defaultEncoder: ZnCharacterEncoder latin1 beLenient; > >> get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'; > >> yourself. > >> HTH, > >> Regards, > >> Sven > > > > > > > > > >

May 15, 2017

Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding

by PBKResearch

Monty Many thanks for this. My original purpose was just to answer Paul deBruicker's query, namely to parse an html file and stop reading at the end of the <head> section. I solved this by trial and error using the code shown below ( which actually stops at the opening tag of the body). This was not my problem at all, but Paul's; I just tackled it for fun. However, you note has prompted me to update my version of the whole XML system - I was using the version I downloaded with Moose 6.0, which was dated August 2016. I am looking at the StAX parsers as a possible way of simplifying what I currently do, which involves downloading an entire web page as a DOM and then manipulating it with XPath to extract the bits I am interested in. I may be able to use StAX to do some of the selection and manipulation as I am reading. It's all a new topic to me, so I foresee a lot of experimentation. It all helps to keep the grey matter active. Thanks again Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty Sent: 15 May 2017 12:15 To: pharo-users(a)lists.pharo.org Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte for utf-8 encoding For that kind of incremental parsing, you could also use XMLParserStAX, a pull-parser that parses a document as a stream of event objects you control with #next, #peek, and #atEnd. It also supports pull-DOM parsing with messages like #nextNode, #nextElement, and #nextElementNamed:, which return the next event object(s) as DOM subtrees (searchable with XPath). See the StAXParser class comment for an example. (The StAXHTMLParser class requires XMLParserHTML be installed to work.) > Sent: Friday, May 12, 2017 at 5:30 AM > From: PBKResearch <peter(a)pbkresearch.co.uk> > To: "'Any question about pharo is welcome'" > <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte > for utf-8 encoding > > With reference to Norbert's comment, there /may/ be an ambiguity about > the word 'header' in Udo's reply. It could refer to the http HEAD > section, in which case Norbert is of course right. It could also refer > to the <head> section of the html file, which is part of the content of the http response. > If it is the latter, this is similar to a question that Paul > deBruicker posted last November ("[Pharo-users] ZnClient GET, but just > the content of the <head> tag?"). I tried the method I devised for > Paul's case on Udo's problem website, and read the html header with no > problem. Incidentally, the header includes 'charset=iso-8859-1', which > does not agree with Sven's findings. > > In case it is of interest, I used XMLHTMLParser to read and parse the > header. Try the following in a Playground: > > par := XMLHTMLParser onURL: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'. > par parseDocumentUntil: [|top|(top := par topNode) notNil and: [ top > isElement and:[ top isNamed: 'body']]]. > par parsingResult findElementNamed: 'head'. > > If you 'Do it and go', the full header appears. The way I get it to > stop after the header may not be quite correct, because it uses > XMLHTMLParser>>topNode, which is a private method. On the other hand, > XMLHTMLParser>>I > can't see how to make the stop condition for > XMLHTMLParser>>parseDocumentUntil: depend on the parsed results > XMLHTMLParser>>without > using a private method. > > Hope this is helpful > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On > Behalf Of Norbert Hartl > Sent: 12 May 2017 08:04 > To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] [Zinc] ZnInvalidUTF8: Illegal leading byte > for > utf-8 encoding > > Just to mention. If you are not interested in the content body you > could do a HEAD request instead of GET. > > Norbert > > > Am 11.05.2017 um 22:44 schrieb Udo Schneider > <udo.schneider(a)homeaddress.de>: > > > > Hi Sven, > > > > that's perfect. To be honest I don't care about the content - I'm > > just > parsing the header. And even if there is a wrong decoding in there... > I can live with that. > > > > Thank you very very much! For your help but also your stuff in general. > > > > CU, > > > > Udo > > > > > >> Am 11/05/17 um 22:35 schrieb Sven Van Caekenberghe: > >> Hi Udo, > >>> On 11 May 2017, at 21:37, Udo Schneider > >>> <udo.schneider(a)homeaddress.de> > wrote: > >>> > >>> All, > >>> > >>> I'm hitting an error where fetching web content fails. The website > >>> does > indeed use invalid characters. > >>> > >>> The easiest way to reproduce: > >>> > >>> ZnEasy get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723' > >>> > >>> Is there any way to tell Zinc to simply ignore that error and to > continue? > >>> > >>> CU, > >>> > >>> Udo > >> That server/page has a mime-type text/plain with no explicit > >> encoding > (charset) setting, so we have to guess. Like utf-8, pure > latin1/iso88591 does not work. The following does work, but you can't > be sure everything went well (beLenient takes some bytes as they are). > >> ZnDefaultCharacterEncoder > >> value: ZnCharacterEncoder latin1 beLenient > >> during: [ > >> ZnClient new > >> get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'; > >> yourself ]. > >> I added some API earlier today, so that the following should also > >> work > (you need to load Zn #bleedingEdge first). > >> ZnClient new > >> defaultEncoder: ZnCharacterEncoder latin1 beLenient; > >> get: > 'http://www.darkreading.com/partner-perspectives/malwarebytes/locky-re > turns- with-a-new-(borrowed)-distribution-method/a/d-id/1328723'; > >> yourself. > >> HTH, > >> Regards, > >> Sven > > > > > > > > > >

May 15, 2017

Is there a bind key to go to next argument?

by Andreas Sunardi

Is there a bind key to jump to the next argument slot? After using TAB for method completion, I wonder if there's a key/command to jump to the next argument slot. I checked the keymap browser and either it isn't there or I missed it. The closest I've done so far is Meta + Right, but this needs to be performed 3 times to go to the next argument. -- Andreas

May 15, 2017