March 2019 - Pharo-users - lists.pharo.org

Re: [Pharo-users] complex json parsing.

by Ben Coman

On Sun, 10 Mar 2019 at 23:19, Ben Coman <btc(a)openinworld.com> wrote: > > Now to mold the IDE to your domain... > using Spotter to browser gtInspector* methods, a promising find is... > AbstractFileReference>>gitInspectorJpegIn: > from which I produced... > > Painting >> gtInspectorJpegIn: composite > <gtInspectorPresentationOrder: 0> > composite morph > title: 'Painting'; > display: [ ImageReadWriter formFromStream: self imageEntity readStream ] > @all, can someone suggest a better #display: block that would shrink the image to the window size, and/or zooming. cheers -ben

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Ben Coman

On Sun, 10 Mar 2019 at 23:40, Roelof Wobben <r.wobben(a)home.nl> wrote: > Op 10-3-2019 om 16:19 schreef Ben Coman: > > > > On Sun, 10 Mar 2019 at 17:55, Roelof Wobben <r.wobben(a)home.nl> wrote: > >> I could do something like this : >> >> getImages >> | json numbers json2 | >> json := NeoJSONReader >> fromString: >> (ZnEasy >> get: >> ' >> https://www.rijksmuseum.nl/api/nl/collection?key=14OGzuak&format=json&type=… >> ') >> contents. >> numbers := self class fromJSON: json >> numbers do: [each | json := ......... >> imageData = self class ??? >> > > I'm not sure what "numbers" refers to. It seems a quite non-domain related > identifier. > To understand the domain, pasting the json contents of your link above > into https://jsonformatter.curiousconcept.com/ > and collapsing objects I see the structure is... > > <snip> Did you work through this example I provided? Please do so. In a clean image... > Start with only your original code (which was good btw)... > > Paintings class >> fromJSON: json > | instance artObjects | > instance := self new. > artObjects := json at: #artObjects. > artObjects > do: > [ :eachArtObject | instance addPainting: (Painting > fromJSON: eachArtObject) ]. > ^ instance > > except objectNumber(s) seem associated with each painting, so should be stored within the painting object, as ADDED here... Painting class >> fromJSON: json | instance | instance := self new. instance objectNumber: (json at: #objectNumber); "<<<<<<<<ADDED" title: (json at: #title); painter: (json at: #principalOrFirstMaker); imageUrl: ((json at: #webImage) at: #url). ^ instance Evaluate the following in Playground... collectionUrl := ' https://www.rijksmuseum.nl/api/nl/collection?key=14OGzuak&format=json&type=… '. collectionJson := NeoJSONReader fromString: (ZnEasy get: collectionUrl) contents. paintings := Paintings fromJSON: collectionJson. paintings inspect. then clean the DNU errors as they occur (i.e. click the <Create> button shown by the debugger, and fill in as follows...) when you save the methods, choose <Declare new instance variable>" Painting >> objectNumber: aString objectNumber := aString > Painting >> title: aString > title := aString > > Painting >> painter: aString > painter := aString > > Painting >> imageUrl: aString > imageUrl := aString > > Paintings >> addPainting: aPainting > paintings := paintings ifNil: [ OrderedCollection new ]. > paintings add: aPainting. > > and an inspector will appear showing a list of paintings, which you can > drill down to a painting and see each has the expected data. > Now just to round things off... > > Painting >> title > ^ title > > Painting >> printOn: aStream > super printOn: aStream. > aStream << ' (' << self title << ')' > > to help distinguish each item in the Inspector. > > > To extend the Playground code to download and display a painting, in > Playground evaluate... > > painting := paintings first. > imageResponse := ZnEasy get: painting imageUrl. > image := ImageReadWriter formFromStream: imageResponse entity > readStream. > image inspect. > > Paintings >> first > ^ paintings first > > Painting >> imageUrl > ^ imageUrl > > and an inspector on the `image` variable displays the painting on the > Morph tab. > > > Now to mold the IDE to your domain... > using Spotter to browser gtInspector* methods, a promising find is... > AbstractFileReference>>gitInspectorJpegIn: > from which I produced... > > Painting >> gtInspectorJpegIn: composite > <gtInspectorPresentationOrder: 0> > composite morph > title: 'Painting'; > display: [ ImageReadWriter formFromStream: self imageEntity readStream ] > > Then inspecting the `paintings` variable and drilling down to a painting > pops up a DNU #imageEntity, which can be resolved by... > > Painting >> imageEntity > ^ imageEntity ifNil: [ imageEntity := (ZnEasy get: self imageUrl) entity ]. > > and you get to see the painting shown in the Inspector. > > > Now if I understand your question... "Do I need to make some more object > to get this working..." > I'd say... No. You only want one object for each painting. Once you have a > painting object, it should handle all getting all further data it needs for > itself. > You don't want duplicate objects each having half of the data. > > > can I for example name the function fromJson2 or fromJSONFromLink2 ? >> > > By naming convention #fromJson: implies it sits on the class side. > To get further data for an existing object you want an instance-side > method, maybe named #getLink2Json. > > HTH, > cheers -ben > > > > oke > > numbers should be the objectNumbers because for the rest I need them > > > > I use a instance variable called painting to hold the data. > > > so I can change the call to get data for the painting to : > > Painting class >> fromJSON: json > | instance | > instance := self new. > instance > objectNumber: ((json at: #objectNumber). > ^ instance > > > so numbers should be a collection of 10 ObjectNumbers > I'm presuming that 10 objectNumbers relate to 10 different paintings, in which case, No, you should never need to deal with a collection of 10 objectNumbers. What you should have is a collection of painting-objects which know their own objectNumber and then iterate that collection sending each painting-object the message "getLink2Json". > so I should use a loop to make a call to the second api call > In Pharo OO approach, you don't "loop" on the "calls", you "iterate" on the "objects" asking them to use their own data to do the stuff they know how to do. i.e. Let each object take care of itself. Which class is getImages defined on? That is really important to understand your proposed solution. I'm just presuming its defined on the class side of Paintings since your are referring to the same URL as before. > > getImages > | json numbers | > json := NeoJSONReader > fromString: > (ZnEasy > get: > ' > https://www.rijksmuseum.nl/api/nl/collection?key=14OGzuak&format=json&type=… > ') > contents. > numbers painting := self class fromJSON: json > btw, it won't work with two identifiers "numbers painting" to the left of the assignment symbol. Space here bad. numbers do: [each | json := call to the second url.here I used each > imageData = self class > getLink2Json > json2 := call to the 3th url > otherData := self class > getLink3Json > > no :) the "self class" seems wrong. getting extra data about a painting should be handled by the instance side of its painting-object. > Then I can I hope on every json method fill in the data I need > > Do I understand you well > I think not so well yet :) (or else I am missing something) Keep trying. It will change the way you think about programming. Please work though the example I provided. cheers -ben

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Roelof Wobben

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Ben Coman

On Sun, 10 Mar 2019 at 17:55, Roelof Wobben <r.wobben(a)home.nl> wrote: > I could do something like this : > > getImages > | json numbers json2 | > json := NeoJSONReader > fromString: > (ZnEasy > get: > ' > https://www.rijksmuseum.nl/api/nl/collection?key=14OGzuak&format=json&type=… > ') > contents. > numbers := self class fromJSON: json > numbers do: [each | json := ......... > imageData = self class ??? > I'm not sure what "numbers" refers to. It seems a quite non-domain related identifier. To understand the domain, pasting the json contents of your link above into https://jsonformatter.curiousconcept.com/ and collapsing objects I see the structure is... { "elapsedMilliseconds":0, "count":4782, "countFacets":{ "hasimage":4359, "ondisplay":748 }, "artObjects":[ + ] , "facets":[ + ] } where artObjects is an array with each art object like this==> { "links":{ "self":"https://www.rijksmuseum.nl/api/nl/collection/SK-A-3580", "web":"https://www.rijksmuseum.nl/nl/collectie/SK-A-3580" }, "id":"nl-SK-A-3580", "objectNumber":"SK-A-3580", "title":"De Singelbrug bij de Paleisstraat in Amsterdam", "hasImage":true, "principalOrFirstMaker":"George Hendrik Breitner", "longTitle":"De Singelbrug bij de Paleisstraat in Amsterdam, George Hendrik Breitner, 1898", "showImage":true, "permitDownload":true, "webImage":{ "guid":"cd15e2fa-b1f6-41f4-9e9e-c2ca87abcca3", "offsetPercentageX":0, "offsetPercentageY":0, "width":2880, "height":1897, "url":" https://lh3.googleusercontent.com/Bawo7r1nPZV6sJ4OHZJHdKV_4Ky59vquAR7KoUXcN… " }, "headerImage":{ "guid":"fef39f74-9783-44c9-acae-69eac3a76b01", "offsetPercentageX":0, "offsetPercentageY":0, "width":1920, "height":460, "url":" https://lh3.googleusercontent.com/jR6y5a8lFvgYe37WxLPQJZTjBAaBB6aIFYMsFw9gG… " }, "productionPlaces":[ "Amsterdam" ] } So evaluating the following in Playground... collectionUrl := ' https://www.rijksmuseum.nl/api/nl/collection?key=14OGzuak&format=json&type=… '. collectionJson := NeoJSONReader fromString: (ZnEasy get: collectionUrl) contents. paintings := Paintings fromJSON: collectionJson. paintings inspect. then cleaning the DNU errors as i went, starting with your original code (which was good btw)... Paintings class >> fromJSON: json | instance artObjects | instance := self new. artObjects := json at: #artObjects. artObjects do: [ :eachArtObject | instance addPainting: (Painting fromJSON: eachArtObject) ]. ^ instance Painting class >> fromJSON: json | instance | instance := self new. instance title: (json at: #title); painter: (json at: #principalOrFirstMaker); imageUrl: ((json at: #webImage) at: #url). ^ instance I added (roughly in order that each DNU occurred)... Painting >> title: aString title := aString Painting >> painter: aString painter := aString Painting >> imageUrl: aString imageUrl := aString Paintings >> addPainting: aPainting paintings := paintings ifNil: [ OrderedCollection new ]. paintings add: aPainting. I get an inspector on a list of paintings and can drill down to a painting and see each has the expected data, and just to round things off... Painting >> title ^ title Painting >> printOn: aStream super printOn: aStream. aStream << ' (' << self title << ')' helps distinguish each item in the Inspector. Now to extend the Playground code to download and display a painting, in Playground I evaluated... painting := paintings first. imageResponse := ZnEasy get: painting imageUrl. image := ImageReadWriter formFromStream: imageResponse entity readStream. image inspect. Paintings >> first ^ paintings first Painting >> imageUrl ^ imageUrl and an inspector on the `image` variable displays the painting on the Morph tab. Now to mold the IDE to your domain... using Spotter to browser gtInspector* methods, a promising find is... AbstractFileReference>>gitInspectorJpegIn: from which I produced... Painting >> gtInspectorJpegIn: composite <gtInspectorPresentationOrder: 0> composite morph title: 'Painting'; display: [ ImageReadWriter formFromStream: self imageEntity readStream ] Then inspecting the `paintings` variable and drilling down to a painting pops up a DNU #imageEntity, which can be resolved by... Painting >> imageEntity ^ imageEntity ifNil: [ imageEntity := (ZnEasy get: self imageUrl) entity ]. and you get to see the painting shown in the Inspector. Now if I understand your question... "Do I need to make some more object to get this working..." I'd say... No. You only want one object for each painting. Once you have a painting object, it should handle all getting all further data it needs for itself. You don't want duplicate objects each having half of the data. can I for example name the function fromJson2 or fromJSONFromLink2 ? > By naming convention #fromJson: implies it sits on the class side. To get further data for an existing object you want an instance-side method, maybe named #getLink2Json. HTH, cheers -ben

March 10, 2019

Re: [Pharo-users] Pharo 7.0 image size

by Sven Van Caekenberghe

Hi, I am confused by what you are reporting. If you add 10.000 entries to the Transcript, then I don't see how that results in 1000s of those Strings being alive (and certainly not after a GC). I tried the following (both with the Transcript window open and closed): Transcript. Transcript open. Transcript clear. 1 to: 1e4 do: [ :each | Transcript crShow: DateAndTime now asString , ' MARKTEST ' , each asString ]. 3 timesRepeat: [ Smalltalk garbageCollect ]. [ | count | count := 0. String allSubInstancesDo: [ :each | (each includesSubstring: 'MARKTEST') ifTrue: [ count := count + 1 ] ]. count ] value. The count never went above 100. I am using Pharo 8 but 7 would be similar. If you are retaining so many of the messages that you wrote to the Transcript, then you must be doing something else. That being said, we do have an issue with the WriteStream held by the stream instance variable inside ThreadSafeTranscript. It is being reset from time to time and when #clear is called, but that does not bring down the size of the underlying collection below its 'one time peak'. Sven > On 8 Mar 2019, at 11:59, Trussardi Dario Romano <dario.trussardi(a)tiscali.it> wrote: > > Ciao, > > this morning i started with an image of about 240 MB. > > I have 3 System Browser open ( for a long time ). > > I close all the System Browser and save the image. > > The image size after the save is about 131 MB. > > The new SpaceTally printSpaceAnalysis report: > > Class code space # instances inst space percent inst average size > Array 4115 561766 37353480 25.60 66.49 > ByteString 2942 249761 31313136 21.50 125.37 > > > The Transcript entry reference basically does not change: > > ByteString allInstances select:[ :i | i includesSubstring: 'NETWORK' ] > > where 'NETWORK' is a string used in the methods to add Transcript entry Transcript show: ...... NETWORK' ......' . > > > The system found 2958 ByteString instances.... ( relative to the NETWORK ) > > The system found 22655 ByteString instances.... ( relative to the CASH ) > > > Why are there many references in the system to the ByteString instances, created in the methods to report the data in Transcript ? > > Someone to considerations ? > > Thanks, > > Dario > >> Ciao, >> >> I thought that using Transcript as a report to analyze the operation of the code was a good thing - solution. >> >> Unfortunately, however, the size of the image continues to increase ( and I think it's due to the use of the transcript ) >> >> and i can not find the solution to avoid the problem. >> >> I could create a file to report everything about the operations code, but then it becomes difficult to manage, or am i wrong? >> >> Other solutions - indications - references ? >> >> Thanks, >> >> Dario >> >>> Ciao, >>> >>> thanks. >>> >>> i have a Pharo 7.0 alpha build 1262. >>> >>> I development a Seaside application. >>> >>>> May be you have a lot of still open Seaside session, which if I remember >>>> correctly are automatically shutdown after 10 min when not active. >>>> >>>> Looking at Seaside session instances may give clue. >>> >>> OK, i clear the seaside session with the relative seaside status Clear action. >>> >>> >>> >>> But the image size keeps increasing, without an apparent reason. >>> >>> The consideration that I can do and that in the method code i often use Transcript show: '.....NETWORK | CASH | .....' to check the correct functioning. >>> >>> With the SpaceTally printSpaceAnalysis i note: >>> >>> as of February 16th: >>> >>> Class code space # in stances inst space percent inst average size >>> Array 4115 656764 43904528 23.30 66.85 >>> ByteString 2942 233871 27527272 14.60 117.70 >>> >>> >>> as of March 4th >>> Class code space # instances inst space percent inst average size >>> Array 4115 837017 50564552 23.40 60.41 >>> ByteString 2942 266620 31933216 14.80 119.77 >>> >>> Now i do: >>> >>> ByteString allInstances select:[ :i | i includesSubstring: 'NETWORK' ] >>> >>> where 'NETWORK' is a string used in the Transcript show: ...... NETWORK' ......' report entry. >>> >>> >>> The system found 3214 ByteString instances.... ( relative to the NETWORK ) >>> >>> The system found 22650 ByteString instances.... ( relative to the CASH ) >>> >>> The system found .............. >>> >>> >>> Because ????? >>> >>> >>> Does the system remember something about the string create for the Transcript show: entry ? >>> >>> I need to reset something? >>> >>> How can I analyze the situation? >>> >>> I follow some aByteString entry with the pointersTo method >>> >>> but for now I have not found the solution. >>> >>> >>> >>> Thanks, >>> >>> Dario >>> >>> >> >> > >

March 10, 2019

Re: [Pharo-users] Pharo and FPGA

by CÃ©drick BÃ©ler

Could we do discuss such subjects in one pharo day for instance ? May would be perfect for me. Cheers, CÃ©drick > Le 10 mars 2019 Ã 11:36, LoÃ¯c LAGADEC <loic.lagadec(a)ensta-bretagne.fr> a Ã©crit : > > Yes! > Feel free to get in touch :) > Cheers > LoÃ¯c > > LoÃ¯c Lagadec > Professor > Lab-STICC UMR CNRS 6285 > Ensta Bretagne Deputy Head of Research - Cybersecurity > > +33 (0)2 98 34 89 08 <tel:+33%202%2098%2034%2089%2008> > Loic.lagadec(a)ensta-bretagne.fr <mailto:Loic.lagadec@ensta-bretagne.fr> > > National Institute of Advanced Technology, Brittany > www.ensta-bretagne.fr <http://www.ensta-bretagne.fr/> - 2 rue FranÃ§ois Verny - 29806 Brest Cedex 9 <x-apple-data-detectors://4/2> > > > Le 10 mars 2019 Ã 09:12, CÃ©drick BÃ©ler <cdrick65(a)gmail.com <mailto:cdrick65@gmail.com>> a Ã©crit : > >> If I remember well, smalltalkers from britany have worked on FPGA design on visualwork. I think there were Loic Lagadec, Alain Plantec and Bernard Pottier. >> https://www.ensta-bretagne.fr/lagadec/ <https://www.ensta-bretagne.fr/lagadec/> >> >> >> Maybe they can help here ;-) >> >> Cheers, >> >> Cedrick >> >> Le 10 mars 2019 Ã 01:38, Ben Coman <btc(a)openinworld.com <mailto:btc@openinworld.com>> a Ã©crit : >> >>> not quite what you asked, but... apart from their size, one advantage of FPGAs and Arduino & ESP32 microprocessors >>> over purpose CPUs is providing deterministic real-time latency (i.e. sensor inputs controlling motors). >>> For that domain a strong contender in a Pi-like format is the Programmable Realtime Unit of the BeagleBone Black. >>> http://beagleboard.org/pru <http://beagleboard.org/pru> >>> >>> 40x faster response time than the main CPU of BeagleBone (and Pi). See slide 11... >>> https://elinux.org/images/1/1c/Birkett--enhancing_rt_capabilities_with_the_… <https://elinux.org/images/1/1c/Birkett--enhancing_rt_capabilities_with_the_…> >>> >>> cheers -ben >>> >>> On Sat, 9 Mar 2019 at 22:32, Norbert Hartl <norbert(a)hartl.name <mailto:norbert@hartl.name>> wrote: >>> Iâm trying to collect information about programming in pharo for FPGA. The only things I found are around PharoROS like this [1]. If anyone has more information or is willing to talk about Iâm open ears. >>> >>> The background is that we did little things with the PharoThings toolkit and want to research which other combinations like Arduino, ESP32, FPGAs are there to be a good extensions to a Pi. >>> >>> Any hint is welcomed! >>> >>> Thanks, >>> >>> Norbert >>> >>> [1] http://esug.org/data/ESUG2014/IWST/Papers/iwst2014_From%20Smalltalk%20to%20… <http://esug.org/data/ESUG2014/IWST/Papers/iwst2014_From%20Smalltalk%20to%20…>

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Roelof Wobben

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Roelof Wobben

March 10, 2019

Re: [Pharo-users] Pharo and FPGA

by CÃ©drick BÃ©ler

If I remember well, smalltalkers from britany have worked on FPGA design on visualwork. I think there were Loic Lagadec, Alain Plantec and Bernard Pottier. https://www.ensta-bretagne.fr/lagadec/ Maybe they can help here ;-) Cheers, Cedrick > Le 10 mars 2019 Ã 01:38, Ben Coman <btc(a)openinworld.com> a Ã©crit : > > not quite what you asked, but... apart from their size, one advantage of FPGAs and Arduino & ESP32 microprocessors > over purpose CPUs is providing deterministic real-time latency (i.e. sensor inputs controlling motors). > For that domain a strong contender in a Pi-like format is the Programmable Realtime Unit of the BeagleBone Black. > http://beagleboard.org/pru > > 40x faster response time than the main CPU of BeagleBone (and Pi). See slide 11... > https://elinux.org/images/1/1c/Birkett--enhancing_rt_capabilities_with_the_… > > cheers -ben > >> On Sat, 9 Mar 2019 at 22:32, Norbert Hartl <norbert(a)hartl.name> wrote: >> Iâm trying to collect information about programming in pharo for FPGA. The only things I found are around PharoROS like this [1]. If anyone has more information or is willing to talk about Iâm open ears. >> >> The background is that we did little things with the PharoThings toolkit and want to research which other combinations like Arduino, ESP32, FPGAs are there to be a good extensions to a Pi. >> >> Any hint is welcomed! >> >> Thanks, >> >> Norbert >> >> [1] http://esug.org/data/ESUG2014/IWST/Papers/iwst2014_From%20Smalltalk%20to%20…

March 10, 2019

Re: [Pharo-users] complex json parsing.

by Ben Coman

On Sun, 10 Mar 2019 at 04:55, Roelof Wobben <r.wobben(a)home.nl> wrote: > Hello, > > I try to make a app which displays images from a external api . > > I was hoping I could do it with only 1 json parsing but to get the right > size of images I need more then 1 > > Right now I have two classes with fromJson methods > > Painting class >> fromJSON: json > | instance | > instance := self new. > instance > title: (json at: #title); > painter: (json at: #principalOrFirstMaker); > imageUrl: ((json at: #webImage) at: #url). > ^ instance > > > Paintings class > fromJSON: json > > fromJSON: json > | instance artObjects | > instance := self new. > artObjects := json at: #artObjects. > artObjects > do: > [ :eachArtObject | instance addPainting: (Painting fromJSON: > eachArtObject) ]. > ^ instance > > but to get it working right , I have to do it like this : > > Painting class >> fromJSON: json > | instance | > instance := self new. > instance > title: (json at: #objectNumber); > ^ instance > > > but the problem is now I need a second call to this url : > > https://www.rijksmuseum.nl/api/nl/collection/<<objectNumber>/tiles?key=[API_KEY]&format=json > > > and then need a new imageUrl out of it. > and then have one object with objectNumber and the new url > > Also I have to make a call to this url : https://www.rijksmuseum.nl/api/nl/collection/<<<objectNumber>>?key=[API_KEY]&format=json > > and also get some data out of it so that a painting will be one object with the objectNumber, new imageurl > and some data out of the last call > > Do I need to make some more object to get this working or is there a way I can change the code I use now > to get things working > > Regards, > > Roelof > > Its hard to tell from your description. You say "the problem is now I need a second call" but you've not shown the code that does the calling, and you've not shown the first call, or where you are looping through multiple paintings (I'm guessing that is the requirement) You say "Do I need to make some more object", but are you referring to more Paintings objects or more Painting objects? So speaking only very generally, this might be implemented in two ways... A1. Request the json of paintings into a Paintings object A2 Loop that json requesting each individual painting json to fully initialized Painting object or... B1. Request the json of paintings into a Paintings object B2. Loop that json create empty Painting objects B3. Iterate those Painting objects with each one requesting its own json from the server, and can do multiple requests if necessary And btw, very very generally, if you see even a hint of one way to do it - do it that way first and see what insights arise from that to improve it. Particularly when exploring a new domain like an unfamiliar API, the usual approach is... 1. Make it work 2. Make it right 3. Make it fast Use whatever hackery you need to get step 1 done before moving to step 2, which you are inquiring about. And often you don't even need step 3. cheers -ben

March 10, 2019