- Pharo-users - lists.pharo.org

Re: Implementing DataFrame>>dtypes feature in Pharo PolyMath Project

by Balaji G

Thank you sir for the clarification. On Sat, Jun 19, 2021 at 7:24 PM Konrad Hinsen <konrad.hinsen(a)cnrs.fr> wrote: > Dear Balaji, > > > I am working on implementing the DataFrame>>dtypes feature which > > checks the datatypes of columns in a DataFrame, as part of my GSOC > > project. I have tried to explain my theoretical work done so far on > > this blog post. Please kindly go through it , as I need advice on what > > could be the optimal way to implement this feature. Any kind of input > > and discussions is most welcome. > > Your post looks like an overall accurate description of the current > state of everything - with one exception, and that is Pandas. You say > you didn't look at the Pandas code yet, so that's not surprising. > > You seem to assume that Pandas stores Python objects as elements of > DataFrames, but that isn't true. Pandas uses NumPy arrays instead. And > NumPy arrays are very different from standard Python objects, because > their internal data layout is by design the same as used in C or > Fortran. For a full description, see > > https://numpy.org/doc/stable/user/basics.rec.html > > However, I am not sure you need to understand this in all detail, as I > am pretty sure that you do not want to copy this approach in Pharo. > > The one point that does matter for you is where NumPy and Pandas take a > column's dtype from. The answer is that it's defined when a DataFrame is > created, and it cannot be changed afterwards. If a column is "integer", > it will remain "integer" forever. If you try to assign a string to an > element of such a column, you get an error message. When you create a > DataFrame from existing data, e.g. by reading a CSV file, Pandas scans > the data and determines a suitable dtype, much in the same way as V1.0 > in Pharo/PolyMath did. But since Pandas doesn't allow any later change, > there is no serious performance issue. > > So that's an option you can add to your list: define the dtypes once and > for all when the DataFrame is created. The main drawback is that you > would have to change the API for DataFrame creation to make this work. > > Cheers, > Konrad > -- > --------------------------------------------------------------------- > Konrad Hinsen > Centre de Biophysique MolÃ©culaire, CNRS OrlÃ©ans > Synchrotron Soleil - Division ExpÃ©riences > Saint Aubin - BP 48 > 91192 Gif sur Yvette Cedex, France > Tel. +33-1 69 35 97 15 > E-Mail: konrad DOT hinsen AT cnrs DOT fr > http://dirac.cnrs-orleans.fr/~hinsen/ > ORCID: https://orcid.org/0000-0003-0330-9428 > Twitter: @khinsen > --------------------------------------------------------------------- >

June 19, 2021

Implementing DataFrame>>dtypes feature in Pharo PolyMath Project

by Balaji G

Hi I am working on implementing the DataFrame>>dtypes feature which checks the datatypes of columns in a DataFrame, as part of my GSOC project. I have tried to explain my theoretical work done so far on this blog post. Please kindly go through it , as I need advice on what could be the optimal way to implement this feature. Any kind of input and discussions is most welcome. Link to the post : Background work on DataFrame dtypes <https://balaji612141526.wordpress.com/2021/06/19/background-work-on-datafra…> Thank you Balaji G

June 19, 2021

Re: [ANN] Camp Smalltalk Supreme Finalized!

by Russ Whaley

I will be required to take personal time from my employer, so primarily weekend is appreciated from my perspective :) On Fri, Jun 18, 2021 at 11:04 AM James Foster <Smalltalk(a)jgfoster.net> wrote: > Is there anyone else out there for whom weekdays (M-F) would be better > than a primarily weekend event? > > James > > On Jun 17, 2021, at 11:31 AM, Richard Kenneth Eng < > horrido.hobbies(a)gmail.com> wrote: > > As of today, Wednesday, June 17, 2021, I have successfully nailed down the > most exciting roster of keynote speakers for *Camp Smalltalk Supreme* next > year. Here they are: > > 1. *Adele Goldberg* â co-creator of Smalltalk > 2. *Dan Ingalls* â co-creator of Smalltalk > 3. *Kent Beck* â Smalltalk pioneer > 4. *David Buck* â Smalltalk veteran and podcaster > > This will be the greatest programming language birthday bash ever. As I > wrote to Dan Ingalls... > > "I checked birthday celebrations for other languages like FORTRAN, LISP, > COBOL, BASIC, and Pascal, and they were quite anemic." > > For more information, visit: https://gofund.me/225341e3 > > I invite presenters to register. > > Cheers, > Richard > > > -- Russ Whaley whaley.russ(a)gmail.com

June 18, 2021

Re: [ANN] Camp Smalltalk Supreme Finalized!

by James Foster

Is there anyone else out there for whom weekdays (M-F) would be better than a primarily weekend event? James > On Jun 17, 2021, at 11:31 AM, Richard Kenneth Eng <horrido.hobbies(a)gmail.com> wrote: > > As of today, Wednesday, June 17, 2021, I have successfully nailed down the most exciting roster of keynote speakers for Camp Smalltalk Supreme next year. Here they are: > Adele Goldberg â co-creator of Smalltalk > Dan Ingalls â co-creator of Smalltalk > Kent Beck â Smalltalk pioneer > David Buck â Smalltalk veteran and podcaster > This will be the greatest programming language birthday bash ever. As I wrote to Dan Ingalls... > > "I checked birthday celebrations for other languages like FORTRAN, LISP, COBOL, BASIC, and Pascal, and they were quite anemic." > > For more information, visit: https://gofund.me/225341e3 <https://gofund.me/225341e3> > > I invite presenters to register. > > Cheers, > Richard

June 18, 2021

Re: Zn and AWS - retrieving object versions?

by Sven Van Caekenberghe

Tim, > On 17 Jun 2021, at 19:05, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > PS: I think there is other AWS S3 for Pharo out there, but I can't remember. I found two of them: - https://github.com/newapplesho/aws-sdk-smalltalk - https://github.com/jvdsandt/pharo-aws-toolbox I am sure both of these are more up to date and get more AWS details right that my old minimal, proof of concept package. Sven

June 18, 2021

[ANN] Camp Smalltalk Supreme Finalized!

by Richard Kenneth Eng

As of today, Wednesday, June 17, 2021, I have successfully nailed down the most exciting roster of keynote speakers for *Camp Smalltalk Supreme* next year. Here they are: 1. *Adele Goldberg* â co-creator of Smalltalk 2. *Dan Ingalls* â co-creator of Smalltalk 3. *Kent Beck* â Smalltalk pioneer 4. *David Buck* â Smalltalk veteran and podcaster This will be the greatest programming language birthday bash ever. As I wrote to Dan Ingalls... "I checked birthday celebrations for other languages like FORTRAN, LISP, COBOL, BASIC, and Pascal, and they were quite anemic." For more information, visit: https://gofund.me/225341e3 I invite presenters to register. Cheers, Richard

June 17, 2021

Re: Zn and AWS - retrieving object versions?

by Sven Van Caekenberghe

Tim, I think that from the moment there is a query part added to the REST URL something goes wrong. Step 1 is to make sure the signing works. Now, ZnAWSS3RequestSignatureTool was written for an earlier version (I don't remember which one, probably V1). This is tricky to figure out. I know for example that if there are multiple query arguments, they have to be sorted, which is currently not happening. But that won't be the only problem (encoding could be another). The challenge is to construct the correct 'canonical' string to sign. First you got a clear 'signature wrong' error, that seems to be gone. But you still get an authentication error, I am not sure what is going on. I am afraid you will have to try to debug this a bit yourself. It has been years since I looked at this code. If you get this to work with another tool, maybe you can enable debug logging to see what is happening with the URL and headers. Sven PS: I think there is other AWS S3 for Pharo out there, but I can't remember. > On 17 Jun 2021, at 18:32, Tim Mackinnon <tim(a)testit.works> wrote: > > Hi Sven - actually I think its not just the versions thing - I canât get filtering to work on the standard query either: > > client keysIn: âmtt-data' query: (Dictionary with: 'prefix'->'sampleâ). > > Given a normal query works, there must be something extra that is being missed/added that impacts things. > > Tim > >> On 17 Jun 2021, at 17:19, Tim Mackinnon <tim(a)testit.works> wrote: >> >> Hi Sven - thanks for taking a quick look, this change did stop me getting an immediate error - but I seem to get no results back. >> >> Digging into it a bit more - I seem to be issuing the GET request as indicated by https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html >> >> e.g. >> >> GET /?versions HTTP/1.1 >> >> Headers: >> a ZnMultiValueDictionary('User-Agent'->'Zinc HTTP Components 1.0 (Pharo/8.0)â >> 'Accept'->'*/*' 'Host'->'mtt-data.s3.amazonaws.comâ >> 'Date'->'Thu, 17 Jun 2021 16:08:44 GMTâ >> 'Authorization'->'AWS AKIAIP2xxxxxxx:DqPPbDyyyyyâ) >> >> From the spec (which Iâm no expert on) - the get command looks right (the other params are optional) and the Bucket name is prefixed onto s3.amazonaws.com in the header. >> >> But is seems that I am getting a 200 - 403 response : >> "a ZnResponse(200 OK application/xml 1899B)" >> HTTP/1.1 403 Forbidden >> >> (Aside - the code should possibly handle this better - the spec seems to say that response content needs to be interpreted - but thatâs something I can look to help with when we know the actual issue). >> >> But Iâm a bit mystified why the previous: client keysIn: 'mtt-dataâ >> Invocation would work, but this seemingly extra step would be forbidden. My IAM role says it has full S3 access, so Iâm not sure if this is still that we havenât got the right fix or if its something environmental. Iâm not sure what to do - if you have another idea, Iâm happy to be a guniea pig to try it. >> >> Tim >> >> >> >>> On 17 Jun 2021, at 09:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hi Tim, >>> >>> It has been a very long time since I last looked at this code. It is a good thing that at least part of it still works. >>> >>> Just by reading the code and reasoning about your problem, I would suggest you try the following change: >>> >>> In ZnAWSS3RequestSignatureTool>>#canonicalStringFor: replace the last >>> >>> request uri pathPrintString >>> >>> by >>> >>> request uri pathQueryFragmentPrintString >>> >>> I did not try this myself, so it might not solve your issue. >>> >>> Please let me know if this works. >>> >>> Sven >>> >>>> On 17 Jun 2021, at 03:05, Tim Mackinnon <tim(a)testit.works> wrote: >>>> >>>> Hi everyone - Iâm wondering if someone knows the trick to listing object versions in AWS S3? >>>> >>>> I was previously using a non-Zn library (there are a few around - but they are quite old and Iâm not sure how much they are maintained) - however I hadnât realised that Zn actually supports S3 until I read it a bit carefully and realised that I needed to load an extra AWS group. >>>> >>>> So - I have a bucket that is versioned, and I wanted to read the versions of an object - and hopefully be able to read the contents of an older version. >>>> >>>> I can list the contents of a bucket, but when I try to read the versions of an object - I get a forbidden error - which when I dig deeper gives a stranger explanation? >>>> >>>> I have confirmed that the AWS CLI is able to list versions in that bucket - so Iâm a bit confused what the issue might be? >>>> >>>> (client := ZnAWSS3Client new) >>>> accessKeyId: 'xxx'; >>>> secretAccessKey: âyyyyy'; >>>> checkIntegrity: true. >>>> >>>> client buckets. âWorks" >>>> client keysIn: 'mtt-dataâ. âWorks" >>>> >>>> client at: 'mtt-data' -> 'sample.txtâ. âWorks" >>>> >>>> client keysIn: 'mtt-data' query: (Dictionary with: 'versions'->nil). âGives an error?â >>>> >>>> >>>> HTTP/1.1 403 Forbidden >>>> >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>xxx</AWSAccessKeyId><StringToSign>GET >>>> >>>> >>>> Has anyone tried doing this - Iâm sure there is something really simple that I am missing? >>>> >>>> Tim

June 17, 2021

Re: Zn and AWS - retrieving object versions?

by Tim Mackinnon

Hi Sven - actually I think its not just the versions thing - I canât get filtering to work on the standard query either: client keysIn: âmtt-data' query: (Dictionary with: 'prefix'->'sampleâ). Given a normal query works, there must be something extra that is being missed/added that impacts things. Tim > On 17 Jun 2021, at 17:19, Tim Mackinnon <tim(a)testit.works> wrote: > > Hi Sven - thanks for taking a quick look, this change did stop me getting an immediate error - but I seem to get no results back. > > Digging into it a bit more - I seem to be issuing the GET request as indicated by https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html > > e.g. > > GET /?versions HTTP/1.1 > > Headers: > a ZnMultiValueDictionary('User-Agent'->'Zinc HTTP Components 1.0 (Pharo/8.0)â > 'Accept'->'*/*' 'Host'->'mtt-data.s3.amazonaws.comâ > 'Date'->'Thu, 17 Jun 2021 16:08:44 GMTâ > 'Authorization'->'AWS AKIAIP2xxxxxxx:DqPPbDyyyyyâ) > > From the spec (which Iâm no expert on) - the get command looks right (the other params are optional) and the Bucket name is prefixed onto s3.amazonaws.com in the header. > > But is seems that I am getting a 200 - 403 response : > "a ZnResponse(200 OK application/xml 1899B)" > HTTP/1.1 403 Forbidden > > (Aside - the code should possibly handle this better - the spec seems to say that response content needs to be interpreted - but thatâs something I can look to help with when we know the actual issue). > > But Iâm a bit mystified why the previous: client keysIn: 'mtt-dataâ > Invocation would work, but this seemingly extra step would be forbidden. My IAM role says it has full S3 access, so Iâm not sure if this is still that we havenât got the right fix or if its something environmental. Iâm not sure what to do - if you have another idea, Iâm happy to be a guniea pig to try it. > > Tim > > > >> On 17 Jun 2021, at 09:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Hi Tim, >> >> It has been a very long time since I last looked at this code. It is a good thing that at least part of it still works. >> >> Just by reading the code and reasoning about your problem, I would suggest you try the following change: >> >> In ZnAWSS3RequestSignatureTool>>#canonicalStringFor: replace the last >> >> request uri pathPrintString >> >> by >> >> request uri pathQueryFragmentPrintString >> >> I did not try this myself, so it might not solve your issue. >> >> Please let me know if this works. >> >> Sven >> >>> On 17 Jun 2021, at 03:05, Tim Mackinnon <tim(a)testit.works> wrote: >>> >>> Hi everyone - Iâm wondering if someone knows the trick to listing object versions in AWS S3? >>> >>> I was previously using a non-Zn library (there are a few around - but they are quite old and Iâm not sure how much they are maintained) - however I hadnât realised that Zn actually supports S3 until I read it a bit carefully and realised that I needed to load an extra AWS group. >>> >>> So - I have a bucket that is versioned, and I wanted to read the versions of an object - and hopefully be able to read the contents of an older version. >>> >>> I can list the contents of a bucket, but when I try to read the versions of an object - I get a forbidden error - which when I dig deeper gives a stranger explanation? >>> >>> I have confirmed that the AWS CLI is able to list versions in that bucket - so Iâm a bit confused what the issue might be? >>> >>> (client := ZnAWSS3Client new) >>> accessKeyId: 'xxx'; >>> secretAccessKey: âyyyyy'; >>> checkIntegrity: true. >>> >>> client buckets. âWorks" >>> client keysIn: 'mtt-dataâ. âWorks" >>> >>> client at: 'mtt-data' -> 'sample.txtâ. âWorks" >>> >>> client keysIn: 'mtt-data' query: (Dictionary with: 'versions'->nil). âGives an error?â >>> >>> >>> HTTP/1.1 403 Forbidden >>> >>> <?xml version="1.0" encoding="UTF-8"?> >>> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>xxx</AWSAccessKeyId><StringToSign>GET >>> >>> >>> Has anyone tried doing this - Iâm sure there is something really simple that I am missing? >>> >>> Tim

June 17, 2021

Re: Zn and AWS - retrieving object versions?

by Tim Mackinnon

Hi Sven - thanks for taking a quick look, this change did stop me getting an immediate error - but I seem to get no results back. Digging into it a bit more - I seem to be issuing the GET request as indicated by https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html e.g. GET /?versions HTTP/1.1 Headers: a ZnMultiValueDictionary('User-Agent'->'Zinc HTTP Components 1.0 (Pharo/8.0)â 'Accept'->'*/*' 'Host'->'mtt-data.s3.amazonaws.comâ 'Date'->'Thu, 17 Jun 2021 16:08:44 GMTâ 'Authorization'->'AWS AKIAIP2xxxxxxx:DqPPbDyyyyyâ) >From the spec (which Iâm no expert on) - the get command looks right (the other params are optional) and the Bucket name is prefixed onto s3.amazonaws.com in the header. But is seems that I am getting a 200 - 403 response : "a ZnResponse(200 OK application/xml 1899B)" HTTP/1.1 403 Forbidden (Aside - the code should possibly handle this better - the spec seems to say that response content needs to be interpreted - but thatâs something I can look to help with when we know the actual issue). But Iâm a bit mystified why the previous: client keysIn: 'mtt-dataâ Invocation would work, but this seemingly extra step would be forbidden. My IAM role says it has full S3 access, so Iâm not sure if this is still that we havenât got the right fix or if its something environmental. Iâm not sure what to do - if you have another idea, Iâm happy to be a guniea pig to try it. Tim > On 17 Jun 2021, at 09:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi Tim, > > It has been a very long time since I last looked at this code. It is a good thing that at least part of it still works. > > Just by reading the code and reasoning about your problem, I would suggest you try the following change: > > In ZnAWSS3RequestSignatureTool>>#canonicalStringFor: replace the last > > request uri pathPrintString > > by > > request uri pathQueryFragmentPrintString > > I did not try this myself, so it might not solve your issue. > > Please let me know if this works. > > Sven > >> On 17 Jun 2021, at 03:05, Tim Mackinnon <tim(a)testit.works> wrote: >> >> Hi everyone - Iâm wondering if someone knows the trick to listing object versions in AWS S3? >> >> I was previously using a non-Zn library (there are a few around - but they are quite old and Iâm not sure how much they are maintained) - however I hadnât realised that Zn actually supports S3 until I read it a bit carefully and realised that I needed to load an extra AWS group. >> >> So - I have a bucket that is versioned, and I wanted to read the versions of an object - and hopefully be able to read the contents of an older version. >> >> I can list the contents of a bucket, but when I try to read the versions of an object - I get a forbidden error - which when I dig deeper gives a stranger explanation? >> >> I have confirmed that the AWS CLI is able to list versions in that bucket - so Iâm a bit confused what the issue might be? >> >> (client := ZnAWSS3Client new) >> accessKeyId: 'xxx'; >> secretAccessKey: âyyyyy'; >> checkIntegrity: true. >> >> client buckets. âWorks" >> client keysIn: 'mtt-dataâ. âWorks" >> >> client at: 'mtt-data' -> 'sample.txtâ. âWorks" >> >> client keysIn: 'mtt-data' query: (Dictionary with: 'versions'->nil). âGives an error?â >> >> >> HTTP/1.1 403 Forbidden >> >> <?xml version="1.0" encoding="UTF-8"?> >> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>xxx</AWSAccessKeyId><StringToSign>GET >> >> >> Has anyone tried doing this - Iâm sure there is something really simple that I am missing? >> >> Tim

June 17, 2021

Re: Spec Gtk Applications - Implementing File open/save/(print?) dialogs

by kmo

Wow! Thanks so much Esteban for taking all the time and trouble to provide this. I'll have a play with it. Ken -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

June 17, 2021