May 2019 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] FloatArray

by Jimmie Houchin

I wasn't worried about how to do sliding windows. My problem is that using LapackDGEMatrix in my example was 18x slower than FloatArray, which is slower than Numpy. It isn't what I was expecting. What I didn't know is if I was doing something wrong to cause such a tremendous slow down. Python and Numpy is not my favorite. But it isn't uncomfortable. So I gave up and went back to Numpy. Thanks. On 5/20/19 5:17 PM, Nicolas Cellier wrote: > Hi Jimmie, > effectively I did not subsribe... > Having efficient methods for sliding window average is possible, here > is how I would do it: > > "Create a vector with 100,000 rows filles with random values (uniform > distrubution in [0,1]" > v := LapackDGEMatrix randUniform: #(100000 1). > > "extract values from rank 10001 to 20000" > w1 := v atIntervalFrom: 10001 to: 20000 by: 1. > > "create a left multiplier matrix for performing average of w1" > a := LapackDGEMatrix nrow: 1 ncol: w1 nrow withAll: 1.0 / w1 size. > > "get the average (this is a 1x1 matrix from which we take first element)" > avg1 := (a * w1) at: 1. > > [ "select another slice of same size" > w2 := v atIntervalFrom: 15001 to: 25000 by: 1. > > "get the average (we can recycle a)" > avg2 := (a * w2) at: 1 ] bench. > > This gives: > Â '16,500 per second. 60.7 microseconds per run.' > versus: > [w2 sum / w2 size] bench. > Â '1,100 per second. 908 microseconds per run.' > > For max and min, it's harder. Lapack/Blas only provide max of absolute > value as primitive: > [w2 absMax] bench. > Â '19,400 per second. 51.5 microseconds per run.' > > Everything else will be slower, unless we write new primitives in C > and connect them... > [w2 maxOf: [:each | each]] bench. > Â '984 per second. 1.02 milliseconds per run.' > > LeÂ dim. 19 mai 2019 Ã Â 14:58, Jimmie <jlhouchin(a)gmail.com > <mailto:jlhouchin@gmail.com>> a Ã©critÂ : > > On 5/16/19 1:26 PM, Nicolas Cellier wrote:> Any feedback on this? > Â > Did someone tried to use Smallapack in Pharo? > Â > Jimmie? > Â > > > I am going to guess that you are not on pharo-users. My bad. > I posted this in pharo-users as I it wasn't Pharo development > question. > > I probably should have posted here or emailed you directly. > > All I really need is good performance with a simple array of > floats. No > matrix math. Nothing complicated. Moving Averages over a slice of the > array. A variety of different averages, weighted, etc. Max/min of the > array. But just a single simple array. > > Any help greatly appreciated. > > Thanks. > > > On 4/28/19 8:32 PM, Jimmie Houchin wrote: > Hello, > > I have installed Smallapack into Pharo 7.0.3. Thanks Nicholas. > > I am very unsure on my use of Smallapack. I am not a mathematician or > scientist. However the only part of Smallapack I am trying to use > at the > moment is something that wouldÂ be 64bit and compare to FloatArray so > that I can do some simple accessing, slicing, sum, and average on > the array. > > Here is some sample code I wrote just to play in a playground. > > I have an ExternalDoubleArray, LapackDGEMatrix, and a FloatArray > samples. The ones not in use are commented out for any run. > > fp is a download from > http://ratedata.gaincapital.com/2018/12%20December/EUR_USD_Week1.zip > and unzipped to a directory. > > fp := '/home/jimmie/data/EUR_USD_Week1.csv' > index := 0. > pricesSum := 0. > asum := 0. > ttr := [ > Â Â Â lines := fp asFileReference contents lines allButFirst. > Â Â Â a := ExternalDoubleArray new: lines size. > Â Â Â "la := LapackDGEMatrix allocateNrow: lines size ncol: 1. > Â Â Â a := la columnAt: 1." > Â Â Â "a := FloatArray new: lines size." > Â Â Â lines do: [ :line || parts price | > Â Â Â Â Â parts := ',' split: line. > Â Â Â Â Â index := index + 1. > Â Â Â Â Â price := Float readFrom: (parts last). > Â Â Â Â Â a at: index put: price. > Â Â Â Â Â pricesSum := pricesSum + price. > Â Â Â Â Â (index rem: 100) = 0 ifTrue: [ > Â Â Â Â Â Â Â asum := a sum. > Â Â Â ]]] timeToRun. > { index. pricesSum. asum. ttr }. > Â "ExternalDoubleArray an Array(337588 383662.5627699992 > 383562.2956199993 0:00:01:59.885)" > Â "FloatArrayÂ an Array(337588 383662.5627699992 383562.2954441309 > 0:00:00:06.555)" > > FloatArray is not the precision I need. But it is over 18x faster. > > I am afraid I must be doing something badly wrong. Python/Numpy is > over > 4x faster than FloatArray for the above. > > If I am using Smallapack incorrectly please help. > > Any help greatly appreciated. > > Thanks. > >

May 20, 2019

Re: [Pharo-dev] FloatArray

by Nicolas Cellier

Hi Jimmie, effectively I did not subsribe... Having efficient methods for sliding window average is possible, here is how I would do it: "Create a vector with 100,000 rows filles with random values (uniform distrubution in [0,1]" v := LapackDGEMatrix randUniform: #(100000 1). "extract values from rank 10001 to 20000" w1 := v atIntervalFrom: 10001 to: 20000 by: 1. "create a left multiplier matrix for performing average of w1" a := LapackDGEMatrix nrow: 1 ncol: w1 nrow withAll: 1.0 / w1 size. "get the average (this is a 1x1 matrix from which we take first element)" avg1 := (a * w1) at: 1. [ "select another slice of same size" w2 := v atIntervalFrom: 15001 to: 25000 by: 1. "get the average (we can recycle a)" avg2 := (a * w2) at: 1 ] bench. This gives: '16,500 per second. 60.7 microseconds per run.' versus: [w2 sum / w2 size] bench. '1,100 per second. 908 microseconds per run.' For max and min, it's harder. Lapack/Blas only provide max of absolute value as primitive: [w2 absMax] bench. '19,400 per second. 51.5 microseconds per run.' Everything else will be slower, unless we write new primitives in C and connect them... [w2 maxOf: [:each | each]] bench. '984 per second. 1.02 milliseconds per run.' Le dim. 19 mai 2019 Ã 14:58, Jimmie <jlhouchin(a)gmail.com> a Ã©crit : > On 5/16/19 1:26 PM, Nicolas Cellier wrote:> Any feedback on this? > > Did someone tried to use Smallapack in Pharo? > > Jimmie? > > > > I am going to guess that you are not on pharo-users. My bad. > I posted this in pharo-users as I it wasn't Pharo development question. > > I probably should have posted here or emailed you directly. > > All I really need is good performance with a simple array of floats. No > matrix math. Nothing complicated. Moving Averages over a slice of the > array. A variety of different averages, weighted, etc. Max/min of the > array. But just a single simple array. > > Any help greatly appreciated. > > Thanks. > > > On 4/28/19 8:32 PM, Jimmie Houchin wrote: > Hello, > > I have installed Smallapack into Pharo 7.0.3. Thanks Nicholas. > > I am very unsure on my use of Smallapack. I am not a mathematician or > scientist. However the only part of Smallapack I am trying to use at the > moment is something that would be 64bit and compare to FloatArray so > that I can do some simple accessing, slicing, sum, and average on the > array. > > Here is some sample code I wrote just to play in a playground. > > I have an ExternalDoubleArray, LapackDGEMatrix, and a FloatArray > samples. The ones not in use are commented out for any run. > > fp is a download from > http://ratedata.gaincapital.com/2018/12%20December/EUR_USD_Week1.zip > and unzipped to a directory. > > fp := '/home/jimmie/data/EUR_USD_Week1.csv' > index := 0. > pricesSum := 0. > asum := 0. > ttr := [ > lines := fp asFileReference contents lines allButFirst. > a := ExternalDoubleArray new: lines size. > "la := LapackDGEMatrix allocateNrow: lines size ncol: 1. > a := la columnAt: 1." > "a := FloatArray new: lines size." > lines do: [ :line || parts price | > parts := ',' split: line. > index := index + 1. > price := Float readFrom: (parts last). > a at: index put: price. > pricesSum := pricesSum + price. > (index rem: 100) = 0 ifTrue: [ > asum := a sum. > ]]] timeToRun. > { index. pricesSum. asum. ttr }. > "ExternalDoubleArray an Array(337588 383662.5627699992 > 383562.2956199993 0:00:01:59.885)" > "FloatArray an Array(337588 383662.5627699992 383562.2954441309 > 0:00:00:06.555)" > > FloatArray is not the precision I need. But it is over 18x faster. > > I am afraid I must be doing something badly wrong. Python/Numpy is over > 4x faster than FloatArray for the above. > > If I am using Smallapack incorrectly please help. > > Any help greatly appreciated. > > Thanks. > > >

May 20, 2019

[Pharo 8.0] Build #280: 3369-Context-should-not-depend-on-Halt

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #280 was: FAILURE. The Pull Request #3372 was integrated: "3369-Context-should-not-depend-on-Halt" Pull request url: https://github.com/pharo-project/pharo/pull/3372 Issue Url: https://github.com/pharo-project/pharo/issues/3369 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

May 20, 2019

Re: [Pharo-dev] [Ann] Lifeware, Schmidt Pro contracts for the consortium

by Torsten Bergmann

May 20, 2019

Re: [Pharo-dev] [Ann] Lifeware, Schmidt Pro contracts for the consortium

by Esteban Maringolo

This is really good news! Congratulations! Esteban A. Maringolo On Mon, May 20, 2019 at 3:04 AM ducasse <stepharo(a)netcourrier.com> wrote: > The Pharo consortium is very excited and super happy to bring your > attention to the following announce > that was presented during Pharodays 2019: > https://fr.slideshare.net/pharoproject/ > > The consortium got two contracts to support financially one year of > engineer to improve Pharo. > The companies Lifeware and Schmidt Pro fund work on Pharo to improve > Pharo. The total amount > of the two contracts is around 190 K Euros. > In addition the RMOD team got some resources from Inria. > The net result is that in 2019 the consortium will have 3.5 engineers > working full time on Pharo. > > - Esteban Lorenzano > - Pablo Tesone > - Cyril Ferlicot > - Guillermo Polito > > It will boost Pharo. Note that the issues raised by Schmidt Pro and > Lifeware > and their impacts on the roadmap of Pharo 8 and Pharo 9 are presented in > https://fr.slideshare.net/pharoproject/. > > What is key to notice is that the consortium is working because many > contributing companies are sharing ressources. > This is built a strong soil to grow Pharo and business around. > > On the behalf of Inria, the consortium and the community we would like to > thank > Lifeware and SchmidtPro for their strong support. > > Stef on the behalf of the Pharo consortium > >

May 20, 2019

Re: [Pharo-dev] [Ann] Lifeware, Schmidt Pro contracts for the consortium

by Sven Van Caekenberghe

Indeed, thank you for the support ! > On 20 May 2019, at 10:58, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Great news! > > Cheers, > Doru > >> On May 20, 2019, at 8:03 AM, ducasse <stepharo(a)netcourrier.com> wrote: >> >> The Pharo consortium is very excited and super happy to bring your attention to the following announce >> that was presented during Pharodays 2019: https://fr.slideshare.net/pharoproject/ >> >> The consortium got two contracts to support financially one year of engineer to improve Pharo. >> The companies Lifeware and Schmidt Pro fund work on Pharo to improve Pharo. The total amount >> of the two contracts is around 190 K Euros. >> In addition the RMOD team got some resources from Inria. >> The net result is that in 2019 the consortium will have 3.5 engineers working full time on Pharo. >> >> - Esteban Lorenzano >> - Pablo Tesone >> - Cyril Ferlicot >> - Guillermo Polito >> >> It will boost Pharo. Note that the issues raised by Schmidt Pro and Lifeware >> and their impacts on the roadmap of Pharo 8 and Pharo 9 are presented in https://fr.slideshare.net/pharoproject/. >> >> What is key to notice is that the consortium is working because many contributing companies are sharing ressources. >> This is built a strong soil to grow Pharo and business around. >> >> On the behalf of Inria, the consortium and the community we would like to thank >> Lifeware and SchmidtPro for their strong support. >> >> Stef on the behalf of the Pharo consortium >> > > -- > www.feenk.com > > âProgramming is executable philosophy." > >

May 20, 2019

Re: [Pharo-dev] [Ann] Lifeware, Schmidt Pro contracts for the consortium

by Tudor Girba

Great news! Cheers, Doru > On May 20, 2019, at 8:03 AM, ducasse <stepharo(a)netcourrier.com> wrote: > > The Pharo consortium is very excited and super happy to bring your attention to the following announce > that was presented during Pharodays 2019: https://fr.slideshare.net/pharoproject/ > > The consortium got two contracts to support financially one year of engineer to improve Pharo. > The companies Lifeware and Schmidt Pro fund work on Pharo to improve Pharo. The total amount > of the two contracts is around 190 K Euros. > In addition the RMOD team got some resources from Inria. > The net result is that in 2019 the consortium will have 3.5 engineers working full time on Pharo. > > - Esteban Lorenzano > - Pablo Tesone > - Cyril Ferlicot > - Guillermo Polito > > It will boost Pharo. Note that the issues raised by Schmidt Pro and Lifeware > and their impacts on the roadmap of Pharo 8 and Pharo 9 are presented in https://fr.slideshare.net/pharoproject/. > > What is key to notice is that the consortium is working because many contributing companies are sharing ressources. > This is built a strong soil to grow Pharo and business around. > > On the behalf of Inria, the consortium and the community we would like to thank > Lifeware and SchmidtPro for their strong support. > > Stef on the behalf of the Pharo consortium > -- www.feenk.com âProgramming is executable philosophy."

May 20, 2019

[Ann] Lifeware, Schmidt Pro contracts for the consortium

by ducasse

The Pharo consortium is very excited and super happy to bring your attention to the following announce that was presented during Pharodays 2019: https://fr.slideshare.net/pharoproject/ <https://fr.slideshare.net/pharoproject/> The consortium got two contracts to support financially one year of engineer to improve Pharo. The companies Lifeware and Schmidt Pro fund work on Pharo to improve Pharo. The total amount of the two contracts is around 190 K Euros. In addition the RMOD team got some resources from Inria. The net result is that in 2019 the consortium will have 3.5 engineers working full time on Pharo. - Esteban Lorenzano - Pablo Tesone - Cyril Ferlicot - Guillermo Polito It will boost Pharo. Note that the issues raised by Schmidt Pro and Lifeware and their impacts on the roadmap of Pharo 8 and Pharo 9 are presented in https://fr.slideshare.net/pharoproject/ <https://fr.slideshare.net/pharoproject/>. What is key to notice is that the consortium is working because many contributing companies are sharing ressources. This is built a strong soil to grow Pharo and business around. On the behalf of Inria, the consortium and the community we would like to thank Lifeware and SchmidtPro for their strong support. Stef on the behalf of the Pharo consortium

May 20, 2019

Re: [Pharo-dev] FloatArray

by Jimmie

On 5/16/19 1:26 PM, Nicolas Cellier wrote:> Any feedback on this? > Did someone tried to use Smallapack in Pharo? > Jimmie? > I am going to guess that you are not on pharo-users. My bad. I posted this in pharo-users as I it wasn't Pharo development question. I probably should have posted here or emailed you directly. All I really need is good performance with a simple array of floats. No matrix math. Nothing complicated. Moving Averages over a slice of the array. A variety of different averages, weighted, etc. Max/min of the array. But just a single simple array. Any help greatly appreciated. Thanks. On 4/28/19 8:32 PM, Jimmie Houchin wrote: Hello, I have installed Smallapack into Pharo 7.0.3. Thanks Nicholas. I am very unsure on my use of Smallapack. I am not a mathematician or scientist. However the only part of Smallapack I am trying to use at the moment is something that would be 64bit and compare to FloatArray so that I can do some simple accessing, slicing, sum, and average on the array. Here is some sample code I wrote just to play in a playground. I have an ExternalDoubleArray, LapackDGEMatrix, and a FloatArray samples. The ones not in use are commented out for any run. fp is a download from http://ratedata.gaincapital.com/2018/12%20December/EUR_USD_Week1.zip and unzipped to a directory. fp := '/home/jimmie/data/EUR_USD_Week1.csv' index := 0. pricesSum := 0. asum := 0. ttr := [ lines := fp asFileReference contents lines allButFirst. a := ExternalDoubleArray new: lines size. "la := LapackDGEMatrix allocateNrow: lines size ncol: 1. a := la columnAt: 1." "a := FloatArray new: lines size." lines do: [ :line || parts price | parts := ',' split: line. index := index + 1. price := Float readFrom: (parts last). a at: index put: price. pricesSum := pricesSum + price. (index rem: 100) = 0 ifTrue: [ asum := a sum. ]]] timeToRun. { index. pricesSum. asum. ttr }. "ExternalDoubleArray an Array(337588 383662.5627699992 383562.2956199993 0:00:01:59.885)" "FloatArray an Array(337588 383662.5627699992 383562.2954441309 0:00:00:06.555)" FloatArray is not the precision I need. But it is over 18x faster. I am afraid I must be doing something badly wrong. Python/Numpy is over 4x faster than FloatArray for the above. If I am using Smallapack incorrectly please help. Any help greatly appreciated. Thanks.

May 19, 2019

Is it me or P8 feels snappier than P7?

by ducasse

Hi Is it me or P8 feels snappier than P7? Stef

May 18, 2019