- Pharo-users - lists.pharo.org

Re: [Pharo-users] Porting Transducers to Pharo

by Alexandre Bergel

I second Sven. This is very exciting! Let us know when you have something ready to be tested. Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > On May 31, 2017, at 9:00 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > >> On 31 May 2017, at 14:23, Steffen MÃ¤rcker <merkste(a)web.de> wrote: >> >> Hi, >> >> I am the developer of the library 'Transducers' for VisualWorks. It was formerly known as 'Reducers', but this name was a poor choice. I'd like to port it to Pharo, if there is any interest on your side. I hope to learn more about Pharo in this process, since I am mainly a VW guy. And most likely, I will come up with a bunch of questions. :-) >> >> Meanwhile, I'll cross-post the introduction from VWnc below. I'd be very happy to hear your optinions, questions and I hope we can start a fruitful discussion - even if there is not Pharo port yet. >> >> Best, Steffen > > Hi Steffen, > > Looks like very interesting stuff. Would make an nice library/framework for Pharo. > > Sven > >> Transducers are building blocks that encapsulate how to process elements >> of a data sequence independently of the underlying input and output source. >> >> >> >> # Overview >> >> ## Encapsulate >> Implementations of enumeration methods, such as #collect:, have the logic >> how to process a single element in common. >> However, that logic is reimplemented each and every time. Transducers make >> it explicit and facilitate re-use and coherent behavior. >> For example: >> - #collect: requires mapping: (aBlock1 map) >> - #select: requires filtering: (aBlock2 filter) >> >> >> ## Compose >> In practice, algorithms often require multiple processing steps, e.g., >> mapping only a filtered set of elements. >> Transducers are inherently composable, and hereby, allow to make the >> combination of steps explicit. >> Since transducers do not build intermediate collections, their composition >> is memory-efficient. >> For example: >> - (aBlock1 filter) * (aBlock2 map) "(1.) filter and (2.) map elements" >> >> >> ## Re-Use >> Transducers are decoupled from the input and output sources, and hence, >> they can be reused in different contexts. >> For example: >> - enumeration of collections >> - processing of streams >> - communicating via channels >> >> >> >> # Usage by Example >> >> We build a coin flipping experiment and count the occurrence of heads and >> tails. >> >> First, we associate random numbers with the sides of a coin. >> >> scale := [:x | (x * 2 + 1) floor] map. >> sides := #(heads tails) replace. >> >> Scale is a transducer that maps numbers x between 0 and 1 to 1 and 2. >> Sides is a transducer that replaces the numbers with heads an tails by >> lookup in an array. >> Next, we choose a number of samples. >> >> count := 1000 take. >> >> Count is a transducer that takes 1000 elements from a source. >> We keep track of the occurrences of heads an tails using a bag. >> >> collect := [:bag :c | bag add: c; yourself]. >> >> Collect is binary block (reducing function) that collects events in a bag. >> We assemble the experiment by transforming the block using the transducers. >> >> experiment := (scale * sides * count) transform: collect. >> >> From left to right we see the steps involved: scale, sides, count and >> collect. >> Transforming assembles these steps into a binary block (reducing function) >> we can use to run the experiment. >> >> samples := Random new >> reduce: experiment >> init: Bag new. >> >> Here, we use #reduce:init:, which is mostly similar to #inject:into:. >> To execute a transformation and a reduction together, we can use >> #transduce:reduce:init:. >> >> samples := Random new >> transduce: scale * sides * count >> reduce: collect >> init: Bag new. >> >> We can also express the experiment as data-flow using #<~. >> This enables us to build objects that can be re-used in other experiments. >> >> coin := sides <~ scale <~ Random new. >> flip := Bag <~ count. >> >> Coin is an eduction, i.e., it binds transducers to a source and >> understands #reduce:init: among others. >> Flip is a transformed reduction, i.e., it binds transducers to a reducing >> function and an initial value. >> By sending #<~, we draw further samples from flipping the coin. >> >> samples := flip <~ coin. >> >> This yields a new Bag with another 1000 samples. >> >> >> >> # Basic Concepts >> >> ## Reducing Functions >> >> A reducing function represents a single step in processing a data sequence. >> It takes an accumulated result and a value, and returns a new accumulated >> result. >> For example: >> >> collect := [:col :e | col add: e; yourself]. >> sum := #+. >> >> A reducing function can also be ternary, i.e., it takes an accumulated >> result, a key and a value. >> For example: >> >> collect := [:dic :k :v | dict at: k put: v; yourself]. >> >> Reducing functions may be equipped with an optional completing action. >> After finishing processing, it is invoked exactly once, e.g., to free >> resources. >> >> stream := [:str :e | str nextPut: each; yourself] completing: #close. >> absSum := #+ completing: #abs >> >> A reducing function can end processing early by signaling Reduced with a >> result. >> This mechanism also enables the treatment of infinite sources. >> >> nonNil := [:res :e | e ifNil: [Reduced signalWith: res] ifFalse: [res]]. >> >> The primary approach to process a data sequence is the reducing protocol >> with the messages #reduce:init: and #transduce:reduce:init: if transducers >> are involved. >> The behavior is similar to #inject:into: but in addition it takes care of: >> - handling binary and ternary reducing functions, >> - invoking the completing action after finishing, and >> - stopping the reduction if Reduced is signaled. >> The message #transduce:reduce:init: just combines the transformation and >> the reducing step. >> >> However, as reducing functions are step-wise in nature, an application may >> choose other means to process its data. >> >> >> ## Reducibles >> >> A data source is called reducible if it implements the reducing protocol. >> Default implementations are provided for collections and streams. >> Additionally, blocks without an argument are reducible, too. >> This allows to adapt to custom data sources without additional effort. >> For example: >> >> "XStreams adaptor" >> xstream := filename reading. >> reducible := [[xstream get] on: Incomplete do: [Reduced signal]]. >> >> "natural numbers" >> n := 0. >> reducible := [n := n+1]. >> >> >> ## Transducers >> >> A transducer is an object that transforms a reducing function into another. >> Transducers encapsulate common steps in processing data sequences, such as >> map, filter, concatenate, and flatten. >> A transducer transforms a reducing function into another via #transform: >> in order to add those steps. >> They can be composed using #* which yields a new transducer that does both >> transformations. >> Most transducers require an argument, typically blocks, symbols or numbers: >> >> square := Map function: #squared. >> take := Take number: 1000. >> >> To facilitate compact notation, the argument types implement corresponding >> methods: >> >> squareAndTake := #squared map * 1000 take. >> >> Transducers requiring no argument are singletons and can be accessed by >> their class name. >> >> flattenAndDedupe := Flatten * Dedupe. >> >> >> >> # Advanced Concepts >> >> ## Data flows >> >> Processing a sequence of data can often be regarded as a data flow. >> The operator #<~ allows define a flow from a data source through >> processing steps to a drain. >> For example: >> >> squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). >> fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. >> >> In both examples #<~ is only used to set up the data flow using reducing >> functions and transducers. >> In contrast to streams, transducers are completely independent from input >> and output sources. >> Hence, we have a clear separation of reading data, writing data and >> processing elements. >> - Sources know how to iterate over data with a reducing function, e.g., >> via #reduce:init:. >> - Drains know how to collect data using a reducing function. >> - Transducers know how to process single elements. >> >> >> ## Reductions >> >> A reduction binds an initial value or a block yielding an initial value to >> a reducing function. >> The idea is to define a ready-to-use process that can be applied in >> different contexts. >> Reducibles handle reductions via #reduce: and #transduce:reduce: >> For example: >> >> sum := #+ init: 0. >> sum1 := #(1 1 1) reduce: sum. >> sum2 := (1 to: 1000) transduce: #odd filter reduce: sum. >> >> asSet := [:set :e | set add: e; yourself] initializer: [Set new]. >> set1 := #(1 1 1) reduce: asSet. >> set2 := #(1 to: 1000) transduce: #odd filter reduce: asSet. >> >> By combining a transducer with a reduction, a process can be further >> modified. >> >> sumOdds := sum <~ #odd filter >> setOdds := asSet <~ #odd filter >> >> >> ## Eductions >> >> An eduction combines a reducible data sources with a transducer. >> The idea is to define a transformed (virtual) data source that needs not >> to be stored in memory. >> >> odds1 := #odd filter <~ #(1 2 3) readStream. >> odds2 := #odd filter <~ (1 to 1000). >> >> Depending on the underlying source, eductions can be processed once >> (streams, e.g., odds1) or multiple times (collections, e.g., odds2). >> Since no intermediate data is stored, transducers actions are lazy, i.e., >> they are invoked each time the eduction is processed. >> >> >> >> # Origins >> >> Transducers is based on the same-named Clojure library and its ideas. >> Please see: >> http://clojure.org/transducers >> > >

May 31, 2017

[ANN] Zeroconf stable will be moved tomorrow to 6.0 (as part of the release process)

by Esteban Lorenzano

Hi, We will move stable to 6.0 and alpha to 7.0. So if you depend on something that can be affected by this, is the moment to update your scripts. cheers, Esteban

May 31, 2017

Re: [Pharo-users] Porting Transducers to Pharo

by Damien Pollet

As you know I experimented with that a while ago. My code is at http://smalltalkhub.com/#!/~cdlm/Experiments/source On 31 May 2017 at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > On 31 May 2017, at 14:23, Steffen MÃ¤rcker <merkste(a)web.de> wrote: > > > > Hi, > > > > I am the developer of the library 'Transducers' for VisualWorks. It was > formerly known as 'Reducers', but this name was a poor choice. I'd like to > port it to Pharo, if there is any interest on your side. I hope to learn > more about Pharo in this process, since I am mainly a VW guy. And most > likely, I will come up with a bunch of questions. :-) > > > > Meanwhile, I'll cross-post the introduction from VWnc below. I'd be very > happy to hear your optinions, questions and I hope we can start a fruitful > discussion - even if there is not Pharo port yet. > > > > Best, Steffen > > Hi Steffen, > > Looks like very interesting stuff. Would make an nice library/framework > for Pharo. > > Sven > > > Transducers are building blocks that encapsulate how to process elements > > of a data sequence independently of the underlying input and output > source. > > > > > > > > # Overview > > > > ## Encapsulate > > Implementations of enumeration methods, such as #collect:, have the logic > > how to process a single element in common. > > However, that logic is reimplemented each and every time. Transducers > make > > it explicit and facilitate re-use and coherent behavior. > > For example: > > - #collect: requires mapping: (aBlock1 map) > > - #select: requires filtering: (aBlock2 filter) > > > > > > ## Compose > > In practice, algorithms often require multiple processing steps, e.g., > > mapping only a filtered set of elements. > > Transducers are inherently composable, and hereby, allow to make the > > combination of steps explicit. > > Since transducers do not build intermediate collections, their > composition > > is memory-efficient. > > For example: > > - (aBlock1 filter) * (aBlock2 map) "(1.) filter and (2.) map elements" > > > > > > ## Re-Use > > Transducers are decoupled from the input and output sources, and hence, > > they can be reused in different contexts. > > For example: > > - enumeration of collections > > - processing of streams > > - communicating via channels > > > > > > > > # Usage by Example > > > > We build a coin flipping experiment and count the occurrence of heads and > > tails. > > > > First, we associate random numbers with the sides of a coin. > > > > scale := [:x | (x * 2 + 1) floor] map. > > sides := #(heads tails) replace. > > > > Scale is a transducer that maps numbers x between 0 and 1 to 1 and 2. > > Sides is a transducer that replaces the numbers with heads an tails by > > lookup in an array. > > Next, we choose a number of samples. > > > > count := 1000 take. > > > > Count is a transducer that takes 1000 elements from a source. > > We keep track of the occurrences of heads an tails using a bag. > > > > collect := [:bag :c | bag add: c; yourself]. > > > > Collect is binary block (reducing function) that collects events in a > bag. > > We assemble the experiment by transforming the block using the > transducers. > > > > experiment := (scale * sides * count) transform: collect. > > > > From left to right we see the steps involved: scale, sides, count and > > collect. > > Transforming assembles these steps into a binary block (reducing > function) > > we can use to run the experiment. > > > > samples := Random new > > reduce: experiment > > init: Bag new. > > > > Here, we use #reduce:init:, which is mostly similar to #inject:into:. > > To execute a transformation and a reduction together, we can use > > #transduce:reduce:init:. > > > > samples := Random new > > transduce: scale * sides * count > > reduce: collect > > init: Bag new. > > > > We can also express the experiment as data-flow using #<~. > > This enables us to build objects that can be re-used in other > experiments. > > > > coin := sides <~ scale <~ Random new. > > flip := Bag <~ count. > > > > Coin is an eduction, i.e., it binds transducers to a source and > > understands #reduce:init: among others. > > Flip is a transformed reduction, i.e., it binds transducers to a reducing > > function and an initial value. > > By sending #<~, we draw further samples from flipping the coin. > > > > samples := flip <~ coin. > > > > This yields a new Bag with another 1000 samples. > > > > > > > > # Basic Concepts > > > > ## Reducing Functions > > > > A reducing function represents a single step in processing a data > sequence. > > It takes an accumulated result and a value, and returns a new accumulated > > result. > > For example: > > > > collect := [:col :e | col add: e; yourself]. > > sum := #+. > > > > A reducing function can also be ternary, i.e., it takes an accumulated > > result, a key and a value. > > For example: > > > > collect := [:dic :k :v | dict at: k put: v; yourself]. > > > > Reducing functions may be equipped with an optional completing action. > > After finishing processing, it is invoked exactly once, e.g., to free > > resources. > > > > stream := [:str :e | str nextPut: each; yourself] completing: #close. > > absSum := #+ completing: #abs > > > > A reducing function can end processing early by signaling Reduced with a > > result. > > This mechanism also enables the treatment of infinite sources. > > > > nonNil := [:res :e | e ifNil: [Reduced signalWith: res] ifFalse: > [res]]. > > > > The primary approach to process a data sequence is the reducing protocol > > with the messages #reduce:init: and #transduce:reduce:init: if > transducers > > are involved. > > The behavior is similar to #inject:into: but in addition it takes care > of: > > - handling binary and ternary reducing functions, > > - invoking the completing action after finishing, and > > - stopping the reduction if Reduced is signaled. > > The message #transduce:reduce:init: just combines the transformation and > > the reducing step. > > > > However, as reducing functions are step-wise in nature, an application > may > > choose other means to process its data. > > > > > > ## Reducibles > > > > A data source is called reducible if it implements the reducing protocol. > > Default implementations are provided for collections and streams. > > Additionally, blocks without an argument are reducible, too. > > This allows to adapt to custom data sources without additional effort. > > For example: > > > > "XStreams adaptor" > > xstream := filename reading. > > reducible := [[xstream get] on: Incomplete do: [Reduced signal]]. > > > > "natural numbers" > > n := 0. > > reducible := [n := n+1]. > > > > > > ## Transducers > > > > A transducer is an object that transforms a reducing function into > another. > > Transducers encapsulate common steps in processing data sequences, such > as > > map, filter, concatenate, and flatten. > > A transducer transforms a reducing function into another via #transform: > > in order to add those steps. > > They can be composed using #* which yields a new transducer that does > both > > transformations. > > Most transducers require an argument, typically blocks, symbols or > numbers: > > > > square := Map function: #squared. > > take := Take number: 1000. > > > > To facilitate compact notation, the argument types implement > corresponding > > methods: > > > > squareAndTake := #squared map * 1000 take. > > > > Transducers requiring no argument are singletons and can be accessed by > > their class name. > > > > flattenAndDedupe := Flatten * Dedupe. > > > > > > > > # Advanced Concepts > > > > ## Data flows > > > > Processing a sequence of data can often be regarded as a data flow. > > The operator #<~ allows define a flow from a data source through > > processing steps to a drain. > > For example: > > > > squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). > > fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. > > > > In both examples #<~ is only used to set up the data flow using reducing > > functions and transducers. > > In contrast to streams, transducers are completely independent from input > > and output sources. > > Hence, we have a clear separation of reading data, writing data and > > processing elements. > > - Sources know how to iterate over data with a reducing function, e.g., > > via #reduce:init:. > > - Drains know how to collect data using a reducing function. > > - Transducers know how to process single elements. > > > > > > ## Reductions > > > > A reduction binds an initial value or a block yielding an initial value > to > > a reducing function. > > The idea is to define a ready-to-use process that can be applied in > > different contexts. > > Reducibles handle reductions via #reduce: and #transduce:reduce: > > For example: > > > > sum := #+ init: 0. > > sum1 := #(1 1 1) reduce: sum. > > sum2 := (1 to: 1000) transduce: #odd filter reduce: sum. > > > > asSet := [:set :e | set add: e; yourself] initializer: [Set new]. > > set1 := #(1 1 1) reduce: asSet. > > set2 := #(1 to: 1000) transduce: #odd filter reduce: asSet. > > > > By combining a transducer with a reduction, a process can be further > > modified. > > > > sumOdds := sum <~ #odd filter > > setOdds := asSet <~ #odd filter > > > > > > ## Eductions > > > > An eduction combines a reducible data sources with a transducer. > > The idea is to define a transformed (virtual) data source that needs not > > to be stored in memory. > > > > odds1 := #odd filter <~ #(1 2 3) readStream. > > odds2 := #odd filter <~ (1 to 1000). > > > > Depending on the underlying source, eductions can be processed once > > (streams, e.g., odds1) or multiple times (collections, e.g., odds2). > > Since no intermediate data is stored, transducers actions are lazy, i.e., > > they are invoked each time the eduction is processed. > > > > > > > > # Origins > > > > Transducers is based on the same-named Clojure library and its ideas. > > Please see: > > http://clojure.org/transducers > > > > >

May 31, 2017

Re: [Pharo-users] Discord server ownership

by Ben Coman

Passing the baton at the right time is an important responsibility. Thanks for your efforts setting up the server in the first place. cheers -ben On Fri, May 26, 2017 at 3:29 AM, Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > Hey there, as you may or may not know I am the person to blame for > starting the Pharo server in Discord , out of curiousity at first then it > grew up to our main online chat. Unfortunately I have been using Pharo , > less and less lately because for better or worse the libraries I depend on > are based on Python and C++ (Blender and Unreal ) so I was intending to > transfer ownership of the Server for some time now as the fair thing to do. > > So as of this moment the owner of the discord server is no longer me but > Esteban Lorenzano , so he is the person to bother... sorry I mean address , > with any of your ideas , remarks etc from now on. > > It was a pleasure to see the community embracing Discord so fast and of > course I will be still around as an Admin trying to do my small part in > sharing my love for Pharo with newcomers. > > A sever owner essentially has full privileges . Technically I gave admins > every privilege but one which I did not understood what it was so I left > it be. While on the other hand channel admins (that was my idea for > allowing me to be more lazy) have full privileges only for the channel they > are admins of. > > I suspect that Esteban eventually will transfer ownership to the Pharo > board but since he is THE DUDE for anything Pharo I decided he was the > right person to get the crown , the glory and the money that come with > being a Discord Server owner. > > Goodbye and thank you for all the fish :) >

May 31, 2017

Re: [Pharo-users] Porting Transducers to Pharo

by Sven Van Caekenberghe

> On 31 May 2017, at 14:23, Steffen MÃ¤rcker <merkste(a)web.de> wrote: > > Hi, > > I am the developer of the library 'Transducers' for VisualWorks. It was formerly known as 'Reducers', but this name was a poor choice. I'd like to port it to Pharo, if there is any interest on your side. I hope to learn more about Pharo in this process, since I am mainly a VW guy. And most likely, I will come up with a bunch of questions. :-) > > Meanwhile, I'll cross-post the introduction from VWnc below. I'd be very happy to hear your optinions, questions and I hope we can start a fruitful discussion - even if there is not Pharo port yet. > > Best, Steffen Hi Steffen, Looks like very interesting stuff. Would make an nice library/framework for Pharo. Sven > Transducers are building blocks that encapsulate how to process elements > of a data sequence independently of the underlying input and output source. > > > > # Overview > > ## Encapsulate > Implementations of enumeration methods, such as #collect:, have the logic > how to process a single element in common. > However, that logic is reimplemented each and every time. Transducers make > it explicit and facilitate re-use and coherent behavior. > For example: > - #collect: requires mapping: (aBlock1 map) > - #select: requires filtering: (aBlock2 filter) > > > ## Compose > In practice, algorithms often require multiple processing steps, e.g., > mapping only a filtered set of elements. > Transducers are inherently composable, and hereby, allow to make the > combination of steps explicit. > Since transducers do not build intermediate collections, their composition > is memory-efficient. > For example: > - (aBlock1 filter) * (aBlock2 map) "(1.) filter and (2.) map elements" > > > ## Re-Use > Transducers are decoupled from the input and output sources, and hence, > they can be reused in different contexts. > For example: > - enumeration of collections > - processing of streams > - communicating via channels > > > > # Usage by Example > > We build a coin flipping experiment and count the occurrence of heads and > tails. > > First, we associate random numbers with the sides of a coin. > > scale := [:x | (x * 2 + 1) floor] map. > sides := #(heads tails) replace. > > Scale is a transducer that maps numbers x between 0 and 1 to 1 and 2. > Sides is a transducer that replaces the numbers with heads an tails by > lookup in an array. > Next, we choose a number of samples. > > count := 1000 take. > > Count is a transducer that takes 1000 elements from a source. > We keep track of the occurrences of heads an tails using a bag. > > collect := [:bag :c | bag add: c; yourself]. > > Collect is binary block (reducing function) that collects events in a bag. > We assemble the experiment by transforming the block using the transducers. > > experiment := (scale * sides * count) transform: collect. > > From left to right we see the steps involved: scale, sides, count and > collect. > Transforming assembles these steps into a binary block (reducing function) > we can use to run the experiment. > > samples := Random new > reduce: experiment > init: Bag new. > > Here, we use #reduce:init:, which is mostly similar to #inject:into:. > To execute a transformation and a reduction together, we can use > #transduce:reduce:init:. > > samples := Random new > transduce: scale * sides * count > reduce: collect > init: Bag new. > > We can also express the experiment as data-flow using #<~. > This enables us to build objects that can be re-used in other experiments. > > coin := sides <~ scale <~ Random new. > flip := Bag <~ count. > > Coin is an eduction, i.e., it binds transducers to a source and > understands #reduce:init: among others. > Flip is a transformed reduction, i.e., it binds transducers to a reducing > function and an initial value. > By sending #<~, we draw further samples from flipping the coin. > > samples := flip <~ coin. > > This yields a new Bag with another 1000 samples. > > > > # Basic Concepts > > ## Reducing Functions > > A reducing function represents a single step in processing a data sequence. > It takes an accumulated result and a value, and returns a new accumulated > result. > For example: > > collect := [:col :e | col add: e; yourself]. > sum := #+. > > A reducing function can also be ternary, i.e., it takes an accumulated > result, a key and a value. > For example: > > collect := [:dic :k :v | dict at: k put: v; yourself]. > > Reducing functions may be equipped with an optional completing action. > After finishing processing, it is invoked exactly once, e.g., to free > resources. > > stream := [:str :e | str nextPut: each; yourself] completing: #close. > absSum := #+ completing: #abs > > A reducing function can end processing early by signaling Reduced with a > result. > This mechanism also enables the treatment of infinite sources. > > nonNil := [:res :e | e ifNil: [Reduced signalWith: res] ifFalse: [res]]. > > The primary approach to process a data sequence is the reducing protocol > with the messages #reduce:init: and #transduce:reduce:init: if transducers > are involved. > The behavior is similar to #inject:into: but in addition it takes care of: > - handling binary and ternary reducing functions, > - invoking the completing action after finishing, and > - stopping the reduction if Reduced is signaled. > The message #transduce:reduce:init: just combines the transformation and > the reducing step. > > However, as reducing functions are step-wise in nature, an application may > choose other means to process its data. > > > ## Reducibles > > A data source is called reducible if it implements the reducing protocol. > Default implementations are provided for collections and streams. > Additionally, blocks without an argument are reducible, too. > This allows to adapt to custom data sources without additional effort. > For example: > > "XStreams adaptor" > xstream := filename reading. > reducible := [[xstream get] on: Incomplete do: [Reduced signal]]. > > "natural numbers" > n := 0. > reducible := [n := n+1]. > > > ## Transducers > > A transducer is an object that transforms a reducing function into another. > Transducers encapsulate common steps in processing data sequences, such as > map, filter, concatenate, and flatten. > A transducer transforms a reducing function into another via #transform: > in order to add those steps. > They can be composed using #* which yields a new transducer that does both > transformations. > Most transducers require an argument, typically blocks, symbols or numbers: > > square := Map function: #squared. > take := Take number: 1000. > > To facilitate compact notation, the argument types implement corresponding > methods: > > squareAndTake := #squared map * 1000 take. > > Transducers requiring no argument are singletons and can be accessed by > their class name. > > flattenAndDedupe := Flatten * Dedupe. > > > > # Advanced Concepts > > ## Data flows > > Processing a sequence of data can often be regarded as a data flow. > The operator #<~ allows define a flow from a data source through > processing steps to a drain. > For example: > > squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). > fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. > > In both examples #<~ is only used to set up the data flow using reducing > functions and transducers. > In contrast to streams, transducers are completely independent from input > and output sources. > Hence, we have a clear separation of reading data, writing data and > processing elements. > - Sources know how to iterate over data with a reducing function, e.g., > via #reduce:init:. > - Drains know how to collect data using a reducing function. > - Transducers know how to process single elements. > > > ## Reductions > > A reduction binds an initial value or a block yielding an initial value to > a reducing function. > The idea is to define a ready-to-use process that can be applied in > different contexts. > Reducibles handle reductions via #reduce: and #transduce:reduce: > For example: > > sum := #+ init: 0. > sum1 := #(1 1 1) reduce: sum. > sum2 := (1 to: 1000) transduce: #odd filter reduce: sum. > > asSet := [:set :e | set add: e; yourself] initializer: [Set new]. > set1 := #(1 1 1) reduce: asSet. > set2 := #(1 to: 1000) transduce: #odd filter reduce: asSet. > > By combining a transducer with a reduction, a process can be further > modified. > > sumOdds := sum <~ #odd filter > setOdds := asSet <~ #odd filter > > > ## Eductions > > An eduction combines a reducible data sources with a transducer. > The idea is to define a transformed (virtual) data source that needs not > to be stored in memory. > > odds1 := #odd filter <~ #(1 2 3) readStream. > odds2 := #odd filter <~ (1 to 1000). > > Depending on the underlying source, eductions can be processed once > (streams, e.g., odds1) or multiple times (collections, e.g., odds2). > Since no intermediate data is stored, transducers actions are lazy, i.e., > they are invoked each time the eduction is processed. > > > > # Origins > > Transducers is based on the same-named Clojure library and its ideas. > Please see: > http://clojure.org/transducers >

May 31, 2017

Porting Transducers to Pharo

by Steffen MÃ¤rcker

Hi, I am the developer of the library 'Transducers' for VisualWorks. It was formerly known as 'Reducers', but this name was a poor choice. I'd like to port it to Pharo, if there is any interest on your side. I hope to learn more about Pharo in this process, since I am mainly a VW guy. And most likely, I will come up with a bunch of questions. :-) Meanwhile, I'll cross-post the introduction from VWnc below. I'd be very happy to hear your optinions, questions and I hope we can start a fruitful discussion - even if there is not Pharo port yet. Best, Steffen Transducers are building blocks that encapsulate how to process elements of a data sequence independently of the underlying input and output source. # Overview ## Encapsulate Implementations of enumeration methods, such as #collect:, have the logic how to process a single element in common. However, that logic is reimplemented each and every time. Transducers make it explicit and facilitate re-use and coherent behavior. For example: - #collect: requires mapping: (aBlock1 map) - #select: requires filtering: (aBlock2 filter) ## Compose In practice, algorithms often require multiple processing steps, e.g., mapping only a filtered set of elements. Transducers are inherently composable, and hereby, allow to make the combination of steps explicit. Since transducers do not build intermediate collections, their composition is memory-efficient. For example: - (aBlock1 filter) * (aBlock2 map) "(1.) filter and (2.) map elements" ## Re-Use Transducers are decoupled from the input and output sources, and hence, they can be reused in different contexts. For example: - enumeration of collections - processing of streams - communicating via channels # Usage by Example We build a coin flipping experiment and count the occurrence of heads and tails. First, we associate random numbers with the sides of a coin. scale := [:x | (x * 2 + 1) floor] map. sides := #(heads tails) replace. Scale is a transducer that maps numbers x between 0 and 1 to 1 and 2. Sides is a transducer that replaces the numbers with heads an tails by lookup in an array. Next, we choose a number of samples. count := 1000 take. Count is a transducer that takes 1000 elements from a source. We keep track of the occurrences of heads an tails using a bag. collect := [:bag :c | bag add: c; yourself]. Collect is binary block (reducing function) that collects events in a bag. We assemble the experiment by transforming the block using the transducers. experiment := (scale * sides * count) transform: collect. From left to right we see the steps involved: scale, sides, count and collect. Transforming assembles these steps into a binary block (reducing function) we can use to run the experiment. samples := Random new reduce: experiment init: Bag new. Here, we use #reduce:init:, which is mostly similar to #inject:into:. To execute a transformation and a reduction together, we can use #transduce:reduce:init:. samples := Random new transduce: scale * sides * count reduce: collect init: Bag new. We can also express the experiment as data-flow using #<~. This enables us to build objects that can be re-used in other experiments. coin := sides <~ scale <~ Random new. flip := Bag <~ count. Coin is an eduction, i.e., it binds transducers to a source and understands #reduce:init: among others. Flip is a transformed reduction, i.e., it binds transducers to a reducing function and an initial value. By sending #<~, we draw further samples from flipping the coin. samples := flip <~ coin. This yields a new Bag with another 1000 samples. # Basic Concepts ## Reducing Functions A reducing function represents a single step in processing a data sequence. It takes an accumulated result and a value, and returns a new accumulated result. For example: collect := [:col :e | col add: e; yourself]. sum := #+. A reducing function can also be ternary, i.e., it takes an accumulated result, a key and a value. For example: collect := [:dic :k :v | dict at: k put: v; yourself]. Reducing functions may be equipped with an optional completing action. After finishing processing, it is invoked exactly once, e.g., to free resources. stream := [:str :e | str nextPut: each; yourself] completing: #close. absSum := #+ completing: #abs A reducing function can end processing early by signaling Reduced with a result. This mechanism also enables the treatment of infinite sources. nonNil := [:res :e | e ifNil: [Reduced signalWith: res] ifFalse: [res]]. The primary approach to process a data sequence is the reducing protocol with the messages #reduce:init: and #transduce:reduce:init: if transducers are involved. The behavior is similar to #inject:into: but in addition it takes care of: - handling binary and ternary reducing functions, - invoking the completing action after finishing, and - stopping the reduction if Reduced is signaled. The message #transduce:reduce:init: just combines the transformation and the reducing step. However, as reducing functions are step-wise in nature, an application may choose other means to process its data. ## Reducibles A data source is called reducible if it implements the reducing protocol. Default implementations are provided for collections and streams. Additionally, blocks without an argument are reducible, too. This allows to adapt to custom data sources without additional effort. For example: "XStreams adaptor" xstream := filename reading. reducible := [[xstream get] on: Incomplete do: [Reduced signal]]. "natural numbers" n := 0. reducible := [n := n+1]. ## Transducers A transducer is an object that transforms a reducing function into another. Transducers encapsulate common steps in processing data sequences, such as map, filter, concatenate, and flatten. A transducer transforms a reducing function into another via #transform: in order to add those steps. They can be composed using #* which yields a new transducer that does both transformations. Most transducers require an argument, typically blocks, symbols or numbers: square := Map function: #squared. take := Take number: 1000. To facilitate compact notation, the argument types implement corresponding methods: squareAndTake := #squared map * 1000 take. Transducers requiring no argument are singletons and can be accessed by their class name. flattenAndDedupe := Flatten * Dedupe. # Advanced Concepts ## Data flows Processing a sequence of data can often be regarded as a data flow. The operator #<~ allows define a flow from a data source through processing steps to a drain. For example: squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. In both examples #<~ is only used to set up the data flow using reducing functions and transducers. In contrast to streams, transducers are completely independent from input and output sources. Hence, we have a clear separation of reading data, writing data and processing elements. - Sources know how to iterate over data with a reducing function, e.g., via #reduce:init:. - Drains know how to collect data using a reducing function. - Transducers know how to process single elements. ## Reductions A reduction binds an initial value or a block yielding an initial value to a reducing function. The idea is to define a ready-to-use process that can be applied in different contexts. Reducibles handle reductions via #reduce: and #transduce:reduce: For example: sum := #+ init: 0. sum1 := #(1 1 1) reduce: sum. sum2 := (1 to: 1000) transduce: #odd filter reduce: sum. asSet := [:set :e | set add: e; yourself] initializer: [Set new]. set1 := #(1 1 1) reduce: asSet. set2 := #(1 to: 1000) transduce: #odd filter reduce: asSet. By combining a transducer with a reduction, a process can be further modified. sumOdds := sum <~ #odd filter setOdds := asSet <~ #odd filter ## Eductions An eduction combines a reducible data sources with a transducer. The idea is to define a transformed (virtual) data source that needs not to be stored in memory. odds1 := #odd filter <~ #(1 2 3) readStream. odds2 := #odd filter <~ (1 to 1000). Depending on the underlying source, eductions can be processed once (streams, e.g., odds1) or multiple times (collections, e.g., odds2). Since no intermediate data is stored, transducers actions are lazy, i.e., they are invoked each time the eduction is processed. # Origins Transducers is based on the same-named Clojure library and its ideas. Please see: http://clojure.org/transducers

May 31, 2017

Re: [Pharo-users] pharocloud alternative

by Johannes Brauer

You see the message when you log in to the pharocloud management console. It says that there is a âlack of resourcesâ. Johannes > Am 30.05.2017 um 23:05 schrieb Petr Fischer <petr.fischer(a)me.com>: > > Why will be pharocloud closed? Where is the message? Thanks, pf > > >> Hi, >> >> I am running to Pharo/Seaside apps on pharocloud. Now the service will be closed. Does anyone know a comparable alternative? >> >> Johannes >

May 31, 2017

Re: [Pharo-users] pharocloud alternative

by Tim Mackinnon

I really, really wish that we had a vm that worked on AWS Lambdaâ¦ I think this is the future of computing - and I would dearly love to understand the âsmalltalkâ approach to function as a service (FAAS). In particular how you debug apps. I was struck by Philâs demo at Pharo days, where he talked about debugging XMPP apps, and how he couldâ¦ when I run Node apps on Lambda, it works, but its incredibly unproductive both in debugging and deploying small changes. I really think we have much nicer solutions in the Smalltalk world - but I have no idea if we could get a smalltalk vm to work in Lambda or the google app engine alternative. As indicated here - I donât want to spend time worrying about servers any more - I just want to call cloud functions and write my codeâ¦. Tim > On 30 May 2017, at 21:12, Stephan Eggermont <stephan(a)stack.nl> wrote: > > On 30/05/17 18:00, Alejandro Infante wrote: >> This year I will port everything to AWS or Azure. I expect it to be >> easier to deploy there, but Iâll let you know about my experience >> when the time comes. > > I'm pretty sure it is not easier at all. But please provide us with your experiences. > > Stephan > >

May 31, 2017

Recording TechTalk Roassal available

by Marcus Denker

Hello! The recording of the Roassal Techtalk of yesterday is available here: https://www.youtube.com/watch?v=-Pk4q5oMdLo (start at 7:35) A PDF with all the code of the demos shown: http://files.pharo.org/media/techtalk/RoassalTechTalk.pdf

May 31, 2017

Re: [Pharo-users] pharocloud alternative

by Petr Fischer

Why will be pharocloud closed? Where is the message? Thanks, pf > Hi, > > I am running to Pharo/Seaside apps on pharocloud. Now the service will be closed. Does anyone know a comparable alternative? > > Johannes

May 30, 2017