What is offline mode? 2017-10-25 15:10 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

Thanks Denis. I will ! I knew I have seen a telephoto component that could help but forgot about it !

Do you know if it’s possible to handle offline mode ?

Le 25 oct. 2017 à 15:05, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

Look at Seamless https://github.com/dionisiydk/Seamless.

2017-10-25 14:21 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Hi all,

I want to connect two applications (1 by image, each one on a different computer) so as as to exchange information (data) between them.

So my question is about the best (smalltalk) practices to connect two app/image and exchange data.

I imagine either with a direct connection through a network (TCP Socket, Web socket, pure HTTP with Zinc) and/or with a serial connection. At first, without any « security ». But later, information exchanges will be encrypted.

I’ve seen some information on how to use SerialPort, or even FileStream. I could do it (or at least simulate it with HTTP). What are the other options ? Socket ? Do we have P2P libs (I couldn’t find) with eventually discovery features ?

Any comment / suggestion / pointers are greatly welcome.

TIA.

Cédrick

Yes, it is not part of Seamless. 2017-10-25 15:40 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

I had a look and this is not (natively) possible.

The idea of the off-line mode would be to delay messages that are sent to the peer until a connection is established.

I think I have to try to do it by myself (like having a list of information exchange that wait until a connexion is established).

What I try to do is not as complex as two general image exchanging messages on objects (like on TelePharo).

I just want a repository of information (mainly a collection of static information/data versions) on both peers to be synchronized when a connection is established.

Le 25 oct. 2017 à 15:31, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

What is offline mode?

2017-10-25 15:10 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Thanks Denis. I will ! I knew I have seen a telephoto component that could help but forgot about it !

Do you know if it’s possible to handle offline mode ?

Le 25 oct. 2017 à 15:05, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

Look at Seamless https://github.com/dionisiydk/Seamless.

2017-10-25 14:21 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Hi all,

I want to connect two applications (1 by image, each one on a different computer) so as as to exchange information (data) between them.

So my question is about the best (smalltalk) practices to connect two app/image and exchange data.

I imagine either with a direct connection through a network (TCP Socket, Web socket, pure HTTP with Zinc) and/or with a serial connection. At first, without any « security ». But later, information exchanges will be encrypted.

I’ve seen some information on how to use SerialPort, or even FileStream. I could do it (or at least simulate it with HTTP). What are the other options ? Socket ? Do we have P2P libs (I couldn’t find) with eventually discovery features ?

Any comment / suggestion / pointers are greatly welcome.

TIA.

Cédrick

Am 25.10.2017 um 15:52 schrieb Cédrick Béler:

Ok.

I wonder also if Basys is not enough to experiment. https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...

I’ll also try Sebastian suggestion.

ZeroMQ is a wonderful library. We used it together with Gemstone/s to build an "Event data bus" based on ZeroMQ ... and with each successful commit our Gemstone/S send events on this "bus". On the "bus" several different programs written in different languages are listening and do some reaction based on the events they were interested in. Marten Feldtmann

Hi andrew I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo. Stef On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk?

Andrew

On Thu, Oct 26, 2017 at 3:39 AM, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi andrew

I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo.

Stef

BTree is a database indexing implementation. Actually it would be a good feature to have available with Pharo. BDD is diagram[1] . Roassal would be the best bet. If it doesn't have an out of the box BDD, all the pieces are there make one. And the Roassal guys love new use cases. [1] https://www.lucidchart.com/pages/decision-tree cheers -ben

On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

...

Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk?

Andrew

Dear Andrew, I didn't find time to answer earlier. Some time ago, I was looking for a (MT)BDD package in ST as well. I didn't found one. So the only two options left are 1) implementing a new BDD lib in ST and 2) doing FFI to some existing lib, e.g. CUDD, Sylvan, Jinc I'd prefer 2) since the existing libraries are feature-rich and highly optimized - which took quite some time. As a bonus, a solution could potentially switch between those backends. The biggest hurdle, in my option, is memory management, since most libs use some sort of reference counting. And you do not want to end up with nightmarish dozens of ref(bddNode) deref(bddNode) in your application code (like the probabilistic model checker PRISM does). This introduces hard to track bugs easily. However, I have an idea in mind how to tackle this but I didn't found the time to put it into code yet. May I ask, which sort of application do you have in mind? Best, Steffen Am .10.2017, 07:54 Uhr, schrieb Prof. Andrew P. Black <black@cs.pdx.edu>:

Thanks for the responses so far. I see that I need to clarify my enquiry.

B-Trees and BDDs are not the same. BDDs are an efficient and compact representations for Boolean functions, sometimes used in SAT-solvers and electronics design. The key idea is that since the output must be 0 or 1, you can represent any Boolean function as a tree whose depth is the same as the number of bits in the input.

To make the tree small and efficient, though, you need to eliminate any node whose two children are the same, and to share subtrees, so that you really get a DAG, not a tree. The full name for these efficient compressed trees is “Reduced Order Binary Decision Diagrams”, or ROBDDs. I was hoping that someone else had implemented the algorithms necessary to build this representation.

Because sets can be considered to be Booleans functions (true => argument is in the set), you can use ROBDDs to efficiently represent large sets.

To be clear, despite the word “diagram” in the name, one is not normally interested in drawing the BDD — except in the documentation for the package ;-). Normally, BDDs they are used to represent sets, or functions, where the drawing would be hopelessly large.

The BuDDy package (http://buddy.sourceforge.net/manual/main.html) is an example of what I’m looking for, but unfortunately it’s in C++.

Andrew

...

On 25 Oct 2017, at 21:39 , Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi andrew

I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo.

Stef

On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

...

Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk?

Andrew

I see. What is the task in detail? Are some of the set fixed or known in advance? What's the argument against a bitset-based solution? Cheers, Steffen Am 27. Oktober 2017 19:10:35 MESZ schrieb Stephane Ducasse <stepharo.self@gmail.com>:

It was for test inclusion of UTF-8 characters so we do not want to rely on external libraries.

On Fri, Oct 27, 2017 at 1:54 PM, Steffen Märcker <merkste@web.de> wrote:

...

Dear Andrew,

I didn't find time to answer earlier. Some time ago, I was looking for a (MT)BDD package in ST as well. I didn't found one. So the only two options left are

1) implementing a new BDD lib in ST and 2) doing FFI to some existing lib, e.g. CUDD, Sylvan, Jinc

I'd prefer 2) since the existing libraries are feature-rich and highly optimized - which took quite some time. As a bonus, a solution could potentially switch between those backends. The biggest hurdle, in my option, is memory management, since most libs use some sort of reference counting. And you do not want to end up with nightmarish dozens of ref(bddNode) deref(bddNode) in your application code (like the probabilistic model checker PRISM does). This introduces hard to track bugs easily. However, I have an idea in mind how to tackle this but I didn't found the time to put it into code yet.

May I ask, which sort of application do you have in mind?

Best, Steffen

Am .10.2017, 07:54 Uhr, schrieb Prof. Andrew P. Black <black@cs.pdx.edu>:

...

Thanks for the responses so far. I see that I need to clarify my enquiry.

B-Trees and BDDs are not the same. BDDs are an efficient and compact representations for Boolean functions, sometimes used in SAT-solvers and electronics design. The key idea is that since the output must be 0 or 1, you can represent any Boolean function as a tree whose depth is the same as the number of bits in the input.

To make the tree small and efficient, though, you need to eliminate any node whose two children are the same, and to share subtrees, so that you really get a DAG, not a tree. The full name for these efficient compressed trees is “Reduced Order Binary Decision Diagrams”, or ROBDDs. I was hoping that someone else had implemented the algorithms necessary to build this representation.

Because sets can be considered to be Booleans functions (true => argument is in the set), you can use ROBDDs to efficiently represent large sets.

To be clear, despite the word “diagram” in the name, one is not normally interested in drawing the BDD — except in the documentation for the package ;-). Normally, BDDs they are used to represent sets, or functions, where the drawing would be hopelessly large.

The BuDDy package (http://buddy.sourceforge.net/manual/main.html) is an example of what I’m looking for, but unfortunately it’s in C++.

Andrew

...

On 25 Oct 2017, at 21:39 , Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi andrew

I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo.

Stef

On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

...

Does anyone know of a BDD — that’s Binary Decision Diagram —

package

...

written in Smalltalk?

Andrew

Part of the reasoning is that by writing JVM bytecode, the differences between the various JVM languages become largely irrelevant, though in some ways it becomes slightly less convenient if calling, for example, Scala or Clojure code from Pharo. The other is inherent JVM limitations and discrepancies between Java as a semantic spec and various specific implementations as dynamic systems, particularly since the latter differ both by Java version and by which frameworks might be in use, and dynamically create further discrepancies contingently based not only on the above, but also specifics of the system and its current state. Cheers Andrew Sent from Mail for Windows 10 From: Stephane Ducasse Sent: Saturday, October 28, 2017 10:06 AM To: Any question about pharo is welcome Subject: Re: [Pharo-users] Binary Decision Diagram Package in Smalltalk I think that andrew would like to improve smacc when parsing inputs containing utf-8 characters. On Sat, Oct 28, 2017 at 1:46 PM, Steffen Märcker <merkste@web.de> wrote:

I see. What is the task in detail? Are some of the set fixed or known in advance? What's the argument against a bitset-based solution?

Cheers, Steffen

Am 27. Oktober 2017 19:10:35 MESZ schrieb Stephane Ducasse <stepharo.self@gmail.com>:

...

It was for test inclusion of UTF-8 characters so we do not want to rely on external libraries.

On Fri, Oct 27, 2017 at 1:54 PM, Steffen Märcker <merkste@web.de> wrote:

...

Dear Andrew,

I didn't find time to answer earlier. Some time ago, I was looking for a (MT)BDD package in ST as well. I didn't found one. So the only two options left are

1) implementing a new BDD lib in ST and 2) doing FFI to some existing lib, e.g. CUDD, Sylvan, Jinc

I'd prefer 2) since the existing libraries are feature-rich and highly optimized - which took quite some time. As a bonus, a solution could potentially switch between those backends. The biggest hurdle, in my option, is memory management, since most libs use some sort of reference counting. And you do not want to end up with nightmarish dozens of ref(bddNode) deref(bddNode) in your application code (like the probabilistic model checker PRISM does). This introduces hard to track bugs easily. However, I have an idea in mind how to tackle this but I didn't found the time to put it into code yet.

May I ask, which sort of application do you have in mind?

Best, Steffen

Am .10.2017, 07:54 Uhr, schrieb Prof. Andrew P. Black <black@cs.pdx.edu>:

...

Thanks for the responses so far. I see that I need to clarify my enquiry.

B-Trees and BDDs are not the same. BDDs are an efficient and compact representations for Boolean functions, sometimes used in SAT-solvers and electronics design. The key idea is that since the output must be 0 or 1, you can represent any Boolean function as a tree whose depth is the same as the number of bits in the input.

To make the tree small and efficient, though, you need to eliminate any node whose two children are the same, and to share subtrees, so that you really get a DAG, not a tree. The full name for these efficient compressed trees is “Reduced Order Binary Decision Diagrams”, or ROBDDs. I was hoping that someone else had implemented the algorithms necessary to build this representation.

Because sets can be considered to be Booleans functions (true => argument is in the set), you can use ROBDDs to efficiently represent large sets.

To be clear, despite the word “diagram” in the name, one is not normally interested in drawing the BDD — except in the documentation for the package ;-). Normally, BDDs they are used to represent sets, or functions, where the drawing would be hopelessly large.

The BuDDy package (http://buddy.sourceforge.net/manual/main.html) is an example of what I’m looking for, but unfortunately it’s in C++.

Andrew

...

On 25 Oct 2017, at 21:39 , Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi andrew

I think that Avi did a package about BDD (but I thought it was special binary trees) so this is probably the same. Did you check on Squeaksource? http://www.squeaksource.com/BTree.html If this is what you are looking for I can help porting it to Pharo.

Stef

On Wed, Oct 25, 2017 at 9:02 PM, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

...

Does anyone know of a BDD — that’s Binary Decision Diagram — package written in Smalltalk?

Andrew

Thanks I think that the sets will be known since the scanner definition should not changed once defined. Now andrew told me that he could use an existing solution but not build one because he should make progress on his compiler. But Steffen if you know how to build such package it would be a nice addition and we will use it. Stef On Sat, Oct 28, 2017 at 5:37 PM, Steffen Märcker <merkste@web.de> wrote:

Does that mean the sets/bdd would be constructed mainly at comile time? Anyway, Andrew, feel free to contact me, I might help you with this.

Best, Steffen

Am .10.2017, 16:05 Uhr, schrieb Stephane Ducasse <stepharo.self@gmail.com>:

...

I think that andrew would like to improve smacc when parsing inputs containing utf-8 characters.

On Sat, Oct 28, 2017 at 1:46 PM, Steffen Märcker <merkste@web.de> wrote:

...

I see. What is the task in detail? Are some of the set fixed or known in advance? What's the argument against a bitset-based solution?

Cheers, Steffen

Am 27. Oktober 2017 19:10:35 MESZ schrieb Stephane Ducasse <stepharo.self@gmail.com>:

<---Schnitt--->

...

...

Hi Andrew, Steffen, 2017-11-07 13:10 GMT+01:00 Prof. Andrew P. Black <black@cs.pdx.edu>:

...

On 28 Oct 2017, at 17:37 , Steffen Märcker <merkste@web.de> wrote:

Does that mean the sets/bdd would be constructed mainly at comile time? Anyway, Andrew, feel free to contact me, I might help you with this.

Thanks for the offer, Steffen! The problem is that I need to use SmaCC for my current project, and really do not have a month to take off and re-design the way that it builds its scanner. I’ve talked to Thierry Goubier about, and he doesn’t have time either! It would be a fun project, though, and it ought to be fairly separate from other parts of SmaCC. I’ve spent a fair bit of time thinking about how to do it, but don’t think that I will be able to actually focus on it.

Yes, this is the essence of the issue. There are a few alternatives about it, but none we have the time to pursue.

An alternative approach, which Thierry has suggested, is to make SmaCC work on the UTF-8 representation of the Unicode. Then we could represent character sets as prefix trees. But the core problem would still exist: you can’t run an algorithm that repeatedly executes

for all characters in the alphabet do:

when there are 2^21 characters in the alphabet!

The main issue is that `for all characters`... All the literature on scanner building uses 'for all characters do'. Thierry

Andrew

I am not familiar with the literature on scanners. May I ask you about some details on the "for all characters" algorithms you are referring to? Building a (or connecting to) a BDD library would be fun, indeed. But within that time frame it seems not realistic. Anyway, after finishing my thesis, I'd like to come back to that idea. Ciao, Steffen Am 7. November 2017 16:33:03 MEZ schrieb Andrew Glynn <aglynn42@gmail.com>:

A possible way to accomplish it would be to use an object graph with an incremental query engine, such as EMF/CDO with Viatra or something similar. You could then put different character sets in connected objects and query only as far as you need to.

Andrew Glynn

Sent from Mail for Windows 10

From: Thierry Goubier Sent: Tuesday, November 7, 2017 7:17 AM To: Any question about pharo is welcome Subject: Re: [Pharo-users] Binary Decision Diagram Package in Smalltalk

Hi Andrew, Steffen,

2017-11-07 13:10 GMT+01:00 Prof. Andrew P. Black <black@cs.pdx.edu>:

...

On 28 Oct 2017, at 17:37 , Steffen Märcker <merkste@web.de> wrote:

Does that mean the sets/bdd would be constructed mainly at comile time? Anyway, Andrew, feel free to contact me, I might help you with this.

Thanks for the offer, Steffen!  The problem is that I need to use SmaCC for my current project, and really do not have a month to take off and re-design the way that it builds its scanner.  I’ve talked to Thierry Goubier about, and he doesn’t have time either!  It would be a fun project, though, and it ought to be fairly separate from other parts of SmaCC.  I’ve spent a fair bit of time thinking about how to do it, but don’t think that I will be able to actually focus on it.

Yes, this is the essence of the issue. There are a few alternatives about it, but none we have the time to pursue.  

An alternative approach, which Thierry has suggested, is to make SmaCC work on the UTF-8 representation of the Unicode.  Then we could represent character sets as prefix trees.  But the core problem would still exist: you can’t run an algorithm that repeatedly executes

                for all characters in the alphabet do:

when there are 2^21 characters in the alphabet!

The main issue is that `for all characters`... All the literature on scanner building uses 'for all characters do'.

Thierry  

        Andrew

I see. How about the following (sketched) solution to avoid looping over all characters? It might be very well the case that you already considered (and dismissed) that path. A) Assumption In order to allow any meaningful matching, the input to the scanner is normalized according to the unicode spec. B) Abstraction Treat each character and character group of an regex as a set of intervals in the unicode code points. Lets call them "character tests" and lift the common set operations union, intersection and difference to them. C) Construct NFA NFA has potentially overlapping character tests at the transitions of each state. D) Construct DFA Given a product state s in the DFA and two transitions t1, t2 from the original NFA, add three new transitions to the DFA: - a transition labeled with the character test of t1 minus the character test of t2 - a transition labeled with the intersection of the character tests of t1 and t2 - a transition labeled with the character test of t2 minus the character test of t1 E) Extension Instead of sets of unicode intervals we could also use test-functions, e.g. blocks. Then, in step D), the set operations are translated to boolean operations: - difference t1 - t2 becomes: t1 && not t2 - intersection of t1 and t2 becomes: t1 && t2 This would allow to use optimized test functions, e.g., bdds, instead of relying on charcter tests only. Cheers, Steffen Am .11.2017, 23:16 Uhr, schrieb Thierry Goubier <thierry.goubier@gmail.com>:

Le 07/11/2017 à 23:00, Steffen Märcker a écrit :

...

I am not familiar with the literature on scanners. May I ask you about some details on the "for all characters" algorithms you are referring to?

The two main ones available, from the typical Aho/Ullman textbook, are:

- NFA to DFA conversion (i.e. build a NFA with your regular expressions, then convert it to a DFA)

- Direct regular expression to DFA construction

Both of them have loop of the type:

for (each input symbol a) { ...

...

Building a (or connecting to) a BDD library would be fun, indeed. But within that time frame it seems not realistic. Anyway, after finishing my thesis, I'd like to come back to that idea.

It would certainly be interesting. Please contact us again when you'll have time :)

Regards,

Thierry

...

Ciao, Steffen Am 7. November 2017 16:33:03 MEZ schrieb Andrew Glynn <aglynn42@gmail.com>: A possible way to accomplish it would be to use an object graph with an incremental query engine, such as EMF/CDO with Viatra or something similar. You could then put different character sets in connected objects and query only as far as you need to. Andrew Glynn Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 *From: *Thierry Goubier <mailto:thierry.goubier@gmail.com> *Sent: *Tuesday, November 7, 2017 7:17 AM *To: *Any question about pharo is welcome <mailto:pharo-users@lists.pharo.org> *Subject: *Re: [Pharo-users] Binary Decision Diagram Package in Smalltalk Hi Andrew, Steffen, 2017-11-07 13:10 GMT+01:00 Prof. Andrew P. Black <black@cs.pdx.edu <mailto:black@cs.pdx.edu>>: > On 28 Oct 2017, at 17:37 , Steffen Märcker <merkste@web.de <mailto:merkste@web.de>> wrote: > > Does that mean the sets/bdd would be constructed mainly at comile time? Anyway, Andrew, feel free to contact me, I might help you with this. > Thanks for the offer, Steffen! The problem is that I need to use SmaCC for my current project, and really do not have a month to take off and re-design the way that it builds its scanner. I’ve talked to Thierry Goubier about, and he doesn’t have time either! It would be a fun project, though, and it ought to be fairly separate from other parts of SmaCC. I’ve spent a fair bit of time thinking about how to do it, but don’t think that I will be able to actually focus on it. Yes, this is the essence of the issue. There are a few alternatives about it, but none we have the time to pursue. An alternative approach, which Thierry has suggested, is to make SmaCC work on the UTF-8 representation of the Unicode. Then we could represent character sets as prefix trees. But the core problem would still exist: you can’t run an algorithm that repeatedly executes for all characters in the alphabet do: when there are 2^21 characters in the alphabet! The main issue is that `for all characters`... All the literature on scanner building uses 'for all characters do'. Thierry Andrew

Nothing complex about two images exchanging messages, its not even complex to transmit objects via fuel, you even transmit a debugger or any part of the live environment or even make an "internet" of images that join objects together. Sky is the limit. Pharo already provides you will all the tools/libraries to do this. A streamsocket (not to be confused with regular sockets) will delay the messages (a collection of bytes) until they arrive to the receiver or until they have reached the timeout period. Offline mode is pretty much obligatory even for plain old internet web apps because of drops in connection or the plain fact a connection can become slow. I used streamsockets in my Pharo to Python bridge (Atlas) because the execution was not necessary synchronous , I was sending Python command to a 3d application from Pharo and I had to make sure that the bridge was working even in the case of a command that could take hours to execute like a rendering process. One cool trick I did was to send the Python errors back to Pharo and trigger them as Pharo's regular MessageNotUnderstood , this is a nice way to make sure that you can fix a wrong message after it has been executing without going to the image that is executing it. The limitation with sockets which what every frameworks uses because AFAIK they are the only means to communicate remotely is that they are not top performance so that mean that you can send long loops but executing communication inside long loops will either slow you down considerably or simply timeout your socket. Sockets can timeout from both ends if they feel that for some reason they lost communication with the other side and reached their timeout period. Timeout period is customization. This is a problem I did not tackle with my implementation because I dont think there can be an actual practical solution better than avoiding this scenario. Only shared memory seems to overcome this but it can be used only locally and not remotely (aka on same computer) . On Wed, Oct 25, 2017 at 4:41 PM Cédrick Béler <cdrick65@gmail.com> wrote:

I had a look and this is not (natively) possible.

The idea of the off-line mode would be to delay messages that are sent to the peer until a connection is established.

I think I have to try to do it by myself (like having a list of information exchange that wait until a connexion is established).

What I try to do is not as complex as two general image exchanging messages on objects (like on TelePharo).

I just want a repository of information (mainly a collection of static information/data versions) on both peers to be synchronized when a connection is established.

Le 25 oct. 2017 à 15:31, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

What is offline mode?

2017-10-25 15:10 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Thanks Denis. I will ! I knew I have seen a telephoto component that could help but forgot about it !

Do you know if it’s possible to handle offline mode ?

Le 25 oct. 2017 à 15:05, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

Look at Seamless https://github.com/dionisiydk/Seamless.

2017-10-25 14:21 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Hi all,

I want to connect two applications (1 by image, each one on a different computer) so as as to exchange information (data) between them.

So my question is about the best (smalltalk) practices to connect two app/image and exchange data.

I imagine either with a direct connection through a network (TCP Socket, Web socket, pure HTTP with Zinc) and/or with a serial connection. At first, without any « security ». But later, information exchanges will be encrypted.

I’ve seen some information on how to use SerialPort, or even FileStream. I could do it (or at least simulate it with HTTP). What are the other options ? Socket ? Do we have P2P libs (I couldn’t find) with eventually discovery features ?

Any comment / suggestion / pointers are greatly welcome.

TIA.

Cédrick

Hi Cedric, a short answer: some of what you're trying to do has been traditionnally handled by object databases - multiple images retrieving and updating objects on a central store ensuring lots of good properties : see Gemstone. Another answer, since you're looking at content-addressable distributed file systems. You can resolve the offline mode with object duplications (duplicate all objects in the images using them), and, when coming back online, have a merge approach to reconcile changes between versions of shared objects. Content-based hashes as identifiers in a distributed store have very nice properties for that purpose, because, if image C has modified object 0xcdaff3, then that object has in fact become object 0xee345d for that image (and the unmodified object is still 0xcdaff3 for images A and B). I wouldn't be against a slightly higher granularity when dealing with object transfers, however. Regards, Thierry 2017-10-27 10:43 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

Nothing complex about two images exchanging messages, its not even complex to transmit objects via fuel, you even transmit a debugger or any part of the live environment or even make an "internet" of images that join objects together. Sky is the limit. Pharo already provides you will all the tools/libraries to do this.

Yes. That’s why I asked for better practices.

A streamsocket (not to be confused with regular sockets) will delay the messages (a collection of bytes) until they arrive to the receiver or until they have reached the timeout period. Offline mode is pretty much obligatory even for plain old internet web apps because of drops in connection or the plain fact a connection can become slow.

Offline mode is to me of first importance.

Let’s say I have a sets of Pharo app (nodes). Each one are independent running on different devices/computers.

I want to exchange information between them according to some kind of contracts and also on connection availability.

What I find different from usual message exchanges between two images is that I want to have full control on it. Plus I consider images offline by default (meaning two images have different versions of the same info). So conflict is the norm.

Also, I don’t want to send message remotely (with behavior), I just want to send/share an information and sync them (when possible).

Connection between 2 images can be through a network but eventually through a serial connection (usb cable, Bluetooth,...).

So what I try to do as a proto is having connection controlled by my apps (I put the app offline (not by switching off wifi but by switching logically). I can see nodes availability and activate connexion manually then see if conflict resolution is ok, + I log every connecitions changes and exchanges). This is my plans for now but it may change ! ^^

I used streamsockets in my Pharo to Python bridge (Atlas) because the execution was not necessary synchronous , I was sending Python command to a 3d application from Pharo and I had to make sure that the bridge was working even in the case of a command that could take hours to execute like a rendering process.

I’ll look at stream sockets and mq stuffs.

Cheers,

Cédrick

One cool trick I did was to send the Python errors back to Pharo and trigger them as Pharo's regular MessageNotUnderstood , this is a nice way to make sure that you can fix a wrong message after it has been executing without going to the image that is executing it.

The limitation with sockets which what every frameworks uses because AFAIK they are the only means to communicate remotely is that they are not top performance so that mean that you can send long loops but executing communication inside long loops will either slow you down considerably or simply timeout your socket. Sockets can timeout from both ends if they feel that for some reason they lost communication with the other side and reached their timeout period. Timeout period is customization.

This is a problem I did not tackle with my implementation because I dont think there can be an actual practical solution better than avoiding this scenario.

Only shared memory seems to overcome this but it can be used only locally and not remotely (aka on same computer) .

On Wed, Oct 25, 2017 at 4:41 PM Cédrick Béler <cdrick65@gmail.com> wrote:

...

I had a look and this is not (natively) possible.

The idea of the off-line mode would be to delay messages that are sent to the peer until a connection is established.

I think I have to try to do it by myself (like having a list of information exchange that wait until a connexion is established).

What I try to do is not as complex as two general image exchanging messages on objects (like on TelePharo).

I just want a repository of information (mainly a collection of static information/data versions) on both peers to be synchronized when a connection is established.

Le 25 oct. 2017 à 15:31, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

What is offline mode?

2017-10-25 15:10 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Thanks Denis. I will ! I knew I have seen a telephoto component that could help but forgot about it !

Do you know if it’s possible to handle offline mode ?

Le 25 oct. 2017 à 15:05, Denis Kudriashov <dionisiydk@gmail.com> a écrit :

Look at Seamless https://github.com/dionisiydk/Seamless.

2017-10-25 14:21 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

...

Hi all,

I want to connect two applications (1 by image, each one on a different computer) so as as to exchange information (data) between them.

So my question is about the best (smalltalk) practices to connect two app/image and exchange data.

I imagine either with a direct connection through a network (TCP Socket, Web socket, pure HTTP with Zinc) and/or with a serial connection. At first, without any « security ». But later, information exchanges will be encrypted.

I’ve seen some information on how to use SerialPort, or even FileStream. I could do it (or at least simulate it with HTTP). What are the other options ? Socket ? Do we have P2P libs (I couldn’t find) with eventually discovery features ?

Any comment / suggestion / pointers are greatly welcome.

TIA.

Cédrick

2017-10-27 11:28 GMT+02:00 Cédrick Béler <cdrick65@gmail.com>:

Hi Cedric,

a short answer: some of what you're trying to do has been traditionnally handled by object databases - multiple images retrieving and updating objects on a central store ensuring lots of good properties : see Gemstone.

Yes , I’d like to avoid this centralized approach.

I mean that they have already some of the mechanisms in place, even if centralized.

Another answer, since you're looking at content-addressable distributed file systems. You can resolve the offline mode with object duplications (duplicate all objects in the images using them), and, when coming back online, have a merge approach to reconcile changes between versions of shared objects. Content-based hashes as identifiers in a distributed store have very nice properties for that purpose, because, if image C has modified object 0xcdaff3, then that object has in fact become object 0xee345d for that image (and the unmodified object is still 0xcdaff3 for images A and B).

You nailed it. This is what I’d like to reproduce.

Existing implementations out there seems to uses whatever nodes on the network to replicate the information.

Yes, because that makes them decentralized :)

I’d like to control nodes where it is replicated. My nodes (all my app instances + nodes of person I’m exchanging information with + eventually friend of friend).

We've done recent work on capability-based content adressing, but, first it's very slow (so you use a two level cryptosystem: the crypted header contains the key to decrypt the content, and the crypted header can only be decrypted if your private key has the right level of capabilities on the data item.

What hash function would you use ?

Anything that is fast, bonus points if it uses CPU special instructions (that you can't use from Pharo, of course ;)), and has the right cryptographic properties. Unless you go into a specific cryptosystem, I'd say that it is not important.

To get something compatible with ipfs, I’d need something like: https://github.com/multiformats/multihash It looks to me quite universal as self describing. But any (existing) hash method compatible with content hashing would do the job.

Interesting, but it looks like a minor issue in the overall scheme. Makes some of your system robust to evolution in the hash used, but, since first bytes are not well distributed, can you use it safely to build a multi-hash function system? Probably not.

I wouldn't be against a slightly higher granularity when dealing with object transfers, however.

You mean at the pharo level ? Higher granularity means having more control on the exchange/merge ?

No, just that the content-based address scheme is costly... and that a practical implementation would probably look to provide addresses only to large enough entities (a page containing objects, for example, or a large objects containing smaller ones). So that you donc create an address for each character of the string object describing the name of a person, say). Regards, Thierry

Cheers,

Cédrick

On 26 October 2017 at 18:37, Dimitris Chloupis <kilon.alios@gmail.com> wrote:

Nothing complex about two images exchanging messages, its not even complex to transmit objects via fuel, you even transmit a debugger or any part of the live environment or even make an "internet" of images that join objects together. Sky is the limit. Pharo already provides you will all the tools/libraries to do this.

A streamsocket (not to be confused with regular sockets) will delay the messages (a collection of bytes) until they arrive to the receiver or until they have reached the timeout period. Offline mode is pretty much obligatory even for plain old internet web apps because of drops in connection or the plain fact a connection can become slow.

I used streamsockets in my Pharo to Python bridge (Atlas) because the execution was not necessary synchronous , I was sending Python command to a 3d application from Pharo and I had to make sure that the bridge was working even in the case of a command that could take hours to execute like a rendering process.

And others are finding similar results... "Inter-process communication is a (surprisingly) effective way to couple two programming languages " R into Go... http://user2015.math.aau.dk/presentations/236.pdf cheers -ben