- Pharo-users - lists.pharo.org

Re: [Pharo-users] [Pharo-dev] [ANN] Success story Mobility Map

by Norbert Hartl

> Am 25.09.2018 um 17:44 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: > > > >> On 25 Sep 2018, at 14:39, Norbert Hartl <norbert(a)hartl.name> wrote: >> >> >> >>> Am 25.09.2018 um 12:52 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >>> >>> Wow. Very nice, well done. >>> >>> Any chance on some more technical details, as in what 'connected by a message queue for the communication' exactly means ? How did you approach micro services exactly ? >>> >> Sure :) > > Thanks, this is very interesting ! > >> The installation spawns multiple physical machines. All the machines are joined to a docker swarm. The installation is reified as either task or service from the view on the docker swarm. Meaning you instantiate an arbitrary amount of services and docker swarm distributes them among the physical machines. Usually you donât take control which is running where but you can. At this point you have spread dozens of pharo images among multiple machines and each of them has an IP address. Furthermore in docker swarm you have a reification of a network meaning that every instance in a network can see all other instances on this network. Each service can be reached by its service name in that network. Docker swarm does all the iptables/firewall and DNS setup for you. > > Are you happy with docker swarm's availability/fail-over behaviour ? In other words: does it work when one image/instance goes bad, does it detect and restore the missing functionality ? > Yes, Iâm very satisified. Instances are automatically restarted if they crash. And not necessarily on the same machine exactly how I expect it. Docker has something called Healthcheck. You can have a command executed every 20 seconds. I hooked this to a curl command and to your SUnit Rest handler. The rest is writing unit tests for server health. If tests fail in sequence the instances are taken out of operation and are replaced with a new instance. The same is done for updating. Instances are started in a fashion that one instance of the new image is started, if it survives a couple of health checks it is taken operational and an old one is taken out. Then the next new is started... For simple software updates you have zero-downtime deployment. >> In order to have communication between those runtimes we use rabbitmq because you were so nice writing a driver for it ;) The rabbitmq does have a support for cluster setup, meaning each of the physical machines has a rabbitmq installation and they know each other. So it does not matter to which instance you send messages to and on which you register for receiving messages. So every pharo image connects to the service rabbitmq and opens a queue for interaction. > > Same question: does RabbitMQ's clustering work well under stress/problems ? Syncing all queues between all machines sounds quite heavy (I never tried it, but maybe it just works). I did not yet have time to real stress test the queue. And you are right copying between nodes might be a lot but still I have the feeling it is better to have then a single instance but no real reason. We also use huge payloads which might have to change if we encounter problems. > >> Each service like the car sharing opens a queue e.g. /queue/carSharing and listens on it. The broker images are stateful so they open queues like /queue/mobility-map-afdeg32 where afdeg32 is the container id of the instance (hostname in docker). In each request the queue name to reply is sent as a header. So we can make sure that the right image gets the message back. This way we can have sticky sessions keeping volatile data in memory for the lifecycle of a session. There is one worker image which opens a queue /queue/mobility-map where session independent requests can be processed. > > I think I understand ;-) > >> In order to ease development we are sharing code between the broker and the micro service. Each micro service has a -Common package where the classes are in that build the interface. The classes in here are a kind of data entity facades. They use NeoJSON to map to and from a stream. The class name is send with the message as a header so the remote side knows what to materialize. The handling is unified for the four cases >> >> - Request as inquiry to another micro service >> - Response returns values to a Request >> - Error is transferred like a Response but is then signalled on the receiving side >> - Notification connects the announcers on the broker and the micro service side. > > Yes, makes total sense. > >> Asynchronous calls we solved using Promises and Futures. Each async call to the Q becomes a promise (that blocks on #value) and is combined to a future value containing all promises with support to generate a delta of all resolved promises. This we need because you issue a search that takes longer and you want to display results as soon as they are resolved not after all haven been resolved. > > Which Promise/Future framework/library are you using in Pharo ? > We rolled our own. And I donât know about other frameworks. > You did not go for single threaded worker images ? For the micro services yes. They read from the queue one by one and process it. Here I scale out with multiple pharo instances. On the API image we have zinc for HTTP that is a multithreaded server and the clients are polling. Norbert > >> And a lot more. This is a coarse grained overview over the architecture. Iâm happy to answer further questions about this. >> >> Norbert >> >>>> On 25 Sep 2018, at 12:20, Norbert Hartl <norbert(a)hartl.name> wrote: >>>> >>>> As presented on ESUG here is the brief description of one of our current projects. >>>> >>>> Mobility Map >>>> ââââââ >>>> >>>> Mobility Map is a broker for mobility services. It offers multi-modal routing search enabling users to find the best travel options between locations. Travel options include car sharing, bikes, trains, busses etc. Rented cars can be offered for ride sharing on booking time letting other people find it to participate in the ride. Single travel options are combined in travel plans that can be booked and managed in a very easy way. >>>> >>>> For this project main requirements were scalability to serve a large user base and flexibility to add more additional providers to the broker. The application has been realized using web technologies for the frontend and pharo for the backend. Using a microservice architecture combined with a broker it is easy to extend the platform with additional brokers. Deployment is done using docker swarm for distributing dozens of pharo images among multiple server machines connected by a message queue for the communication. Pharo supported that scenario very well enabling us the meet the requirements with less effort. >>>> >>>> Pharo turned out to be a perfect fit to develop the application in a agile way. Small development cycles with continuous integration and continuous delivery enables fast turnarounds for the customers to validate progress. >>>> >>>> This is a screenshot of the search page for multi-modal results: >>>> >>>> >>>> <Screen Shot 2018-09-21 at 16.54.30.png> >>> >>> >> >> > >

Sept. 26, 2018

Re: [Pharo-users] Pharo launcher

by Trussardi Dario Romano

Ciao Christophe, > Hi Dario, > > The problem is probably fixed in the latest PharoLauncher version (1.4) available from http://pharo.org/download. Yes, i download the last Pharo launcher ( 1.4 ) and everything works, > If not, as specified in the dialog text, please run the given command from the command line to give us more information on what is the problem. > Thanks, Dario > Regards, > Christophe > >> Le 26 sept. 2018 Ã 12:05, Trussardi Dario Romano <dario.trussardi(a)tiscali.it> a Ã©crit : >> >> Ciao, >> >> on Ubuntu system with Pharo Launcher 1.3 >> >> after create a new Pharo 7.0-64 bit ( development version ) >> >> when i launch the images the system answer : >> >> Cannot determine image version: image version file not found! >> Try to run this command in a shell to get more information on the problem: >> cd "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836" && export SQUEAK_PLUGINS="" && "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836/pharo" --nodisplay "/home/party/Pharo/images/Pharo 7.0 - 64bit (180926)-01/Pharo 7.0 - 64bit (180926)-01.image" eval "(Smalltalk imagePath asFileReference parent) / 'pharo.version' writeStreamDo: [ :stream | |major minor| major := SystemVersion current major. minor := SystemVersion current minor. (major = 6 and: [ SystemVersion current highestUpdate >= 60509 ]) ifTrue: [ minor := 1 ]. stream << major asString; << minor asString ]"pha >> >> Thanks for considerations, >> >> Dario >> >> P.S. In the same environment one hold Pharo 7.0-64 bit ( development version ) image ran well with launch command >

Sept. 26, 2018

GTDocument how to

by Hilaire

Hi, I would like a document where user types in normal text and plugs in DrGeo view. The drgeo view will be dual. It will be alternatively the output of a DrGeo script (drgeo canvas) or the DrGeo script itself (code). The view will be interactive. In drgeo canvas mode, user plays with the geometric construction, when switched to code mode, the user edits the DrGeo script and recompute the DrGeo canvas output. Is GTDocument appropriate to do this? If so, where to start? Thanks Hilaire -- Dr. Geo http://drgeo.eu

Sept. 26, 2018

Re: [Pharo-users] Pharo launcher

by Christophe Demarey

Hi Dario, The problem is probably fixed in the latest PharoLauncher version (1.4) available from http://pharo.org/download. If not, as specified in the dialog text, please run the given command from the command line to give us more information on what is the problem. Regards, Christophe > Le 26 sept. 2018 Ã 12:05, Trussardi Dario Romano <dario.trussardi(a)tiscali.it> a Ã©crit : > > Ciao, > > on Ubuntu system with Pharo Launcher 1.3 > > after create a new Pharo 7.0-64 bit ( development version ) > > when i launch the images the system answer : > > Cannot determine image version: image version file not found! > Try to run this command in a shell to get more information on the problem: > cd "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836" && export SQUEAK_PLUGINS="" && "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836/pharo" --nodisplay "/home/party/Pharo/images/Pharo 7.0 - 64bit (180926)-01/Pharo 7.0 - 64bit (180926)-01.image" eval "(Smalltalk imagePath asFileReference parent) / 'pharo.version' writeStreamDo: [ :stream | |major minor| major := SystemVersion current major. minor := SystemVersion current minor. (major = 6 and: [ SystemVersion current highestUpdate >= 60509 ]) ifTrue: [ minor := 1 ]. stream << major asString; << minor asString ]"pha > > Thanks for considerations, > > Dario > > P.S. In the same environment one hold Pharo 7.0-64 bit ( development version ) image ran well with launch command

Sept. 26, 2018

IoT hackathon @zweidenker

by Norbert Hartl

Hi, we (ZWEIDENKER) and RModD organized an IoT Hackathon in cologne at 19th of october 2018. The plan will be: - Allex Oliviera will present the tutorial he made on PharoThings [1]. The audience will try it in a hands-on-session and give feedback to improve it. Zweidenker is sponsoring the hardware so nobody has to bring anything to the session - We are collecting ideas at the moment what we want to do with the learnings from Allex, meaning what hardware to build - If you have ideas about what to build with real life things that should be connected from pharo, please tell us. We will compile a list of ideas and for the first two or three we will buy the necessary hardware so we can build it at that day The Hackathon will take place at ZWEIDENKER GmbH Luxemburger Str. 72 50674 KÃ¶ln germany As we have a limited amount of seats you need to register if you are interested. Go to http://zweidenker.de/en/iot-hackathon-2018 <http://zweidenker.de/en/iot-hackathon-2018> to register your seat on the session. If there are more participants than seats we will have to do selection by dice rolling. Registration period ends on 5th of october so you will know in advance if you can come. See you there, Norbert on behalf of ZWEIDENKER [1] https://github.com/pharo-iot/PharoThings <https://github.com/pharo-iot/PharoThings>

Sept. 26, 2018

Pharo launcher

by Trussardi Dario Romano

Ciao, on Ubuntu system with Pharo Launcher 1.3 after create a new Pharo 7.0-64 bit ( development version ) when i launch the images the system answer : Cannot determine image version: image version file not found! Try to run this command in a shell to get more information on the problem: cd "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836" && export SQUEAK_PLUGINS="" && "/opt/pharolauncher/pharo-vm/lib/pharo/5.0-201805090836/pharo" --nodisplay "/home/party/Pharo/images/Pharo 7.0 - 64bit (180926)-01/Pharo 7.0 - 64bit (180926)-01.image" eval "(Smalltalk imagePath asFileReference parent) / 'pharo.version' writeStreamDo: [ :stream | |major minor| major := SystemVersion current major. minor := SystemVersion current minor. (major = 6 and: [ SystemVersion current highestUpdate >= 60509 ]) ifTrue: [ minor := 1 ]. stream << major asString; << minor asString ]"pha Thanks for considerations, Dario P.S. In the same environment one hold Pharo 7.0-64 bit ( development version ) image ran well with launch command

Sept. 26, 2018

Re: [Pharo-users] [Pharo-dev] [ANN] Success story Mobility Map

by Marcus Denker

> On 25 Sep 2018, at 20:05, brackendev <bracken.spencer(a)gmail.com> wrote: > > Yes, this is impressive. > > I agree that any technical details and architectural insight (no matter how > high-level) would be useful. > > Related: There's enough in http://pharo.org/success that it could be > sectioned by tags (industries, technologies, etc.). There's a lot there that > could really be showcased. > Yes, we have a lot of TODOs for the website that should have been done long time ago and even more ideasâ¦ one of the things we need to see how to organize better. Marcus

Sept. 26, 2018

Re: [Pharo-users] [Pharo-dev] [ANN] Success story Mobility Map

by brackendev

Yes, this is impressive. I agree that any technical details and architectural insight (no matter how high-level) would be useful. Related: There's enough in http://pharo.org/success that it could be sectioned by tags (industries, technologies, etc.). There's a lot there that could really be showcased. Sven Van Caekenberghe-2 wrote > Wow. Very nice, well done. > > Any chance on some more technical details, as in what 'connected by a > message queue for the communication' exactly means ? How did you approach > micro services exactly ? > >> On 25 Sep 2018, at 12:20, Norbert Hartl < > norbert@ > > wrote: >> >> As presented on ESUG here is the brief description of one of our current >> projects. >> >> Mobility Map >> ââââââ >> >> Mobility Map is a broker for mobility services. It offers multi-modal >> routing search enabling users to find the best travel options between >> locations. Travel options include car sharing, bikes, trains, busses etc. >> Rented cars can be offered for ride sharing on booking time letting other >> people find it to participate in the ride. Single travel options are >> combined in travel plans that can be booked and managed in a very easy >> way. >> >> For this project main requirements were scalability to serve a large user >> base and flexibility to add more additional providers to the broker. The >> application has been realized using web technologies for the frontend and >> pharo for the backend. Using a microservice architecture combined with a >> broker it is easy to extend the platform with additional brokers. >> Deployment is done using docker swarm for distributing dozens of pharo >> images among multiple server machines connected by a message queue for >> the communication. Pharo supported that scenario very well enabling us >> the meet the requirements with less effort. >> >> Pharo turned out to be a perfect fit to develop the application in a >> agile way. Small development cycles with continuous integration and >> continuous delivery enables fast turnarounds for the customers to >> validate progress. >> >> This is a screenshot of the search page for multi-modal results: >> >> >> > <Screen Shot 2018-09-21 at 16.54.30.png> -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Sept. 25, 2018

Unable to install Grafoscopio on Windows (libgit2 issues?)

by Offray Vladimir Luna CÃ¡rdenas

Hi, I'm making a workshop in Visualizar2018, a data visualization event in Madrid, that dedicates this edition to personal data (in my case, the twitter data selfies), but I'm having problems trying to install Grafoscopio on Windos (Mac and Linux work), which gives a not good first impression for participants. In Linux the problem was solved by installing libcurl-gnutils and in Mac this problem doesn't present at all. We tried to install curl and libcurl.dll on Windows and still we are getting the same error message (just says that is unable to install it and nothing else). It happens while trying to install Roassal and seems related with libgit2. How can we install libgit2, so software packages that needs them (seems Roassal2 does) can be installed properly on Windows? Thanks, Offray

Sept. 25, 2018

Re: [Pharo-users] [Pharo-dev] [ANN] Success story Mobility Map

by Sven Van Caekenberghe

> On 25 Sep 2018, at 14:39, Norbert Hartl <norbert(a)hartl.name> wrote: > > > >> Am 25.09.2018 um 12:52 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >> >> Wow. Very nice, well done. >> >> Any chance on some more technical details, as in what 'connected by a message queue for the communication' exactly means ? How did you approach micro services exactly ? >> > Sure :) Thanks, this is very interesting ! > The installation spawns multiple physical machines. All the machines are joined to a docker swarm. The installation is reified as either task or service from the view on the docker swarm. Meaning you instantiate an arbitrary amount of services and docker swarm distributes them among the physical machines. Usually you donât take control which is running where but you can. At this point you have spread dozens of pharo images among multiple machines and each of them has an IP address. Furthermore in docker swarm you have a reification of a network meaning that every instance in a network can see all other instances on this network. Each service can be reached by its service name in that network. Docker swarm does all the iptables/firewall and DNS setup for you. Are you happy with docker swarm's availability/fail-over behaviour ? In other words: does it work when one image/instance goes bad, does it detect and restore the missing functionality ? > In order to have communication between those runtimes we use rabbitmq because you were so nice writing a driver for it ;) The rabbitmq does have a support for cluster setup, meaning each of the physical machines has a rabbitmq installation and they know each other. So it does not matter to which instance you send messages to and on which you register for receiving messages. So every pharo image connects to the service rabbitmq and opens a queue for interaction. Same question: does RabbitMQ's clustering work well under stress/problems ? Syncing all queues between all machines sounds quite heavy (I never tried it, but maybe it just works). > Each service like the car sharing opens a queue e.g. /queue/carSharing and listens on it. The broker images are stateful so they open queues like /queue/mobility-map-afdeg32 where afdeg32 is the container id of the instance (hostname in docker). In each request the queue name to reply is sent as a header. So we can make sure that the right image gets the message back. This way we can have sticky sessions keeping volatile data in memory for the lifecycle of a session. There is one worker image which opens a queue /queue/mobility-map where session independent requests can be processed. I think I understand ;-) > In order to ease development we are sharing code between the broker and the micro service. Each micro service has a -Common package where the classes are in that build the interface. The classes in here are a kind of data entity facades. They use NeoJSON to map to and from a stream. The class name is send with the message as a header so the remote side knows what to materialize. The handling is unified for the four cases > > - Request as inquiry to another micro service > - Response returns values to a Request > - Error is transferred like a Response but is then signalled on the receiving side > - Notification connects the announcers on the broker and the micro service side. Yes, makes total sense. > Asynchronous calls we solved using Promises and Futures. Each async call to the Q becomes a promise (that blocks on #value) and is combined to a future value containing all promises with support to generate a delta of all resolved promises. This we need because you issue a search that takes longer and you want to display results as soon as they are resolved not after all haven been resolved. Which Promise/Future framework/library are you using in Pharo ? You did not go for single threaded worker images ? > And a lot more. This is a coarse grained overview over the architecture. Iâm happy to answer further questions about this. > > Norbert > >>> On 25 Sep 2018, at 12:20, Norbert Hartl <norbert(a)hartl.name> wrote: >>> >>> As presented on ESUG here is the brief description of one of our current projects. >>> >>> Mobility Map >>> ââââââ >>> >>> Mobility Map is a broker for mobility services. It offers multi-modal routing search enabling users to find the best travel options between locations. Travel options include car sharing, bikes, trains, busses etc. Rented cars can be offered for ride sharing on booking time letting other people find it to participate in the ride. Single travel options are combined in travel plans that can be booked and managed in a very easy way. >>> >>> For this project main requirements were scalability to serve a large user base and flexibility to add more additional providers to the broker. The application has been realized using web technologies for the frontend and pharo for the backend. Using a microservice architecture combined with a broker it is easy to extend the platform with additional brokers. Deployment is done using docker swarm for distributing dozens of pharo images among multiple server machines connected by a message queue for the communication. Pharo supported that scenario very well enabling us the meet the requirements with less effort. >>> >>> Pharo turned out to be a perfect fit to develop the application in a agile way. Small development cycles with continuous integration and continuous delivery enables fast turnarounds for the customers to validate progress. >>> >>> This is a screenshot of the search page for multi-modal results: >>> >>> >>> <Screen Shot 2018-09-21 at 16.54.30.png> >> >> > >

Sept. 25, 2018