June 2020 - Pharo-iot - lists.pharo.org

Firmata and concurrency

by Rob van Lopik

Hi All, When I first looked at Firmata, almost a year ago, I noticed that access to acquired data was shared between the thread that reads from the Arduino and the user code. This should be an uninterruptible operation. Later I learned that arrays #at: and #at:put: are atomic operations. As far as I can see this is not by design, but because they are primitives. Changing the workings of the VM could also change that. By the way, it is not very evident from any documentation that this is the case. It so happens that these data are in arrays, so it would be safe. But when adding other devices I also have to use some dictionaries, so now I will have to introduce a Mutex to coordinate access to these instance variables. Please let me know if you don't agree with this analysis. The part that sends commands to the Arduino also has concurrency problems. I expect a measurement and control system to be composed of different processes (threads in Pharo), while there can be only one instance of Firmata. So thread safety is important for all writes to the serial port. Full thread safety may not be necessary, as long as different processes access different functions of Firmata, e.g. one thread controls servo motors, another one steppers and a third one pwm output (the latter already is tricky, because both use analog writes). Anyway this is something the user should be aware of and take care of. But at least inside Firmata sending a sysex string to the serial port should be protected. I don't know whether the actual primitive operation (serialPort >> primWritePort: portNumber from: byteArray startingAt: startIndex count: count) is atomic. But serialPort>>netPutAll: contains (very little) Pharo code. I guess that it is sufficient when each method does only one "port nextPutAll:", so the byte array to be sent has to be prepared in advance. Separate sends of startsysex and endsysex should not be used. The same holds for all non-sysex sends. If not, I will have to protect all writes to the serial port with a mutex. Thirty years ago I once taught an "introduction to real time programming" for agricultural researchers. The main thing I wanted to convey was to be aware of possible problems. That attitude has remained with me, after all those years. But I thought I could better check with people like you, who have more actual/recent experience. The last realtime things I have done were with Arduino code. There a short critical section is easily accomplished by turning off all interrupts. cheers and thanks for any comments. rob van lopik

June 9, 2020

Re: [Pharo-iot] Firmata callbacks

by Rob van Lopik

On Sat, Jun 6, 2020 at 1:59 PM Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > ÑÑ, 4 Ð¸ÑÐ½. 2020 Ð³. Ð² 21:40, Rob van Lopik <robvanlopik(a)gmail.com>: > >> Hi all, >> >> During the last couple of weeks I have worked on implementing most of the >> sysex-based messages. This was great fun, because it greatly facilitates >> interaction between Pharo on any platform and the real world with a variety >> of devices. Now I am at the point where I have to address the fundamentally >> asynchronous nature of the Firmata protocol. At present we store the >> answers from requests in instance variables, but we don't know when these >> are available. Sometimes the protocol allows for continuous updates, so >> reading the data is semi-realtime. This happens with analog inputs, but can >> also be used with i2c devices or encoders. Driving a stepper motor really >> needs a callback or notification mechanism. When you instruct the stepper >> to do a number of steps you must be notified when it is ready for the next >> move. >> >> At this point I want your input on what is the "Pharoic" way of doing >> this. I can think of announcements or using Taskit, or simply passing a >> block to be executed when the answer becomes available. Maybe someone can >> point me to examples in existing packages. >> > > The code example would be helpful to think about it. > Hi Denis, I don't have a specific code example, because that depends on the design of the eventual application. But we can discuss use cases. The Firmata sketch has a sampling interval that can be set from 19 ms upwards. When reporting is enabled for an analog pin or a group of digital pins, their value is reported after each sample interval. The latest value is stored in an instance variable and can be read when convenient; it is never more than one sample-interval old. Inside the Firmata driver I can implement code that detects when the value changes, or exceeds a given limit. If that is the only thing I am interested in, the driver should notify me of such an occurrence, without the program having to continuously read the data. This is use case 1. Probably to be solved with an Announcer (see later for some implementation questions). Firmata has more input modes: - I2C - onewire - encoder These have a sysex command to initiate a reading, but the answer arrives asynchronously. I store the answer in an instance variable (of the Firmata instance) and the user code has to wait some time (probably the sample interval time) before reading. When we issue the read command we could mark these data as "stale". On the other hand, once again, we could announce the presence of valid data. The other mode of operation is to instruct the firmata sketch to continuously (at the sample rate) update the values. This is equivalent to the above mode for analog and digital inputs, so we can freely read them when convenient. The stepper motor control is a separate case. In principle you order the stepper to do a number of steps at a certain speed and possible with specified acceleration and deceleration.But you can't continue sending commands. Firmata reports back when a command is finished and we will have to wait for that notification before sending a new instruction. You will notice that I already seem to show a preference for the Announcer framework. I saw you contributed to a discussion of Announcers in 2016 ( http://forum.world.st/About-the-non-use-of-Announcer-in-Bloc-td4913008.html) so I hope you can answer the following question: Looking at the code it seems that in #when: Announcement do: aBlock the block will be executed in the thread that raised the announcement. Now, the way Firmata is set up this has to occur in the processInput thread. As this thread has to keep up with the incoming data from the Firmata sketch, I think we should minimize the amount of processing in that thread. (I even think we should consider raising its priority a bit). Is this reasoning valid? And would it be solved by only using #when:send:to: ? The other solution (that I also see e.g. in Pymata) is to do the processing of sysex messages in a different thread. Or create a new process/task for each message. It is said that process creation is not too expensive, and otherwise I could use taskit with a task pool. By the way: in the meantime I have all of these devices basically working. Both on a Uno and a Micro. And with sketches generated with Configurable Firmata. The I2C mode also works with the PCA9685 and the DS1307 for which I previously wrote drivers for WiringPi. All comments welcome; I will put some more questions on the present Firmata in a separate thread. cheers Rob van Lopik

June 9, 2020

Re: [Pharo-iot] Firmata callbacks

by Denis Kudriashov

ÑÑ, 4 Ð¸ÑÐ½. 2020 Ð³. Ð² 21:40, Rob van Lopik <robvanlopik(a)gmail.com>: > Hi all, > > During the last couple of weeks I have worked on implementing most of the > sysex-based messages. This was great fun, because it greatly facilitates > interaction between Pharo on any platform and the real world with a variety > of devices. Now I am at the point where I have to address the fundamentally > asynchronous nature of the Firmata protocol. At present we store the > answers from requests in instance variables, but we don't know when these > are available. Sometimes the protocol allows for continuous updates, so > reading the data is semi-realtime. This happens with analog inputs, but can > also be used with i2c devices or encoders. Driving a stepper motor really > needs a callback or notification mechanism. When you instruct the stepper > to do a number of steps you must be notified when it is ready for the next > move. > > At this point I want your input on what is the "Pharoic" way of doing > this. I can think of announcements or using Taskit, or simply passing a > block to be executed when the answer becomes available. Maybe someone can > point me to examples in existing packages. > The code example would be helpful to think about it.

June 6, 2020

Re: [Pharo-iot] Firmata callbacks

by Denis Kudriashov

Hi Rob. About WiringPi callbacks. I tried to use OS interrupts to be notified about pin changes (do not remember what was the wiringPi function). But it just proved that FFI callbacks do not work from non VM process. With Pharo 9 we will have threaded FFI and it will be finely possible. ÑÑ, 4 Ð¸ÑÐ½. 2020 Ð³. Ð² 21:40, Rob van Lopik <robvanlopik(a)gmail.com>: > Hi all, > > During the last couple of weeks I have worked on implementing most of the > sysex-based messages. This was great fun, because it greatly facilitates > interaction between Pharo on any platform and the real world with a variety > of devices. Now I am at the point where I have to address the fundamentally > asynchronous nature of the Firmata protocol. At present we store the > answers from requests in instance variables, but we don't know when these > are available. Sometimes the protocol allows for continuous updates, so > reading the data is semi-realtime. This happens with analog inputs, but can > also be used with i2c devices or encoders. Driving a stepper motor really > needs a callback or notification mechanism. When you instruct the stepper > to do a number of steps you must be notified when it is ready for the next > move. > > At this point I want your input on what is the "Pharoic" way of doing > this. I can think of announcements or using Taskit, or simply passing a > block to be executed when the answer becomes available. Maybe someone can > point me to examples in existing packages. By the way, in WiringPi this > callback facility also is missing (wiringPiISRPin: pinNumber edgeType: > edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented). > > regards > rob van lopik > > >

June 6, 2020

Firmata callbacks

by Rob van Lopik

Hi all, During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move. At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages. By the way, in WiringPi this callback facility also is missing (wiringPiISRPin: pinNumber edgeType: edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented). regards rob van lopik

June 4, 2020

Re: [Pharo-iot] Telepharo and PharoThings on Pharo Launcher

by Marcus Denker

> On 28 May 2020, at 23:22, Allex Oliveira <allex.oliveira(a)msn.com> wrote: > > Hi everyone, > > I was syncing with Christophe how we can put TelePharo and PharoThings on Pharo Launcher, and I created a tool to generate the images of any repository in each commit using Travis CI and release the images on Github. So now it's easy to do that. > > But there are some questions: > > "Then, you should think under which category we should put Telepharo? Official templates? A dedicated category? > If it is the official one, we should have max 2 entries for Telepharo I think. > If you want to have all combinations, a dedicated category? > Also one thing that could be annoying is that it is a continuous build, no stable (reproducible) version." > > I think we could have 2 entries in official one (Full Telepharo and PharoThings to Pharo 8 64) > > and to have an IoT category with telepharo and pharothings to pharo 7/8/9 32/64 client and server, minimal image, etc > I would now just put a dedicated Category. Then we can later see if it makes sense to have some in official or not. > About the continuous build, maybe we can use the tag releases to PharoLaucher. But I think we need to think about a better way to see which version is running on the server, to download the same specific version of the client. > > Maybe when the client connects, it realizes some kind of test and shows a popup saying it's the same version or not, and which one is supposed to use... > Yes, that might be an ideaâ¦ an explicit version check. Marcus

June 4, 2020