pharo-iot@lists.pharo.org

Pharo IoT Developers and Hardcore users

View all threads

Firmata callbacks

RV
Rob van Lopik <robvanlopik@gmail.com>
Thu, Jun 4, 2020 8:39 PM

Hi all,

During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move.

At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages. By the way, in WiringPi this callback facility also is missing (wiringPiISRPin: pinNumber edgeType: edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented).

regards rob van lopik

Hi all, During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move. At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages. By the way, in WiringPi this callback facility also is missing (wiringPiISRPin: pinNumber edgeType: edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented). regards rob van lopik
DK
Denis Kudriashov <dionisiydk@gmail.com>
Sat, Jun 6, 2020 12:15 PM

Hi Rob.

About WiringPi callbacks. I tried to use OS interrupts to be notified about pin changes (do not remember what was the wiringPi function). But it just proved that FFI callbacks do not work from non VM process. With Pharo 9 we will have threaded FFI and it will be finely possible.

чт, 4 июн. 2020 г. в 21:40, Rob van Lopik robvanlopik@gmail.com:

Hi all,

During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move.

At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages. By the way, in WiringPi this callback facility also is missing (wiringPiISRPin: pinNumber edgeType: edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented).

regards rob van lopik

Hi Rob. About WiringPi callbacks. I tried to use OS interrupts to be notified about pin changes (do not remember what was the wiringPi function). But it just proved that FFI callbacks do not work from non VM process. With Pharo 9 we will have threaded FFI and it will be finely possible. чт, 4 июн. 2020 г. в 21:40, Rob van Lopik <robvanlopik@gmail.com>: > Hi all, > > During the last couple of weeks I have worked on implementing most of the > sysex-based messages. This was great fun, because it greatly facilitates > interaction between Pharo on any platform and the real world with a variety > of devices. Now I am at the point where I have to address the fundamentally > asynchronous nature of the Firmata protocol. At present we store the > answers from requests in instance variables, but we don't know when these > are available. Sometimes the protocol allows for continuous updates, so > reading the data is semi-realtime. This happens with analog inputs, but can > also be used with i2c devices or encoders. Driving a stepper motor really > needs a callback or notification mechanism. When you instruct the stepper > to do a number of steps you must be notified when it is ready for the next > move. > > At this point I want your input on what is the "Pharoic" way of doing > this. I can think of announcements or using Taskit, or simply passing a > block to be executed when the answer becomes available. Maybe someone can > point me to examples in existing packages. By the way, in WiringPi this > callback facility also is missing (wiringPiISRPin: pinNumber edgeType: > edgeType function: aFunctionPointerOrCallback >> shouldBeImplemented). > > regards > rob van lopik > > >
DK
Denis Kudriashov <dionisiydk@gmail.com>
Sat, Jun 6, 2020 12:58 PM

чт, 4 июн. 2020 г. в 21:40, Rob van Lopik robvanlopik@gmail.com:

Hi all,

During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move.

At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages.

The code example would be helpful to think about it.

чт, 4 июн. 2020 г. в 21:40, Rob van Lopik <robvanlopik@gmail.com>: > Hi all, > > During the last couple of weeks I have worked on implementing most of the > sysex-based messages. This was great fun, because it greatly facilitates > interaction between Pharo on any platform and the real world with a variety > of devices. Now I am at the point where I have to address the fundamentally > asynchronous nature of the Firmata protocol. At present we store the > answers from requests in instance variables, but we don't know when these > are available. Sometimes the protocol allows for continuous updates, so > reading the data is semi-realtime. This happens with analog inputs, but can > also be used with i2c devices or encoders. Driving a stepper motor really > needs a callback or notification mechanism. When you instruct the stepper > to do a number of steps you must be notified when it is ready for the next > move. > > At this point I want your input on what is the "Pharoic" way of doing > this. I can think of announcements or using Taskit, or simply passing a > block to be executed when the answer becomes available. Maybe someone can > point me to examples in existing packages. > The code example would be helpful to think about it.
RV
Rob van Lopik <robvanlopik@gmail.com>
Tue, Jun 9, 2020 8:25 AM

On Sat, Jun 6, 2020 at 1:59 PM Denis Kudriashov dionisiydk@gmail.com wrote:

чт, 4 июн. 2020 г. в 21:40, Rob van Lopik robvanlopik@gmail.com:

Hi all,

During the last couple of weeks I have worked on implementing most of the sysex-based messages. This was great fun, because it greatly facilitates interaction between Pharo on any platform and the real world with a variety of devices. Now I am at the point where I have to address the fundamentally asynchronous nature of the Firmata protocol. At present we store the answers from requests in instance variables, but we don't know when these are available. Sometimes the protocol allows for continuous updates, so reading the data is semi-realtime. This happens with analog inputs, but can also be used with i2c devices or encoders. Driving a stepper motor really needs a callback or notification mechanism. When you instruct the stepper to do a number of steps you must be notified when it is ready for the next move.

At this point I want your input on what is the "Pharoic" way of doing this. I can think of announcements or using Taskit, or simply passing a block to be executed when the answer becomes available. Maybe someone can point me to examples in existing packages.

The code example would be helpful to think about it.

Hi Denis, I don't have a specific code example, because that depends on the design of the eventual application. But we can discuss use cases. The Firmata sketch has a sampling interval that can be set from 19 ms upwards. When reporting is enabled for an analog pin or a group of digital pins, their value is reported after each sample interval. The latest value is stored in an instance variable and can be read when convenient; it is never more than one sample-interval old. Inside the Firmata driver I can implement code that detects when the value changes, or exceeds a given limit. If that is the only thing I am interested in, the driver should notify me of such an occurrence, without the program having to continuously read the data. This is use case 1. Probably to be solved with an Announcer (see later for some implementation questions).

Firmata has more input modes:

  • I2C
  • onewire
  • encoder These have a sysex command to initiate a reading, but the answer arrives asynchronously. I store the answer in an instance variable (of the Firmata instance) and the user code has to wait some time (probably the sample interval time) before reading. When we issue the read command we could mark these data as "stale". On the other hand, once again, we could announce the presence of valid data. The other mode of operation is to instruct the firmata sketch to continuously (at the sample rate) update the values. This is equivalent to the above mode for analog and digital inputs, so we can freely read them when convenient.

The stepper motor control is a separate case. In principle you order the stepper to do a number of steps at a certain speed and possible with specified acceleration and deceleration.But you can't continue sending commands. Firmata reports back when a command is finished and we will have to wait for that notification before sending a new instruction.

You will notice that I already seem to show a preference for the Announcer framework. I saw you contributed to a discussion of Announcers in 2016 ( http://forum.world.st/About-the-non-use-of-Announcer-in-Bloc-td4913008.html) so I hope you can answer the following question: Looking at the code it seems that in #when: Announcement do: aBlock the block will be executed in the thread that raised the announcement. Now, the way Firmata is set up this has to occur in the processInput thread. As this thread has to keep up with the incoming data from the Firmata sketch, I think we should minimize the amount of processing in that thread. (I even think we should consider raising its priority a bit). Is this reasoning valid? And would it be solved by only using #when:send:to: ? The other solution (that I also see e.g. in Pymata) is to do the processing of sysex messages in a different thread. Or create a new process/task for each message. It is said that process creation is not too expensive, and otherwise I could use taskit with a task pool.

By the way: in the meantime I have all of these devices basically working. Both on a Uno and a Micro. And with sketches generated with Configurable Firmata. The I2C mode also works with the PCA9685 and the DS1307 for which I previously wrote drivers for WiringPi.

All comments welcome; I will put some more questions on the present Firmata in a separate thread.

cheers Rob van Lopik

On Sat, Jun 6, 2020 at 1:59 PM Denis Kudriashov <dionisiydk@gmail.com> wrote: > чт, 4 июн. 2020 г. в 21:40, Rob van Lopik <robvanlopik@gmail.com>: > >> Hi all, >> >> During the last couple of weeks I have worked on implementing most of the >> sysex-based messages. This was great fun, because it greatly facilitates >> interaction between Pharo on any platform and the real world with a variety >> of devices. Now I am at the point where I have to address the fundamentally >> asynchronous nature of the Firmata protocol. At present we store the >> answers from requests in instance variables, but we don't know when these >> are available. Sometimes the protocol allows for continuous updates, so >> reading the data is semi-realtime. This happens with analog inputs, but can >> also be used with i2c devices or encoders. Driving a stepper motor really >> needs a callback or notification mechanism. When you instruct the stepper >> to do a number of steps you must be notified when it is ready for the next >> move. >> >> At this point I want your input on what is the "Pharoic" way of doing >> this. I can think of announcements or using Taskit, or simply passing a >> block to be executed when the answer becomes available. Maybe someone can >> point me to examples in existing packages. >> > > The code example would be helpful to think about it. > Hi Denis, I don't have a specific code example, because that depends on the design of the eventual application. But we can discuss use cases. The Firmata sketch has a sampling interval that can be set from 19 ms upwards. When reporting is enabled for an analog pin or a group of digital pins, their value is reported after each sample interval. The latest value is stored in an instance variable and can be read when convenient; it is never more than one sample-interval old. Inside the Firmata driver I can implement code that detects when the value changes, or exceeds a given limit. If that is the only thing I am interested in, the driver should notify me of such an occurrence, without the program having to continuously read the data. This is use case 1. Probably to be solved with an Announcer (see later for some implementation questions). Firmata has more input modes: - I2C - onewire - encoder These have a sysex command to initiate a reading, but the answer arrives asynchronously. I store the answer in an instance variable (of the Firmata instance) and the user code has to wait some time (probably the sample interval time) before reading. When we issue the read command we could mark these data as "stale". On the other hand, once again, we could announce the presence of valid data. The other mode of operation is to instruct the firmata sketch to continuously (at the sample rate) update the values. This is equivalent to the above mode for analog and digital inputs, so we can freely read them when convenient. The stepper motor control is a separate case. In principle you order the stepper to do a number of steps at a certain speed and possible with specified acceleration and deceleration.But you can't continue sending commands. Firmata reports back when a command is finished and we will have to wait for that notification before sending a new instruction. You will notice that I already seem to show a preference for the Announcer framework. I saw you contributed to a discussion of Announcers in 2016 ( http://forum.world.st/About-the-non-use-of-Announcer-in-Bloc-td4913008.html) so I hope you can answer the following question: Looking at the code it seems that in #when: Announcement do: aBlock the block will be executed in the thread that raised the announcement. Now, the way Firmata is set up this has to occur in the processInput thread. As this thread has to keep up with the incoming data from the Firmata sketch, I think we should minimize the amount of processing in that thread. (I even think we should consider raising its priority a bit). Is this reasoning valid? And would it be solved by only using #when:send:to: ? The other solution (that I also see e.g. in Pymata) is to do the processing of sysex messages in a different thread. Or create a new process/task for each message. It is said that process creation is not too expensive, and otherwise I could use taskit with a task pool. By the way: in the meantime I have all of these devices basically working. Both on a Uno and a Micro. And with sketches generated with Configurable Firmata. The I2C mode also works with the PCA9685 and the DS1307 for which I previously wrote drivers for WiringPi. All comments welcome; I will put some more questions on the present Firmata in a separate thread. cheers Rob van Lopik