[Pharo-project] Proposed (or just written down desired) Keyboard Event Model
Following the Event chronicles :P Since I know that some people are working on event Stuff(Stef, Fernando, Igor), I'd like to listen to what they(or anybody else :) ) think about this. I've no code (yet) except for the one I've written to play with the vm and test the keyboard stuff :P. And I know there is some package where Stef and Igor where refactoring the event stuff and I don't want to duplicate efforts either, jeje. So, this is what from other models I think a complete keyboard event model should have: Keyboard interaction raises 3 events (3 different kind of objects!): - Keydown When a key is pressed, it informs the pressed key with modifiers (it should not send a unicode value of a character). Should understand the messages: "boolean indicating which modifiers where pressed" #alt #cmd #shift #ctrl #opt? "a collection of modifier objects" #modifiers "the key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key "to know if the event wasHandled" #handled: #handled - Keypress When a key with a unicode representation is pressed, informs it. should understand the messages: "the unicode character representation of this event. It is a Character, not an instance of Key" #character "to know if the event wasHandled" #handled: #handled - Keyup When a key is released, it informs the released key with modifiers (it should not send a unicode value of a character). Should understand the messages: "the main key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key "to know if the event wasHandled" #handled: #handled =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= *Glitches* 1) Today every keyboard event handled on the image side is handled on keypress, which: - limits the ammount of shortcuts you can have (because there are keys that you can't handle) - to make it work, there are some hacks (at image and vm side) to allow keys with no char representation enter as keypresses (like arrow keys). - changing this is ALOT of work :P 2) Some key representation varies from platform to platform. i.e.: A Shift-only press generates today a keydown event with: - a 254 keyCode value in unix - a 16 keyCode value in windows So I think a "table" has to be built and documented for mappings for each key in a keyboard. Examples of this can be found in enums/defs from other platforms: Windows platform: http://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).as... .Net: http://msdn.microsoft.com/en-us/library/system.windows.forms.keys(v=vs.71).a... X11 (current used window system in unix vm): http://cgit.freedesktop.org/xorg/proto/x11proto/plain/keysymdef.h Javascript: http://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes Gtk: http://www.koders.com/c/fidD9E5E78FD91FE6ABDD6D3F78DA5E4A0FADE79933.aspx After that, both vm's and image should follow that table :). And of course, these keys should be reified in the image side independently of their character pairs. Guille
On Jan 26, 2012, at 7:07 PM, Guillermo Polito wrote:
Following the Event chronicles :P
Since I know that some people are working on event Stuff(Stef, Fernando, Igor), I'd like to listen to what they(or anybody else :) ) think about this. I've no code (yet) except for the one I've written to play with the vm and test the keyboard stuff :P. And I know there is some package where Stef and Igor where refactoring the event stuff and I don't want to duplicate efforts either, jeje.
have a look at EventModel on squeak source. For now we did not modify the handling of keystrokes. We "just" reified event instead of manipulating eventbuf uglies. I'm sick now so my brain is slow. Two points are - transition from old to new . Is there a path with little impact on users. I guess so. - what about VM changes⦠Stef
So, this is what from other models I think a complete keyboard event model should have:
Keyboard interaction raises 3 events (3 different kind of objects!):
⢠Keydown When a key is pressed, it informs the pressed key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"boolean indicating which modifiers where pressed" #alt #cmd #shift #ctrl #opt?
"a collection of modifier objects" #modifiers
"the key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
⢠Keypress When a key with a unicode representation is pressed, informs it.
should understand the messages:
"the unicode character representation of this event. It is a Character, not an instance of Key" #character
"to know if the event wasHandled" #handled: #handled
⢠Keyup When a key is released, it informs the released key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"the main key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Glitches
1) Today every keyboard event handled on the image side is handled on keypress, which: - limits the ammount of shortcuts you can have (because there are keys that you can't handle) - to make it work, there are some hacks (at image and vm side) to allow keys with no char representation enter as keypresses (like arrow keys). - changing this is ALOT of work :P
2) Some key representation varies from platform to platform. i.e.: A Shift-only press generates today a keydown event with: - a 254 keyCode value in unix - a 16 keyCode value in windows
So I think a "table" has to be built and documented for mappings for each key in a keyboard. Examples of this can be found in enums/defs from other platforms:
Windows platform: http://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).as... .Net: http://msdn.microsoft.com/en-us/library/system.windows.forms.keys(v=vs.71).a... X11 (current used window system in unix vm): http://cgit.freedesktop.org/xorg/proto/x11proto/plain/keysymdef.h Javascript: http://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes Gtk: http://www.koders.com/c/fidD9E5E78FD91FE6ABDD6D3F78DA5E4A0FADE79933.aspx
After that, both vm's and image should follow that table :). And of course, these keys should be reified in the image side independently of their character pairs.
Guille
On Fri, Jan 27, 2012 at 5:23 AM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
On Jan 26, 2012, at 7:07 PM, Guillermo Polito wrote:
Following the Event chronicles :P
Since I know that some people are working on event Stuff(Stef, Fernando, Igor), I'd like to listen to what they(or anybody else :) ) think about this. I've no code (yet) except for the one I've written to play with the vm and test the keyboard stuff :P. And I know there is some package where Stef and Igor where refactoring the event stuff and I don't want to duplicate efforts either, jeje.
have a look at EventModel on squeak source. For now we did not modify the handling of keystrokes. We "just" reified event instead of manipulating eventbuf uglies.
;) I based my code on that.
I'm sick now so my brain is slow.
get better soon! :)
Two points are - transition from old to new . Is there a path with little impact on users. I guess so.
I've written a path in a previous email :).
- what about VM changesâ¦
I can learn and work to modify windows and linux (on x11) vms... :) Guille
*"Event handling, and the philosopher's keyboard"* I've downloaded Stef and Igor's EventModel to play with their new event model :). I've created KeydownEvent and KeyupEvent in contrast to the generic KeyboardEvent with all the information. It's very simple code, but, that is the api I'm talking about. Gofer it squeaksource: 'GuilleUtils'; package: 'EventModel'; load | ann evt | ann := SystemInputEventAnnouncer new. ann subscribe: SystemInputEvent send: #dispatchEvent: to: World activeHand. 10 timesRepeat: [ 500 milliSeconds asDelay wait. evt := Sensor nextEvent. evt ifNotNil: [ ann handleEvent: evt ]. ]. I don't know what other people happen to think about it :/, but a simple path to include this stuff in the image without breaking everything should be: 1) since keyup and keydown events are not used in any morph in the core, adding the new keydown and keyup should not break anything. 2) fetch a #keyStroke: implementor 3) if it's handling shortcuts instead of only chars (like cmd+smthg, ctrl+smthng), create an issue "put my shortcut handling in #keyDown: please" 4) solve the issue 5) if (#keyStrokes: toRefactor) > 0 : goto 2 After all that, the keyStroke users should only be using keyCharacter, and the refactor to use a simplified KeystrokeEvent is pretty straight forward :). On Thu, Jan 26, 2012 at 3:07 PM, Guillermo Polito <guillermopolito@gmail.com
wrote:
2) Some key representation varies from platform to platform. i.e.: A Shift-only press generates today a keydown event with: - a 254 keyCode value in unix - a 16 keyCode value in windows
So I think a "table" has to be built and documented for mappings for each key in a keyboard. Examples of this can be found in enums/defs from other platforms:
X11 (current used window system in unix vm): http://cgit.freedesktop.org/xorg/proto/x11proto/plain/keysymdef.h
Andreas Raab suggested here to follow the X11 table, since it seems to be the more complete one :): http://forum.world.st/Windows-cog-vm-Keyboard-events-related-about-keycode-m... Guille
On Jan 27, 2012, at 1:39 30PM, Guillermo Polito wrote:
"Event handling, and the philosopher's keyboard"
I've downloaded Stef and Igor's EventModel to play with their new event model :). I've created KeydownEvent and KeyupEvent in contrast to the generic KeyboardEvent with all the information. It's very simple code, but, that is the api I'm talking about.
Gofer it squeaksource: 'GuilleUtils'; package: 'EventModel'; load
| ann evt | ann := SystemInputEventAnnouncer new. ann subscribe: SystemInputEvent send: #dispatchEvent: to: World activeHand.
10 timesRepeat: [ 500 milliSeconds asDelay wait. evt := Sensor nextEvent. evt ifNotNil: [ ann handleEvent: evt ]. ].
My 2c: To me, the events just look like thin wrappers around the buffers where you can get values by name rather than by index⦠You need to do the deconstruction of buffers to objects _somewhere_, personally I that place should be before they are handed to specific consumers (like Morphic). I mean, the types are defined by the VM, and not really up for individual interpretation by different frameworks... For SystemKeyboardEvent, that would mean: - Create subclasses of SystemKeyboardEvent, one each for KeyPressed/KeyDown/KeyUp. - Change SystemInputEvent to do newFromBuffer: instead of new initFromBuffer: - Change SystemKeyboardInputEvent to instantiate the correct subclass. Same applies for f.ex. SystemWindowInputEvent, where a consumer might look like: SystemInputAnnouncer subscribe: WindowMetricChanged do: [:ann| ...], and not SystemInputAnnouncer subscribe: SystemInputAnnouncer: SystemWindowInputEvent do: [ann | ann action = WindowEventMetricChange ifTrue: [â¦]] Will there be duplication between Morphic and SystemEvent announcer in distinction of events? Yes, until Morphic is rewritten to use the announcements directly, instead of creating its own representation of the same classes. But removing that duplication (on the Morphic side of things) is a very natural next step... Cheers, Henry
On 27 January 2012 14:25, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Jan 27, 2012, at 1:39 30PM, Guillermo Polito wrote:
"Event handling, and the philosopher's keyboard"
I've downloaded Stef and Igor's EventModel to play with their new event model :). I've created KeydownEvent and KeyupEvent in contrast to the generic KeyboardEvent with all the information. It's very simple code, but, that is the api I'm talking about.
Gofer it       squeaksource: 'GuilleUtils';       package: 'EventModel';       load
| ann evt | ann := SystemInputEventAnnouncer new. ann subscribe: SystemInputEvent send: #dispatchEvent: to: World activeHand.
10 timesRepeat: [     500 milliSeconds asDelay wait.     evt := Sensor nextEvent.     evt ifNotNil: [         ann handleEvent: evt     ]. ].
My 2c:
To me, the events just look like thin wrappers around the buffers where you can get values by name rather than by index⦠You need to do the deconstruction of buffers to objects _somewhere_, personally I that place should be before they are handed to specific consumers (like Morphic). I mean, the types are defined by the VM, and not really up for individual interpretation by different frameworks...
Indeed. This is what we done for SystemXYZEvents. There can be various ways how you can represent "events" in your framework, but it doesn't changing the interface between VM and language side, and those thin wrappers serve as a clean and simple layer. Simply because we don't want a smelly code to spread around in image, which deals with bits and bytes generated by VM.
For SystemKeyboardEvent, that would mean:
- Create subclasses of SystemKeyboardEvent, one each for KeyPressed/KeyDown/KeyUp. - Change SystemInputEvent to do newFromBuffer: instead of new initFromBuffer: - Change SystemKeyboardInputEvent to instantiate the correct subclass.
Same applies for f.ex. SystemWindowInputEvent, where a consumer might look like:
SystemInputAnnouncer subscribe: WindowMetricChanged do: [:ann| ...], and not SystemInputAnnouncer subscribe: SystemInputAnnouncer: SystemWindowInputEvent do: [ann | ann action = WindowEventMetricChange ifTrue: [â¦]]
Will there be duplication between Morphic and SystemEvent announcer in distinction of events? Yes, until Morphic is rewritten to use the announcements directly, instead of creating its own representation of the same classes. But removing that duplication (on the Morphic side of things) is a very natural next step...
Morphic can wrap system events (so you can always access the original in case if you really want them), or convert them into own. For instance , there is no "double-click" events generated by VM, while morphic having them. There can be other examples, like mouse gestures.. so morphic events are operating at completely different level than basic system events and while for some of them there can be straight correspondence between them, for some of them, obviously not . -- Best regards, Igor Stasenko.
On 26 January 2012 19:07, Guillermo Polito <guillermopolito@gmail.com> wrote:
Following the Event chronicles :P
Since I know that some people are working on event Stuff(Stef, Fernando, Igor), I'd like to listen to what they(or anybody else :) ) think about this. I've no code (yet) except for the one I've written to play with the vm and test the keyboard stuff :P. Â And I know there is some package where Stef and Igor where refactoring the event stuff and I don't want to duplicate efforts either, jeje.
see sqs/EventModel
So, this is what from other models I think a complete keyboard event model should have:
Keyboard interaction raises 3 events (3 different kind of objects!):
Keydown
When a key is pressed, it informs the pressed key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"boolean indicating which modifiers where pressed" #alt #cmd #shift #ctrl #opt?
<RANT> why we can't just send a key code to image and be done with it? what if i want , in my image, for some weird reason, to use 'A' key as a 'modifier'? why at all, we need to distinguish between normal keys from 'modifier' keys, when VM and OSes are capable to generate a same events for all keys user punching? IMO, those 'modifiers' are serving for nothing, but just unnecessary increase of complexity in VM , and data structures (events) we exchanging with VM. There are 3 bits for modifiers namely , shift, alt, cmd .. so i cannot distinguish between left and right shift pressed (cool! awesome! wonderful!). Not mentioning that some laptops having a 'fn' modifier key, but who needs that, right? </RANT>
"a collection of modifier objects" #modifiers
"the key pressed. Â It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
Keypress
When a key with a unicode representation is pressed, informs it.
should understand the messages:
"the unicode character representation of this event. Â It is a Character, not an instance of Key" #character
"to know if the event wasHandled" #handled: #handled
who will handle that? if even not handled, what you do next? what is purpose of it? i mean, at low level (between VM and image) you need only the way to deliver events. who handling them or what happens later is absolutely not our concern at this level.
Keyup
When a key is released, it informs the released key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"the main key pressed. Â It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
ditto
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Glitches
1) Today every keyboard event handled on the image side is handled on keypress, which: Â - limits the ammount of shortcuts you can have (because there are keys that you can't handle) Â - to make it work, there are some hacks (at image and vm side) to allow keys with no char representation enter as keypresses (like arrow keys). Â - changing this is ALOT of work :P
yeah :)
2) Some key representation varies from platform to platform.  i.e.: A Shift-only press generates today a keydown event with:  - a 254 keyCode value in unix  - a  16 keyCode value in windows
IMO, we should deal with this at image side. It is much easier to fix code in image, than digging into VM every time we need to fix it.
So I think a "table" has to be built and documented for mappings for each key in a keyboard. Â Examples of this can be found in enums/defs from other platforms:
Windows platform: http://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).as... .Net: http://msdn.microsoft.com/en-us/library/system.windows.forms.keys(v=vs.71).a... X11 (current used window system in unix vm): http://cgit.freedesktop.org/xorg/proto/x11proto/plain/keysymdef.h Javascript: http://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes Gtk: http://www.koders.com/c/fidD9E5E78FD91FE6ABDD6D3F78DA5E4A0FADE79933.aspx
right.
After that, both vm's and image should follow that table :). Â And of course, these keys should be reified in the image side independently of their character pairs.
Guille
-- Best regards, Igor Stasenko.
On Fri, Jan 27, 2012 at 1:52 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 26 January 2012 19:07, Guillermo Polito <guillermopolito@gmail.com> wrote:
Following the Event chronicles :P
Since I know that some people are working on event Stuff(Stef, Fernando, Igor), I'd like to listen to what they(or anybody else :) ) think about this. I've no code (yet) except for the one I've written to play with the vm and test the keyboard stuff :P. And I know there is some package where Stef and Igor where refactoring the event stuff and I don't want to duplicate efforts either, jeje.
see sqs/EventModel
So, this is what from other models I think a complete keyboard event model should have:
Keyboard interaction raises 3 events (3 different kind of objects!):
Keydown
When a key is pressed, it informs the pressed key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"boolean indicating which modifiers where pressed" #alt #cmd #shift #ctrl #opt?
<RANT> why we can't just send a key code to image and be done with it? what if i want , in my image, for some weird reason, to use 'A' key as a 'modifier'? why at all, we need to distinguish between normal keys from 'modifier' keys, when VM and OSes are capable to generate a same events for all keys user punching? IMO, those 'modifiers' are serving for nothing, but just unnecessary increase of complexity in VM , and data structures (events) we exchanging with VM.
There are 3 bits for modifiers namely , shift, alt, cmd .. so i cannot distinguish between left and right shift pressed (cool! awesome! wonderful!). Not mentioning that some laptops having a 'fn' modifier key, but who needs that, right? </RANT>
Ok, but, what if you have alt+a? You have to send 2 keycodes. And Alt+shift+r 3 keycodes. Or maybe we just send a keydown for each pressed key and handle the combinations inside the image playing with keyups and keydowns... Keydown ctrl -> ctrl pressed keydown shit -> ctrl shift pressed Keydown a -> ctrl shift a pressed keyup ctrl -> shift a pressed Keyup a -> shift pressed keyup shift -> nothing pressed
"a collection of modifier objects" #modifiers
"the key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
Keypress
When a key with a unicode representation is pressed, informs it.
should understand the messages:
"the unicode character representation of this event. It is a Character, not an instance of Key" #character
"to know if the event wasHandled" #handled: #handled
who will handle that? if even not handled, what you do next? what is purpose of it?
Hmm, actually, I was modeling them as morphic events, and it has a meaning there so the event can bubble until it is handled :)
i mean, at low level (between VM and image) you need only the way to deliver events. who handling them or what happens later is absolutely not our concern at this level.
Yeap, SystemEvents and Morphic events should differ I think (one wrapping the other or not...).
Keyup
When a key is released, it informs the released key with modifiers (it should not send a unicode value of a character).
Should understand the messages:
"the main key pressed. It is not a Character, it should be an instance of Key (read about Keys in the Glitches below)" #key
"to know if the event wasHandled" #handled: #handled
ditto
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Glitches
1) Today every keyboard event handled on the image side is handled on keypress, which: - limits the ammount of shortcuts you can have (because there are keys that you can't handle) - to make it work, there are some hacks (at image and vm side) to allow keys with no char representation enter as keypresses (like arrow keys). - changing this is ALOT of work :P
yeah :)
2) Some key representation varies from platform to platform. i.e.: A Shift-only press generates today a keydown event with: - a 254 keyCode value in unix - a 16 keyCode value in windows
IMO, we should deal with this at image side. It is much easier to fix code in image, than digging into VM every time we need to fix it.
why every time? :/ Ok, if we change the model every 2 months yes, it will be a trouble. But if we don't? How much time has passed since the last time the event stuff was touched? I think it does not happen every day... Wasn't it the vm's job to abstract these differences between platforms? Cheers, Guille
participants (4)
-
Guillermo Polito -
Henrik Johansen -
Igor Stasenko -
Stéphane Ducasse