The first glimpse appeared during summer of ’11 in Egypt. After having a couple drinks I was talking to my friend about Kinect technology which was quite new back then. Soon we brainstormed about usage ideas and drifted towards having all kinds of virtual stuff: guitar, percussion, puppy to play with etc. But how would you design a virtual keyboard – one that you can use on any surface or even in the air? (and yes, we considered sign language – but for that moment we considered that too complex to learn)
We concluded: Kinect would need to observe fingers-to-wrists relative positions. But then it would be inconvenient to type on plain surface without seeing the buttons (or pretending to push them in the air). A potential solution could be to recognize chords – combinations of finger strokes which would encode the letters. Their position wouldn’t matter then, but only which fingers are down.
Still that would require finger recognition – which Kinect doesn’t support yet :(
But hey! I could actually build a 10-keyed keyboard myself – just to see how it works. After imagining bizarre models made out of bottle-caps, sticks and wires I thought about using Mindstorm Lego blocks. Then I realized I could as well just emulate that on regular keyboard :)
Right after coming home I dug out the napkin design and started experimenting with various layouts on my laptop. It came to my attention that:
- Fingers: Index, Middle, Ring are the fastest ones & easiest to control
- Thumb is slower, but still easy to control
- Pinky is almost as fast as first three, but not as reliable (error-prone)
- Right hand tends to be more intelligent than Left.
Having these observations I came up with that:
There are 5 keys for each hand – I chose them to correspond with natural finger position when rested. You might also notice that the left hand is placed a little higher than right. This is due to strange keyboard wiring – certain key patterns are not recognized when pressed at the same time, e.g open Notepad and try pressing the keys without releasing:
- v + 3 + 2 – all characters are appear :)
- v + 4 + 3 – the last ’3′ character doesn’t show up :(
That’s how the project works:
- Fingers: Index, Middle, Ring: choose row/column inside a square
- Thumbs select the square (middle is default – doesn’t require any action)
- Pinkies are responsible for alternative sets: upper case & special characters
The character mapping was designed based on letter frequency, common bigrams & trigrams. Most frequent letters are located in the middle square (=doesn’t require slow thumb action). Characters that often go together are located in same rows\columns – to speed up switching between them.
Despite my best efforts I didn’t manage to beat the regular keyboard typing speed.
But I haven’t gave up just yet!
When thinking about that, I recalled that one of the fastest geek-species are… the gamers. And of course I needed to try out WASD + Arrow keys as well:
I bet you can figure out the usage by the picture & highlighted pressed keys :)
At that point it took only one spark to to realize that WASD & Arrows might be as well two analog sticks on a game controller. Especially that currently typing text on any of the game consoles is plain TORTURE!
As much as I liked the above design I realized the first impression might be too freaky for some folks. Also the letters are quite mixed up with special characters which might be confusing at start. I also decided to get back to good ol’ qwerty mapping to further minimize the future shock. After some polishing this is the concept I decided to publish:

I hope you like it.