Look at PPM. Your prediction model would work better with personalised data. PPM is efficient (nb. I see you are using python - look at this https://github.com/willwade/pylm - although be warned - i think my code is not quite right..)
Layout shifting for finger movement - well its great if you didnt have to look. The time for visual processing the letters adds a significant lag (its why typical word prediction isnt used that much and when it is - not over 3 predictions (I have papers on this if you are interested). But its not all bad..
Switch users who need next letter prediction this could dramatically support their rate of input. (view https://youtu.be/Bhj5vs9P5cw?si=VnytfH_vdEUWuLok&t=73 - now note how the keyboard blocks the scan up. But imagine if it just scanned each letter first by next most likely - or heck - like this repo - actually changes button position and kept the scan pattern the same. It would be a ton more efficient)
I think that this is an attempt to stop the “clean key” problem. Security system keypads (especially outdoor ones) tend not to be cleaned, so as time passes, it is easy to spot the dirty keys. Dirty keys are not being pressed and are therefore not in the passcode.
So look for the clean keys and try combinations from there. In a 4 digit (0-9) keypad, knowing the clean numbers drops the possible codes from 9999 to 24 (if my early morning math holds up).
Also helps the issue of someone looking over the shoulder of a valid person. Chances are they are just seeing the position and not the character pressed. So the keyboard changes and you actually need to know the character not just the old position.
If they aren't distinct, you wouldn't have 4 clean buttons, but just 3 - in which case we also know the repeating digit repeats exactly once and we get 12x3 (36) possible combinations. With two clean buttons, it's 6 (if both repeat) + 4 (if only one repeats) = 10 and if there's 1 that's just one, and a terrible password.
I guess you would be able to count the number of clean keys and thus know both the number of distinct digits and the digits (but not the order nor which digit that's repeated)
1. You have lots of spy-data samples that reveal which physical key is pressed (perhaps they sound different) and the precise timing of those strikes, but you don't know what scrambled numbers were actually being shown. (And it's always the same code.)
2. The trick is that users take longer to press a number when it's displayed far away from its "normal" position, because they had to seek longer to find it.
3. This means you can infer the true numbers based on how quickly or slowly presses happen versus which physical key is struck.
For a simple example, assume a two-digit code where there are nine keys. If the fastest first press is always the top left corner, and the fastest second press is always the middle, we can guess the code is either 15 or a 75, depending on if the user is accustomed to phones or keyboard numpads.
This would be an interesting one to integrate into password entry forms... although you'd need to show the randomised keyboard layout on screen.
Or have a keyboard with oled or e-ink keys, like the Optimus Maximus [0] promised to deliver. It's kinda weird that nobody else seems to have picked up on this concept since then. Probably just impractical or too expensive.
I read that its patents expired in 2016; around 2015 there was a concept for an e-ink button keyboard, but that site is now a plain gambling ad. There's also https://www.nemeio.com/ that still works, but its buttons look like sunken screens under plastic domes.
Had this at an ATM recently, and it took a couple of tries at my PIN before I looked at the keypad and realized what was going on. One more wrong PIN and I could have lost my card.
This project brilliantly points out that we are limited and shaped by how keys and layouts are presented to us.
What I don’t understand is why not go all the way. Why not also change the concept of what’s needed to enter a character? It would make sense that keys aren’t the ideal affordance to express a key every time.
Use a volume slider for a letter, use radio buttons, on/off toggles. What if I could draw the letter on my trackpad?
I hope this opens the door for a thriving ecosystem of expansions.
I have been proposing to adapt Shorthand* to computer tactile input many times. Touchpads and touchscreens can use it. It seems faster and easier for both the computer and the person than having to recognize slow handwriting.
Secretaries used this for many decades, because it allowed them to record speech in real-time.
There is always a dismissive comment about how this is not a good idea for one reason or another, but I still think it would be a great feature to have.
(Google's gboard offers a handwriting keyboard, but it's on the touch screen and doesn't count. Also, it's not graffiti, so a bit more error prone. It does recognize joined-up handwriting though.)
There's also some keyboards on f-droid for braille text input and morse code input.
I'd put some random keys on the bottom of my seat or wired into the TV remote or the doorbell button of my neighbours. The ordered shape and regular size, position of keys is too much of a burden to mee, constricts my wondering creative mind too much.
> What about analog partial key press? There are Hall effect keyboard switches that do that, I want to try some.
That sounds fun, now I really want to try out a keyboard where half press is lowercase and full press is UPPERCASE! No more pinky strain from shift presses.
Some pipe organs have keyboards that support that. I can't figure out when it started, but my guess is the 1920s. No hall effect, they had mechanical contacts. (I've never got to play one and always wondered if you could feel when the two positions were hit or if you had to guess and rely on sound - organ pipes often have a long delays between when you press the key and when the speak)
This is true. We’ve been so formatted by big keyboard that I never considered just the tip of my fingers. I hope you’ll make good use of our opposable thumbs in this new keyboard paradigm
Took me 1 year to rewire my brain to switch from Qwerty to Dvorak (I struggled with from RSI for nearly 6 years) and nearly 2 years to gain full speed. It is hard, very hard for the brain. It is mentally tiring to rewire your brain like this and you will do your job at a much worse efficiency (forget pair programming for a while). You need new keyboard stickers also. Great job at making this idea but it is not practical.
I switched to Dvorak 20 years ago and I love it. It only took me about 2 weeks to learn and to pass my qwerty typing speed. I could actually touch type on either keyboard for the first year or two after I switched. I recommend not using stickers or dvorak keyboards at all, it helps you learn faster and your family won't get pissy because they can't use your computer anymore.
Also, it is not difficult to use other peoples computers. You can switch keyboards pretty easy these days on windows and mac. In fact, my very first computer (Apple IIc) had a mechanical switch to flip between qwerty and dvorak.
And there's also the problem of using computers that aren't yours. I haven't tried (and I will never), but I imagine switching between layouts is at least very inconvenient.
I switched to Dvorak ±8 years ago, and I can still type qwerty almost as well as I could back then. I could imagine it being a problem if you're using other people's machines on a daily basis though.
Keyboards aside, I find it really frustrating when something I use frequently - such as the toolbars in excel - changes without warning and breaks my muscle memory.
Very nice website with many solid principles! Though one big flaw in all the designs I've seen is that they rely on bad data - text corpus that is the output of human editing efforts, not the input.
So all those "0.00001%" stats aren't as precise as they seem, and you still see the frequently used backspace/enter in the awful pinky position despite the official philosophy of "Hands Down is easy on the pinkies"
Have you seen any design that is based on actual human input?
As far as I'm aware hands down layouts were primarily designed to be used on a split ergonomic keyboard with thumb cluster, the space and enter lives on the thumb cluster.
Even those better thumb variants don't seem to statistically take that into account (judging from the description/heatmaps/data sources), but then it also presents the "slab" variant...
Thank you for the link! That looks fascinating. I have instantly added it to my "keyboard" bookmarks folder!
I will promptly forget it exists and keep typing on the random layout I learned to touch type on 20 years ago, all the while suffering from terrible finger acrobatics and wishing for a better layout every single day.
For those who type on German keyboards or in German, there are a number of layouts that have evolved around the Neo 2 layout [1]. The main driving force is ergonomics, but since not everyone works the same way, many variants have emerged (and still do) [2]. Interesting is the craftsmanship, including customized programs to optimize the layouts [3].
I always changed to ANSI layout, independent of the layout of the actual keys. Now I carry a HHKB with me, and connect through BT to whatever computer I want to work on. No matter what human language I write in (I do some) I prefer using accidents like tildes and umlauts with modifier keys.
I've made my own layouts[0][1] and I'd say the many variants of Hands Down are probably the best example of a layout that's been developed in a thoughtful manner.
Yeah when I was a 20 or so and just started working I taught myself Dvorak for a while, but realized how bad it was for coding in C. So I made my own C language one as well.
Of course you still have to type text and at the time you couldn't easily switch between different layouts, so after a time I got bored of it and gave up.
thats wonderful. id like that to not change the order of the letters - but change the highlight order. Do a round 1 of frequency order first (just do first say 6 letters) then do a round 2 which is standard order..
i probably am not making much sense. Look at where I'm coming from in the world of Assistive Tech - https://docs.acecentre.org.uk/products/echo (go to around 5 min mark in the vide)
This is not a bad idea. It's not dissimilar from an autocomplete engine. If reset per word and using a static probability table it will work its way into muscle memory. In practice fingers will only need to leave the home row for some initial letters.
> (…) In the first year, "s" will replace the soft "c". Sertainly, this will make the sivil servants jump with joy. The hard "c" will be dropped in favour of "k". This should klear up konfusion, and keyboards kan have one less letter. (…)
That joke is many decades older than the creation of the european union: http://www.i18nguy.com/twain.html (probably not written by Mark Twain, but seemingly first appeared when he was still around?)
Repeatedly pressing the home row keys almost always turned into word end sequences like erere, I assume from words like there/were/here. I'm not sure how to go further with single character frequencies.
Nice. Some thoughts.
Look at PPM. Your prediction model would work better with personalised data. PPM is efficient (nb. I see you are using python - look at this https://github.com/willwade/pylm - although be warned - i think my code is not quite right..)
Layout shifting for finger movement - well its great if you didnt have to look. The time for visual processing the letters adds a significant lag (its why typical word prediction isnt used that much and when it is - not over 3 predictions (I have papers on this if you are interested). But its not all bad..
Switch users who need next letter prediction this could dramatically support their rate of input. (view https://youtu.be/Bhj5vs9P5cw?si=VnytfH_vdEUWuLok&t=73 - now note how the keyboard blocks the scan up. But imagine if it just scanned each letter first by next most likely - or heck - like this repo - actually changes button position and kept the scan pattern the same. It would be a ton more efficient)
(and a bit of a rabbit hole.. What if keys had word predictions on them? This is basically the end result of ACE-LP: https://discovery.dundee.ac.uk/en/publications/ace-lp-augmen...)
Yes! I have been wanting this forever. Keys are in too predictable a position
There were, for awhile, some security systems had on-screen keyboards that would change layout on every key press.
I think that this is an attempt to stop the “clean key” problem. Security system keypads (especially outdoor ones) tend not to be cleaned, so as time passes, it is easy to spot the dirty keys. Dirty keys are not being pressed and are therefore not in the passcode.
So look for the clean keys and try combinations from there. In a 4 digit (0-9) keypad, knowing the clean numbers drops the possible codes from 9999 to 24 (if my early morning math holds up).
Also helps the issue of someone looking over the shoulder of a valid person. Chances are they are just seeing the position and not the character pressed. So the keyboard changes and you actually need to know the character not just the old position.
It would be 24 if all digits are distinct
It actually drops from 10^4=10000 to 4^4=256 combinations
If they aren't distinct, you wouldn't have 4 clean buttons, but just 3 - in which case we also know the repeating digit repeats exactly once and we get 12x3 (36) possible combinations. With two clean buttons, it's 6 (if both repeat) + 4 (if only one repeats) = 10 and if there's 1 that's just one, and a terrible password.
With 2 clean buttons, there are 4x2 ways for only 1 to repeat, giving 14 combinations in total.
I guess you would be able to count the number of clean keys and thus know both the number of distinct digits and the digits (but not the order nor which digit that's repeated)
The number that is repeated is likely to be dirtier than the numbers that are not so you get that information too.
Interesting. This would also stop keypress extraction via analyzing audio.
Hmm... It may still be vulnerable if:
1. You have lots of spy-data samples that reveal which physical key is pressed (perhaps they sound different) and the precise timing of those strikes, but you don't know what scrambled numbers were actually being shown. (And it's always the same code.)
2. The trick is that users take longer to press a number when it's displayed far away from its "normal" position, because they had to seek longer to find it.
3. This means you can infer the true numbers based on how quickly or slowly presses happen versus which physical key is struck.
For a simple example, assume a two-digit code where there are nine keys. If the fastest first press is always the top left corner, and the fastest second press is always the middle, we can guess the code is either 15 or a 75, depending on if the user is accustomed to phones or keyboard numpads.
Also spy robots with thermal imaging eyeballs.
This would be an interesting one to integrate into password entry forms... although you'd need to show the randomised keyboard layout on screen.
Or have a keyboard with oled or e-ink keys, like the Optimus Maximus [0] promised to deliver. It's kinda weird that nobody else seems to have picked up on this concept since then. Probably just impractical or too expensive.
I read that its patents expired in 2016; around 2015 there was a concept for an e-ink button keyboard, but that site is now a plain gambling ad. There's also https://www.nemeio.com/ that still works, but its buttons look like sunken screens under plastic domes.
[0] https://en.wikipedia.org/wiki/Optimus_Maximus_keyboard
It’s still a relatively common thing for pin-coded door/gate security.
Had this at an ATM recently, and it took a couple of tries at my PIN before I looked at the keypad and realized what was going on. One more wrong PIN and I could have lost my card.
I have seen this with some of the card swiping machines in India.
I occasionally still get this in certain petrol stations. Always catches me off guard.
This project brilliantly points out that we are limited and shaped by how keys and layouts are presented to us.
What I don’t understand is why not go all the way. Why not also change the concept of what’s needed to enter a character? It would make sense that keys aren’t the ideal affordance to express a key every time.
Use a volume slider for a letter, use radio buttons, on/off toggles. What if I could draw the letter on my trackpad?
I hope this opens the door for a thriving ecosystem of expansions.
> What if I could draw the letter on my trackpad?
I have been proposing to adapt Shorthand* to computer tactile input many times. Touchpads and touchscreens can use it. It seems faster and easier for both the computer and the person than having to recognize slow handwriting.
Secretaries used this for many decades, because it allowed them to record speech in real-time.
There is always a dismissive comment about how this is not a good idea for one reason or another, but I still think it would be a great feature to have.
* https://en.wikipedia.org/wiki/Shorthand
Heard of dasher ? https://www.inference.org.uk/dasher/
> What if I could draw the letter on my trackpad?
The Apple Newton did that!
https://en.wikipedia.org/wiki/Apple_Newton#Notes
Also the palm, they called it graffiti. https://en.wikipedia.org/wiki/Graffiti_%28Palm_OS%29
(Google's gboard offers a handwriting keyboard, but it's on the touch screen and doesn't count. Also, it's not graffiti, so a bit more error prone. It does recognize joined-up handwriting though.)
There's also some keyboards on f-droid for braille text input and morse code input.
That feels like Siri dictation, minus Siri, and minus the dictation
I'd put some random keys on the bottom of my seat or wired into the TV remote or the doorbell button of my neighbours. The ordered shape and regular size, position of keys is too much of a burden to mee, constricts my wondering creative mind too much.
We're also limited and shaped by keyboards. Why only one letter per finger? The datahand does five!
Why are key switches only press?
What about analog partial key press? There are Hall effect keyboard switches that do that, I want to try some.
What about finger twist for input? I don't think anything exists? Would that be a volume knob on a key switch?
> What about analog partial key press? There are Hall effect keyboard switches that do that, I want to try some.
That sounds fun, now I really want to try out a keyboard where half press is lowercase and full press is UPPERCASE! No more pinky strain from shift presses.
Some pipe organs have keyboards that support that. I can't figure out when it started, but my guess is the 1920s. No hall effect, they had mechanical contacts. (I've never got to play one and always wondered if you could feel when the two positions were hit or if you had to guess and rely on sound - organ pipes often have a long delays between when you press the key and when the speak)
The CharaChorder [1] is one attempt to give a completely different shape to input methods
[1] : https://www.charachorder.com
This is true. We’ve been so formatted by big keyboard that I never considered just the tip of my fingers. I hope you’ll make good use of our opposable thumbs in this new keyboard paradigm
This mixed with LED keycaps that change to reflect the current assigned letter would be awesome.
For hunt and peck typists.
Seems like eink would be better, but the smallest I can find is about 3cm so much too big. If you can get them smaller though...
I wrote a second blog post about this project weeks after the first release: https://www.scannedinavian.com/markov-keyboard-the-rabbit-ho...
Hi shae!
Took me 1 year to rewire my brain to switch from Qwerty to Dvorak (I struggled with from RSI for nearly 6 years) and nearly 2 years to gain full speed. It is hard, very hard for the brain. It is mentally tiring to rewire your brain like this and you will do your job at a much worse efficiency (forget pair programming for a while). You need new keyboard stickers also. Great job at making this idea but it is not practical.
I switched to Dvorak 20 years ago and I love it. It only took me about 2 weeks to learn and to pass my qwerty typing speed. I could actually touch type on either keyboard for the first year or two after I switched. I recommend not using stickers or dvorak keyboards at all, it helps you learn faster and your family won't get pissy because they can't use your computer anymore.
Also, it is not difficult to use other peoples computers. You can switch keyboards pretty easy these days on windows and mac. In fact, my very first computer (Apple IIc) had a mechanical switch to flip between qwerty and dvorak.
> You need new keyboard stickers also.
You absolutely do not and it even hampers your ability to quickly learn to type efficiently if you rely on key legends or layout references.
And there's also the problem of using computers that aren't yours. I haven't tried (and I will never), but I imagine switching between layouts is at least very inconvenient.
I switched to Dvorak ±8 years ago, and I can still type qwerty almost as well as I could back then. I could imagine it being a problem if you're using other people's machines on a daily basis though.
This entire discussion reads like perfect satire because I still can't decide whether we are being serious or not
Keyboards aside, I find it really frustrating when something I use frequently - such as the toolbars in excel - changes without warning and breaks my muscle memory.
I learned recently that there are way, way more alt-layouts than the ones you've heard of: Dvorak & Colemak.
Not only that, people just change them to suit their needs! They use things like Hands Down [0] as a guide, and make up their own layout.
Wild.
[0]: https://sites.google.com/alanreiser.com/handsdown
Very nice website with many solid principles! Though one big flaw in all the designs I've seen is that they rely on bad data - text corpus that is the output of human editing efforts, not the input.
So all those "0.00001%" stats aren't as precise as they seem, and you still see the frequently used backspace/enter in the awful pinky position despite the official philosophy of "Hands Down is easy on the pinkies"
Have you seen any design that is based on actual human input?
As far as I'm aware hands down layouts were primarily designed to be used on a split ergonomic keyboard with thumb cluster, the space and enter lives on the thumb cluster.
Even those better thumb variants don't seem to statistically take that into account (judging from the description/heatmaps/data sources), but then it also presents the "slab" variant...
Thank you for the link! That looks fascinating. I have instantly added it to my "keyboard" bookmarks folder!
I will promptly forget it exists and keep typing on the random layout I learned to touch type on 20 years ago, all the while suffering from terrible finger acrobatics and wishing for a better layout every single day.
For those who type on German keyboards or in German, there are a number of layouts that have evolved around the Neo 2 layout [1]. The main driving force is ergonomics, but since not everyone works the same way, many variants have emerged (and still do) [2]. Interesting is the craftsmanship, including customized programs to optimize the layouts [3].
[1] https://www.neo-layout.org/Layouts/
[2] https://maximilian-schillinger.de/keyboard-layouts-neo-adnw-...
[3] https://hg.sr.ht/~arnebab/evolve-keyboard-layout
German layouts with [ ] { } in such horrible positions are killing all joy when programming, especially in curly brace heavy languages.
Changed to US qwerty 30 years ago, never looking back.
I always changed to ANSI layout, independent of the layout of the actual keys. Now I carry a HHKB with me, and connect through BT to whatever computer I want to work on. No matter what human language I write in (I do some) I prefer using accidents like tildes and umlauts with modifier keys.
I've made my own layouts[0][1] and I'd say the many variants of Hands Down are probably the best example of a layout that's been developed in a thoughtful manner.
[0]: https://www.jonashietala.se/series/t-34/
[1]: https://www.jonashietala.se/blog/2024/11/26/the_current_cybe...
Would you say they are hands down the best?
:)
(I do believe that the best for you is a custom layout.)
And there are modal editors, who switch keylaouts depending on the need of the minute.. debug/edit/search
Yeah when I was a 20 or so and just started working I taught myself Dvorak for a while, but realized how bad it was for coding in C. So I made my own C language one as well.
Of course you still have to type text and at the time you couldn't easily switch between different layouts, so after a time I got bored of it and gave up.
Nice! I wrote about something similar for rectangular layouts: https://maxhalford.github.io/blog/dynamic-on-screen-keyboard...
thats wonderful. id like that to not change the order of the letters - but change the highlight order. Do a round 1 of frequency order first (just do first say 6 letters) then do a round 2 which is standard order..
i probably am not making much sense. Look at where I'm coming from in the world of Assistive Tech - https://docs.acecentre.org.uk/products/echo (go to around 5 min mark in the vide)
How about a backlit keyboard, and if the next letter is probably a vowel, only the vowels are lit?
I'd be quite interested in trying that. Maybe as a learning aid. But if you get your typing speed up, I can see it being visually very noisy.
ah, I have been looking for this all my life, thank you! ;-)
(though really what I want is dabbrev/pabbrev to sort predictions by markov order at least 2 or even a tiny recurrent network)
This is not a bad idea. It's not dissimilar from an autocomplete engine. If reset per word and using a static probability table it will work its way into muscle memory. In practice fingers will only need to leave the home row for some initial letters.
What letter is a good start if you do a reset at every word boundary? Maybe whatever letter is most common?
E I guess?
I'm still looking for a way to produce quality keycaps at home.
Fun concept. I had a thought about having it optimize in a way where left and right hands alternate for increased speed
I switch between English and German a lot. I wonder what that does to the algorithm.
There is an old joke that seems approproate:
https://old.reddit.com/r/Jokes/comments/2o4rkq/english_to_be...
> (…) In the first year, "s" will replace the soft "c". Sertainly, this will make the sivil servants jump with joy. The hard "c" will be dropped in favour of "k". This should klear up konfusion, and keyboards kan have one less letter. (…)
That joke is many decades older than the creation of the european union: http://www.i18nguy.com/twain.html (probably not written by Mark Twain, but seemingly first appeared when he was still around?)
Very interesting, I wonder if you could incorporate text prediction so that you can write a sentence in just one key (aside from space).
Repeatedly pressing the home row keys almost always turned into word end sequences like erere, I assume from words like there/were/here. I'm not sure how to go further with single character frequencies.
Too bad this github has latest changes 5 years ago, I wonder what they could update with LLMs, etc.
My guess would be not much. LLMs are pretty useless concerning anything novel. Maybe an LLM could update the docstrings?
I'm assuming they meant replacing the markov model with a modern language model, not having an LLM magically improve the repo.
That sounds like it'd give you no hope at all of ever learning muscle memory for it.