I built this OSK keyboard simulator as a personal project for the purpose of testing alternate keyboard layouts for 10ft UI game controllers and remote controls with D-Pad input (Up, Down, Left, Right). This one allowed the participant to conduct time-comparisons of the native PS4 OSK, YouTube, Netflix with my own custom keyboard layout.
The unique thing about my own "Dawson OSK" was that it was a completely unique alternate keyboard (similar to a Dvorak, Colemak, or Maltron layout) with keys that I chose to organized around frequency of use of single characters, digraphs and trigraphs. My hypothesis was that it would simply take less time to input words and phrases if the keys were organized in such a way that would require the user to enter the fewest number of clicks (assuming they only could move focus up, down, left or right).
For example, in my simple computer test below, you can see that the phrase "Hello World" requires a total of 63 clicks on a standard PS4 QUERTY keyboard, 42 clicks on a YouTube keyboard, 47 clicks on a Netflix keyboard, and only 35 clicks on my version:
Okay fair enough... but what would would be the results of other words and phrases? Would my new alternate keyboard really result in better efficiency? What would happen if we tried 10 strings, 100 strings, 1000 strings?
So what I did was built a simple test which ran a quick bulk test of 1000 game titles. With our PS4 QUERTY keyboard as our baseline / control group, here are the results:
That's right, after entering 1000 phrases, my keyboard proved to be 31.267% more efficient than the default QUERTY keyboard. But that's easy for a computer. What would happen if we tried human subjects?
Well... after several rounds of tests where I would test each participant on how much time it took them to enter five game titles across the four keyboard layouts, I found that on first try, it took participants nearly the same amount of time as the other keyboard layouts. However, I only tested about a dozen subjects, and didn't conduct any sort of formal research study.
At some point, I'd really like to conduct a formal study, and also see how participants faired over repeat attempts. Seems like the better they get at remembering the alternate keyboard layout, the faster they'll get over time.