Ergonomic Virtual Keyboard for a Mobile Device

Idea Add comments

iphone_area_and_keyboardPurpose: Allow faster text entry on mobile devices, esp. on iPhones, iPod touch’s and Android phones.

Kind: Mobile development (general) / App development (local realization)

Target Group: Every smartphone user, esp. iOS users

Can be realized by: Software Developers, App Developers

Scope: days

Problem: Smartphone keyboards, physical or virtual are not ‘smart’ (to paraphrase Steve Jobs). They align the keyboard horizontally and repeat the layout of a much larger physical keyboard (or typewriter). Not only are the keys too small, users do also not face the keyboard like when they are sitting in front of a PC. They usually hold the smartphone in one hand and either type with the other hand or with the thumb of the hand with which they are holding the phone. Furthermore, the keyboard covers the information that is displayed! But with a virtual keyboard there is no need for that — it can be a semitransparent HUD. Thus, the currently used horizontally aligned keyboard is not useful for writing on a mobile device.

Solution: Have a look at the area which the thumb of the hand that is holding the phone can cover (see image), and how easy it is to reach certain place on the display (green = very good, blue = okay, yellow = not so good, red = should be avoided; iPhone held with the right hand on the lower part — actually it was an iPhone 4). This implies that the keys should be placed in these areas depending on character frequency in the native language (yes, this means saying good bye (or riddance) to QWERTZ): Most frequently used keys (e.g., “e”, “l”) should be place in the green area, lesser used characters in the blue and yellow one. Given that the keyboard will cover the whole screen (with the top left area for saving, etc.) the keys should be semi-transparent (HUD-style). The text is either in the text field where it belongs or shown at the bottom of the screen. Notice that the default-keyboard of iOS covers the worst part of the touchscreen in terms of easy of access.

Feature list: It should have the following features:

  • Mirror possible for left-handed users.
  • Keys are not horizontally aligned rectangles but polygons aligned to the circle the thumb makes when moving.
  • Night mode where the keys are shown in dark red.
  • All keys are lower case, upper case by quickly pressing the same key twice — if it is either the beginning of a paragraph or there is a space in front of the current place, this produces an upper case key, if not, it gives the key a second time.
  • Large space key or space by swiping horizontally.
  • Return key is triple space or swipe downwards.
  • Double space equals the same as on iOS Keyboards, space and dot.
  • Auto-completion words are shown above the keyboard and can be selected by swiping in the direction of the word and ending on it (pressing on it will do nothing but select the key below it)
  • One key changes the keyboard to display the special characters (can be selected beforehand, e.g., depending on the country). Likewise umlauts are displayed on the special character view or if there is only one other variant they can be selected by triple-clicking on the basic character.
  • Size of the keys is adapted to the position, e.g., I have less dexterity on the right side of the screen if I hold the device with the right hand and type with the right thumb — consequently, these buttons should be a bit larger. With a virtual keyboard this is no problem.
  • Position of keys (which letter a key produces) could depend on the letters the user types. It’s easy to keep track of the letters that are used and it can even be done retrospectively by feeding a script with some chat transcripts. The strength of virtual keyboards is adaptivity — why not use it?

Similar solutions: None known, although there are similar interfaces in games. The keyboard of the iPad can now be split (in iOS 5), but so far there is no similar solution for smaller devices where you type with the same hand with which you are holding the device.

Influenced by: Games interfaces and my discontent and frustration when typing while walking.

Note: I will program such a keyboard for an App I have in mind, but it would be nice if this was more widely available — and ideally, part of the settings of the phone.

Update:

I played around with different keyboard designs with XCode and found one that will likely work for me. Below is a sample app, a proof of concept.

They keys are semi-transparent (alpha .6), the text view behind it has a large text in lightGrayColor with autoscrolling (found the code for this on the net, programmers are so cool! :-) ). It has two views for typing, one with the major characters (incl. the umlaute, as I occasionally make notes in German), the other with numbers and other characters (switching via the orange key top right). Space and Backspace are easy to reach in the vertical middle at the top (in gray).

There is no shift key as it is not needed. Shift keys make sense with a keyboard where you can use more than one finger at the same time. The ways Shift is realized on virtual keyboards is similar to keyboards for the disabled, usually not a very convenient solution. Here you simply hit the key and drag the finger outside of the key area you have just pressed (UIControlEventTouchDragExit). This triggers the upper key function (some non-alphabetic characters also have uppercase functions). This is much, much faster than the press-and-hold solution of Apple keyboards for alternative uses of the key (you have to stop writing for a second, why?!?!) and also faster than pressing a shift key that is somewhere else on the screen. You just press the key directly and drag your finger outside, simple, easy, convenient and fast. I thought about double taps but there are a lot of keys which you type multiple times because you want the same letter twice or trice, so I thought this would be a bad idea. TouchDragExit works much better.

Not all functions are realized at the moment, but it can copy the text to the clipboard and send it automatically via eMail (address, subject, and text body automatically entered). Storage of multiple notes via an SQLite database comes later. It also has a privacy mode where the text is either very close to the white background or (mode 2) where the text is white (I never liked it when people look at my screen ;-) ). It also has a mode where you can see the text view and have a default keyboard to edit the text, as not all functions are implemented yet. The visual design of the keys is “strongly inspired” by the iOS App “Mass Effect Galaxy” because they somehow draw your finger in. No copyright infringement is intended as I will use the App only for private purposes.

I think the hardest part here is to learn the locations of the keys, which was the reason why I deviated from the “most used keys in the areas that are easiest to reach” idea and choose an alphabetical order. I also tried out ordering them in a spiral but it placed some of the more often used keys in hard to reach areas (and made my thumb spiral around and around in search of the keys).

Next up are some functions that are missing (storing multiple notes via SQLite, Undo, etc.) and later connection via iCloud. I am also thinking about a Quicklink Tool that makes sending SMS, MMS, and eMail easier. Thing is, what the key writes is stored as text in an array and there is no difference between a single letter (like “a”) and a whole sentence like “Hello [Placeholder], one question, [Placeholder 2]. Best regards, Daniel” which could be used to write eMails more quickly. Especially on a mobile device this is very helpful. I’m thinking about getting the name automatically via the address and some kind of form to more quickly write short messages — hence Quicklink (like the tool in the Sid Meier’s Alpha Centauri Stories).

Share and Enjoy:
  • Twitter
  • Facebook
  • del.icio.us
  • StumbleUpon
  • Digg
  • Google Bookmarks
  • Yahoo! Buzz
  • Print
  • email

3 Responses to “Ergonomic Virtual Keyboard for a Mobile Device”

  1. Daniel (posting an eMail) Says:

    Mike send me the following eMail with the idea of using Morse code to type while walking. I think the idea is brilliant and I would like to see how well it works out (having made some keyboard layouts, the devil is usually in the details). I think that the idea has huge potential, esp. if the other sensors of the device are used, e.g., sound, accelerometer, etc. and perhaps audio for feedback.


    Daniel,

    I too would like to be able to type while walking or in bouncing environments like buses that make tiny touchscreen typing difficult. Below is my idea on the topic. Feel free to publish it or pass it on.

    Note that it has the added advantage that you don’t need to be looking at the screen.

    –Mike Perry, Seattle

    *****

    Swipe to Type

    Here’s an alternate text input technique for the iPhone and similar devices that might be faster and more accurate for many people. It uses a feature the iPhone already has, a multi-touch screen, rather than external hardware such as a collapsible Bluetooth keyboard. You not only don’t have to look at the screen, with a little practice you can enter text in the dark even while bouncing around in a car, bus or subway. And since the only requirements for text input are basic hand coordination and a sense of touch, it makes the iPhone much more usable for the visually impaired and those with limited hand-eye coordination.

    What is it?

    * It uses a well-established open source standard—International Morse Code. But instead of short and long key presses, dots are input by short swipes and dashes by long swipes.

    * Speed of input doesn’t matter. Unlike regular Morse, which assumes a pause in sending to be a break between letters, user input can be as slow or fast as the users wants without error. Letters are distinguished by alternating swiping right and left. A user-set delay inputs the last character, i.e. one not followed by a swipe in a different direction. Users can also set the ratio between long and short swipes.

    * Swipe mode changes when the user rotates the screen.

    * Because International Morse Code is already optimized for fast input in many languages, text can be entered very fast. The more often a letter is used, the shorter its Morse Code equivalent is. An e is a single short swipe and a T is a single long swipe. It couldn’t be easier.

    Additional Features

    Morse input would also take advantage of a touch screen’s flexibility to add features that International Morse Code doesn’t have. Examples include:

    * Lowercase letters are made by swiping left-to-right then right-to-left.

    * Uppercase letters are made by swiping down-to-up and then up-to-down.

    * Other gestures can be used. Common punctuation uses diagonal swipes, i.e. upper-left to lower-right for a space, lower-left to upper-right for a period or a period plus space. Diagonal swipes with two or three fingers could have other meanings.

    * Circling CCW might delete the previous character for each circle. Circling CW might enter a Return. Alternately, a short shake of the iPhone deletes the previous letter, while a longer shake deletes the previous word.

    * Because text input is always a swipe that doesn’t need for anything to be displayed for it to work, the entire screen is free for other uses, either display or touching without swiping. It can be used to display the text being entered, to have buttons for commands, or to show a chart for those just learning Morse. This makes maximum use of scarce screen space.

    * Certain easy-to-make touches could be used to make common commands easy to do. Touching the keyboard with another finger, perhaps the thumb in the lower-left corner for right-handed people, might signify something. For instance, it might bring up a scrolling list of long, user-set text strings (i.e. a phone number or address) from which the user could select. Inside applications, it could be used for something important. Inside an email program, for instance, it could send the just-entered email. Inside a writing program, it could be used to start a new paragraph.

    * In learner mode, the screen would display the Morse alphabet and text input would be on a scrolling line. Letters or words could be spoken as typed to speed up learning and accuracy.

    For those willing to learn Morse, which is far easier than most people think (especially for sending), it offers a fast, virtually error-free text interface for the iPhone, one that has tactile feedback built into the design. Most important of all, it’s a text input technique that doesn’t require users to constantly look at the screen. Since the target is the entire screen, it’s impossible to miss and the touch of the screen provides the tactile feedback lacking in the on-screen keyboard.

    Feel free to pass the idea along to anyone who might want to implement it.

    —Mike Perry

  2. Creating your own Tools | ORGANIZING CREATIVITY Says:

    [...] I am still doing it, for example, after posting on “Ark of Ideas” about the need of a “smart” virtual keyboard…, I have started to develop one. Reason is, I love typing while walking, because the information is [...]

  3. Learn from Mobile Apps | ORGANIZING CREATIVITY Says:

    [...] and they want to have the information quickly, but then they are done with it. And given that entry of information is inconvenient of a mobile device (esp. text), they want to have it as easy as possible. You can learn a lot from situations where much stronger [...]

Site Copyright by Daniel Wessel 2011. The ideas I had and have published on this site are released under the Creative Commons License (Attribution) -- legal code here.
This site uses a modified Wordpress Theme created by Ray Creations.