The following is the first few sections of a chapter from The Busy Coder's Guide to Android Development, plus headings for the remaining major sections, to give you an idea about the content of the chapter.


Keyboard and Mouse Input

More and more Android users are starting to use external keyboards and mice with their devices. Sometimes, the device is designed for such use, such as the Jide Remix Mini or all-on-one units like the HP Slate 21. Some people use Android devices designed for use with a TV as quasi-desktops. And, starting in 2016, we have Android available on some Chrome OS devices, most of which rely on keyboard and mouse/trackpad input.

Over time, more and more Android users are going to be expecting Android apps to behave like desktop apps with respect to keyboards and mice. Some of this capability will be built into Android. Some of this capability will need to be handled by apps or libraries.

In this chapter, we will explore various techniques for making your Android app more friendly to keyboards and mice.

Prerequisites

Understanding this chapter requires that you have read the core chapters. Many of the examples use RecyclerView, so you may wish to review that chapter if you have not used RecyclerView very much. Also, some of the examples are based on drag-and-drop samples covered elsewhere in the book.

Offering Keyboard Shortcuts

One thing that users will expect from desktop-style apps is the ability to use keyboard shortcuts. Basic keyboard navigation comes “for free” for a lot of Android use cases, though in some situations you will need to add in your own keyboard smarts, such as for navigating a RecyclerView. And some keyboard shortcuts will come automatically, such as Ctrl-C and Ctrl-V for copy and paste within an EditText.

Anything beyond that, though, you would have to provide yourself.

Action Bar Item Shortcuts

The simplest way to add keyboard shortcuts is to use android:alphabeticShortcut or android:numericShortcut on your <item> elements in a <menu> resource that you use to populate an action bar. Android will automatically support those in concert with the Ctrl key. So, for example, if you had:

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
  <item
    android:id="@+id/play_video"
    android:alphabeticShortcut="p"
    android:icon="@drawable/ic_movie_white_24dp"
    android:showAsAction="always"
    android:title="@string/menu_video" />
  <item
    android:id="@+id/show_thumbnail"
    android:alphabeticShortcut="t"
    android:icon="@drawable/ic_insert_photo_white_24dp"
    android:showAsAction="always"
    android:title="@string/menu_thumbnail" />
</menu>

then your onOptionsItemSelected() method would be called not only if the user taps on the action bar items on-screen, but also if the user pressed Ctrl-P or Ctrl-T on the keyboard.

As the names suggest, android:alphabeticShortcut takes a letter and android:numericShortcut takes a number. However, try to avoid overriding existing shortcuts with unrelated logic. For example, you might consider using android:alphabeticShortcut="v" for a “play video” action, but that would conflict with the Ctrl-V shortcut used for paste. You would be better off going with android:alphabeticShortcut="p" to avoid the conflict.

In Android 8.0+, in Java, we can call variations on setAlphabeticShortcut(), setNumericShortcut(), and setShortcuts() that allow us to change the modifier keys from the default. There are equivalent android:alphabeticModifiers and android:numericModifiers attributes for <item> elements in a menu resource, to indicate the modifier keys to be used with alphabetic and numeric shortcuts.

Arbitrary Hotkeys

You may find that the action bar approach is insufficient:

You are welcome to make anything be a keyboard shortcut or “hotkey”, by overriding the appropriate KeyEvent methods in an Activity or View.

The KBMouse/Hotkeys sample project is a clone of the DragDrop/Simple sample app from the chapter on supporting drag-and-drop. As with the original app, we show a list of available videos on the device, which the user can play or view a larger thumbnail from the video. However, in addition to drag-and-drop as a way of doing those things, this sample app also supports keyboard shortcuts:

However, these keyboard shortcuts imply that the user has chosen a video to play. So, this sample app blends in the keyboard-enabled RecyclerView code from the corresponding section in the RecyclerView chapter. So, as the user presses the Down and Up arrow keys, the chosen row is highlighted. That will be the video that we work with, if the user then goes and presses Alt-Right or Ctrl-Right.

This means that we need our play-the-video and show-the-large-thumbnail code to be accessible from multiple entry points, so we pull those out into dedicated playVideo() and showLargeThumbnail() methods that take the video Uri as input:

  private void playVideo(Uri videoUri) {
    player.setVideoURI(videoUri);
    player.start();
  }

  private void showLargeThumbnail(Uri videoUri) {
    Picasso.with(thumbnailLarge.getContext())
      .load(videoUri.toString())
      .fit().centerCrop()
      .placeholder(R.drawable.ic_media_video_poster)
      .into(thumbnailLarge);
  }

We then use those from the ACTION_DROP processing, for when the user drops the video into either the VideoView (referenced in the player field) or the ImageView:

      case DragEvent.ACTION_DROP:
        ClipData.Item clip=event.getClipData().getItemAt(0);
        Uri videoUri=clip.getUri();

        if (v==player) {
          playVideo(videoUri);
        }
        else {
          showLargeThumbnail(videoUri);
        }

        break;

Handling the keyboard shortcuts is relatively straightforward, courtesy of the onKeyDown() callback that we override:

  @Override
  public boolean onKeyDown(int keyCode, KeyEvent event) {
    if (keyCode==KeyEvent.KEYCODE_DPAD_RIGHT && event.getRepeatCount()==0) {
      int position=adapter.getCheckedPosition();

      if (position>=0) {
        Uri videoUri=adapter.getVideoUri(position);

        if (event.isAltPressed()) {
          playVideo(videoUri);
        }
        else if (event.isCtrlPressed()) {
          showLargeThumbnail(videoUri);
        }

        return(true);
      }
    }

    return(super.onKeyDown(keyCode, event));
  }

We are passed an int keycode (keyCode) and the full KeyEvent for whatever key that the user pressed. If the main key was Right (identified as KEYCODE_DPAD_RIGHT for historical reasons, and to support D-pad directional navigation options), we find out which row in the RecyclerView is checked, if any. If we have a checked row, we find out what the Uri is of the video, then call isAltPressed() and isCtrlPressed() on the KeyEvent to find out which modifier key was pressed in conjunction with Right, if any. If we have a match, we call the associated playVideo() or showLargeThumbnail() method.

onKeyDown() tends to model user expectations, in that the user expects the event to occur when the key is pressed. However, if the user continues holding down the key, we will get a stream of onKeyDown() calls. That is why we also check getRepeatCount(), to ignore the repeated keypresses, so we only try playing the video or showing the large thumbnail once if the user holds down Alt-Right or Ctrl-Right.

Android 7.0 Keyboard Shortcuts Helper

The next challenge is letting the user know what keyboard shortcuts are available. Historically, our primary option would be to hope that the user reads the app’s documentation.

(you can stop laughing now)

Android 7.0 recognizes this and provides a system-wide keyboard shortcuts helper. The user can invoke this using one keyboard shortcut that (hopefully) the user will remember: Meta-/. On a Windows-centric keyboard, the Meta key is the one with the Windows logo on it.

On pre-N devices, you could offer your own keyboard shortcut mapped to Alt-/ and pop up your own keyboard shortcut dialog. Since / is neither a letter or a number, and since having a keyboard shortcut action bar item might not make sense, you would do this using the onKeyDown() technique profiled in the previous section.

In the KBMouse/HotkeysN sample project, we will see how to:

Mostly, the project is a clone of the hotkey sample shown above, with the build.gradle file updated to build for Android 7.0:

apply plugin: 'com.android.application'

dependencies {
    implementation 'com.android.support:recyclerview-v7:24.1.1'
    implementation 'com.squareup.picasso:picasso:2.5.2'
}

android {
    compileSdkVersion 24
    buildToolsVersion '26.0.2'

    defaultConfig {
        applicationId "com.commonsware.android.kbmouse.hotkeys.n"
        minSdkVersion 23
        targetSdkVersion 24
    }
}

We have a menu resource now for our action bar, which happens to be the one shown towards the start of this chapter:

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
  <item
    android:id="@+id/play_video"
    android:alphabeticShortcut="p"
    android:icon="@drawable/ic_movie_white_24dp"
    android:showAsAction="always"
    android:title="@string/menu_video" />
  <item
    android:id="@+id/show_thumbnail"
    android:alphabeticShortcut="t"
    android:icon="@drawable/ic_insert_photo_white_24dp"
    android:showAsAction="always"
    android:title="@string/menu_thumbnail" />
</menu>

We inflate that menu resource in onCreateOptionsMenu() using the typical recipe:

  @Override
  public boolean onCreateOptionsMenu(Menu menu) {
    getMenuInflater().inflate(R.menu.actions, menu);

    return(super.onCreateOptionsMenu(menu));
  }

In onOptionsItemSelected(), we need to confirm that the user has selected a row in the RecyclerView using the keyboard. If that is the case, we can play the video or show the thumbnail, depending upon which action bar item the user used. Otherwise, we show a Toast to point out the problem:

  @Override
  public boolean onOptionsItemSelected(MenuItem item) {
    int position=adapter.getCheckedPosition();

    if (item.getItemId()==R.id.play_video) {
      if (position>=0) {
        playVideo(adapter.getVideoUri(position));
      }
      else {
        Toast.makeText(this, R.string.msg_choose,
          Toast.LENGTH_LONG).show();
      }

      return(true);
    }
    else if (item.getItemId()==R.id.show_thumbnail) {
      if (position>=0) {
        showLargeThumbnail(adapter.getVideoUri(position));
      }
      else {
        Toast.makeText(this, R.string.msg_choose,
          Toast.LENGTH_LONG).show();
      }

      return(true);
    }

    return(super.onOptionsItemSelected(item));
  }

Alternatively, you might elect to disable or hide those action bar items until the user selects a row with the keyboard.

We do not need to do anything special in our code to handle the alphabetic shortcuts — those are applied by Android automatically, routing to the same onOptionsItemSelected(). In other words, whether the user chooses the action bar item via a keyboard, mouse, or touchscreen, our same code runs.

The user will find out about those shortcuts through a keyboard shortcuts helper. On Android 7.0 and higher, the system will provide one for us. Note that this helper is implemented as a system-supplied dialog-themed activity. As such, our activity is paused (as we no longer get input) but not stopped (as the helper dialog is not full-screen).

We do not need to do anything special in our code to enable the keyboard shortcuts helper on Android 7.0. However, that helper only knows about our action bar item alphabetic shortcuts, plus system-wide shortcuts. It does not know anything about the Alt-Right and Ctrl-Right shortcuts that we are handling ourselves. However, we can override onProvideKeyboardShortcuts() in our activity to add information to this dialog about our custom shortcuts:

  @Override
  public void onProvideKeyboardShortcuts(
    List<KeyboardShortcutGroup> data, Menu menu, int deviceId) {
    super.onProvideKeyboardShortcuts(data, menu, deviceId);

    List<KeyboardShortcutInfo> shortcuts=new ArrayList<>();
    String caption=getString(R.string.menu_video);

    shortcuts.add(new KeyboardShortcutInfo(caption,
      KeyEvent.KEYCODE_DPAD_RIGHT, KeyEvent.META_ALT_ON));

    caption=getString(R.string.menu_thumbnail);
    shortcuts.add(new KeyboardShortcutInfo(caption,
      KeyEvent.KEYCODE_DPAD_RIGHT, KeyEvent.META_CTRL_ON));
    data.add(new KeyboardShortcutGroup(getString(R.string.msg_custom),
      shortcuts));
  }

This is not especially well documented at this point. What seems to work is:

This gives us:

Android 7.0 Keyboard Shortcuts Helper, With Custom Info
Figure 728: Android 7.0 Keyboard Shortcuts Helper, With Custom Info

The alphabetic shortcuts from the menu appear in a group whose name matches our activity’s label. That is followed by our “Custom App Hotkeys” group. Ideally, these two groups would be merged, since both lists are fairly short and both pertain to this app. While we might be able to retrieve the existing group from the supplied list of KeyboardShortcutGroup objects and modify it, since that is not documented, that is not safe.

This shortcut helper dialog is only available on Android 7.0. For consistency, it might be nice to offer a similar helper on older devices. To do that, we need to find out when the user presses Meta-/ on those older devices, which we handle in onKeyDown():

  @Override
  public boolean onKeyDown(int keyCode, KeyEvent event) {
    if (event.getRepeatCount()==0) {
      if (keyCode==KeyEvent.KEYCODE_DPAD_RIGHT) {
        int position=adapter.getCheckedPosition();

        if (position>=0) {
          Uri videoUri=adapter.getVideoUri(position);

          if (event.isAltPressed()) {
            playVideo(videoUri);
          }
          else if (event.isCtrlPressed()) {
            showLargeThumbnail(videoUri);
          }

          return(true);
        }
      }
      else if (keyCode==KeyEvent.KEYCODE_SLASH &&
        event.isMetaPressed() &&
        Build.VERSION.SDK_INT<Build.VERSION_CODES.N) {
        new ShortcutDialogFragment().show(getFragmentManager(),
          "shortcuts");

        return(true);
      }
    }

    return(super.onKeyDown(keyCode, event));
  }

On those devices, we show a ShortcutDialogFragment, which just displays an AlertDialog with some simple text about our shortcuts:

package com.commonsware.android.kbmouse.hotkeys;

import android.annotation.TargetApi;
import android.app.AlertDialog;
import android.app.Dialog;
import android.app.DialogFragment;
import android.os.Build;
import android.os.Bundle;

@TargetApi(Build.VERSION_CODES.HONEYCOMB)
public class ShortcutDialogFragment extends DialogFragment {
  @Override
  public Dialog onCreateDialog(Bundle savedInstanceState) {
    AlertDialog.Builder builder=new AlertDialog.Builder(getActivity());
    Dialog dlg=builder
      .setTitle(R.string.title_shortcuts)
      .setMessage(R.string.msg_shortcuts)
      .setPositiveButton(android.R.string.ok, null)
      .create();

    return(dlg);
  }
}

Not every device will have a Meta key — the Pixel C’s keyboard dock, for example, lacks this key. However, we have no good way of determining if a given hardware keyboard has a Meta key. Our options would be:

Custom Copy-and-Paste

The preview of this section is unavailable right now, but if you leave your name and number at the sound of the tone, it might get back to you (BEEEEEEEEEEEEP!).

Physical Keyboards and Focusing

The preview of this section was traded for a bag of magic beans.

Offering Mouse Context Menus

The preview of this section was whisked away by a shark-infested tornado.

Offering Tooltips

The preview of this section was eaten by a grue.

Pointer Capture

The preview of this section is being chased by zombies.