Think Hard About @hide

UPDATE 2018-03-01: The restrictions covered in this blog post are officially coming to Android P

Yesterday, XDA Developers wrote about a possible upcoming change to Android, whereby attempts to use hidden APIs may be blocked. Here, by “hidden”, I mean classes and members marked with the @hide pseudo-annotation.

I am not going to dive into the technical details of the change — XDA’s post has links to the relevant Git commits to accompany their analysis. Instead, here, I want to explain a bit more about @hide and what you should be doing given this potential Android change.

“These Are Not the APIs That You Are Looking For”

Some of you may not know what @hide means. The simplest analogy is to think of the Android SDK as an iceberg: the portion that you see in the JavaDocs is merely the fraction that is visible to you.

The framework classes — Activity, AsyncTask, AlarmManager, and other classes that might not even begin with A — are ordinary Java classes. They are not magic. Hence, they are subject to the same rules as any other Java class in terms of visibility. Classes can be:

public, which makes up most of what you see in the JavaDocs
private
“package-private” (i.e., no particular scope notation, and so visible only to classes in the same Java package)

Members, such as fields and methods, can also be protected, meaning that they are visible to subclasses but otherwise are inaccessible.

In an ideal world, that would be all that is needed.

However, the implication is that everything that is public and protected is part of the visible API. In some cases, Android’s framework developers had classes and members that needed public or protected visibility for internal technical reasons… but where they did not want those classes and methods to be part of the visible API.

That is where @hide comes into play.

When you have compileSdkVersion 27 in your build.gradle file, what that really tells the build system is to:

Go into the $ANDROID_SDK/platforms/android-27/ directory (where $ANDROID_SDK is wherever your Android SDK is installed),
Find the android.jar file in that directory, and
Add that JAR to the compile-time classpath

When javac compiles your own Java code, it resolves all references to framework classes and methods based on what is in that android.jar file. However, that JAR is not packaged into your app, the way that your dependencies are. Instead, at runtime, a JAR file with the same visible API is linked into your process.

There are two key differences between the android.jar that you compile against and the replacement JAR that gets used at runtime:

The android.jar that you compile against does not have the real method implementations
The android.jar that you compile against does not have anything marked with @hide

So, in the source code for Android itself, the framework developers simply mark classes and members with an @hide string in the JavaDoc comment. The tools that package up the Android SDK strip those classes and members out of the android.jar that you compile against. This way, the framework can have public and protected things that are not part of the Android SDK.

This gets used a lot. In the Android 8.1 version of Activity, @hide appears 45 times… and that’s just one class.

@hide and Seek

On the whole, Android developers do not take “no” for an answer. So, when they are told that they cannot use certain things, some will try to find ways around the restriction. If the member marked with @hide is a constant, some developers will copy that constant into their own code. For everything else, there is reflection, such as Class.forName() and getDeclaredMethod() and so forth.

There are lots of recipes floating around that use reflection to access things that are marked with @hide, from disabling mobile data and ending phone calls to tweaking TabWidget and forcing icons to display in the overflow menu.

You May Not Like What You Find, and You May Not Find What You Like

Using these approaches has always been risky. On the whole, Google does an admirable job of keeping the visible API stable over the years. A lot of the angst that you hear about new Android versions is where Google winds up making changes that affect the visible API. However, the same protections do not hold for things marked with @hide.

As a result, problems abound:

The hidden API might be removed in a future Android version
The hidden API might be altered in a future Android version, such as changing method signatures or field types
Individual device manufacturers might remove or alter the hidden API, affecting that manufacturer’s devices (or some of them)

These can happen at any point, even without a full ban on accessing hidden APIs, as the XDA analysis suggests.

On the whole, I have been steering developers away from these approaches wherever I can, as the risk frequently is greater than the reward.

Looking For `@hide` In All the Wrong Right Places

So, with all that in mind, what should you be doing?

Bear in mind that while using hidden APIs has never been a great solution, we have only some hints at possible changes in how those hidden APIs behave. We are likely to get the first developer preview of the next major Android release in a few months, and we will get more clarity then (I hope).

However, what is worth doing in the short term is knowing where in your code you are using this sort of trick, and make sure that your test suite adequately covers those uses. That way, no matter what the reason is why the hidden API stops working, you will be able to detect it, at least in lab testing.

For your own code, simply scanning all the places where you are using Java reflection may be sufficient. Search for import statements that pull in java.lang.reflect.* classes (e.g., Method), or search for key reflection methods like Class.forName(). You can do the same for open source libraries that your app happens to use.

Then, have a rough-cut plan for what your fallback will be if the hidden API is no longer usable for whatever reason. If it makes sense, execute that plan now, as if you have a good workaround for using a hidden API, that is likely to be a better long-term solution than what you have now. But, if the hidden API is so useful that you want to continue risking it, have a plan for what you will do if and when that hidden API comes unavailable.

This is one of the reasons why I steer developers away from hidden APIs: if you become dependent upon them, their loss might affect your users. Users do not understand the nuances between hidden APIs and regular APIs. Users just know that your app no longer supports some feature, one that they had been using or they read about in a review. Your plan for dealing with the loss of the hidden API may be as much about explaining what happened to your users as it is about changing your code to avoid crashing on the missing APIs.

Again, it is entirely possible that the Android changes pointed out by XDA will have no practical impact on your apps, if those changes even make it into Android at all. But the commits that XDA shows suggest an increased risk in using hidden APIs, and so this is a fine time for you to audit your use of them.

— Jan 18, 2018