The CommonsBlog

Android's Architecture Components Version 0.8 Released

Subscribers now have access to an update to Android’s Architecture Components, known as Version 0.8, in PDF, EPUB, and MOBI/Kindle formats. Just log into your Warescription page and download away, or set up an account and subscribe!

This update expands the coverage of the Paging library:

  • A new chapter on creating a custom DataSource for use with Paging

  • A new chapter extending the original Model-View-Intent (MVI) sample to use Paging

This update also bumps all of the dependencies up to match the January 2018 release of the Architecture Components, plus makes the usual bug fixes.

The next update to this book is tentatively slated for mid-April. However, the Android “offseason” is nearly over, as developer previews of Android P (Pomegranate? Popsicle?) could show up in March. My release plans will need to adapt to what Google does.

Feb 12, 2018

Vet Your Manifest

It should go without saying that you own what you ship to users. By this, I mean that you are responsible for what goes into the app, and you are responsible for the impacts that your app has on the device and on other apps that may be on that device.

Of course, we tend to think of this a lot, but in terms of code. We test our Java, Kotlin, C, C++, etc. We test our resources, sometimes directly, sometimes indirectly (by their impacts on the UI that we test).

But, what about the manifest?

Certainly, some aspects of it wind up being tested, such as whether we have enough <uses-permission> elements, for all of the secure APIs that we happen to call. But, what are you doing to ensure that your manifest contains what you expect it to contain: nothing more and nothing less?

For example, while writing this blog post, I noticed that Android Studio 3.0.1 (and the related tool chains) can generate duplicate <uses-permission> elements through the manifest merger process. In this case, adding LeakCanary gives your APK duplicate <uses-permission> elements for READ_EXTERNAL_STORAGE. This particular problem should be benign, but at best, we are wasting a bit of space. At worst, this causes problems with some future version of Android.

For larger apps, the problem with the manifest is that it can be this “wall of XML”, even before you start thinking about manifest merger. And to illustrate that, I’ll point you to one of the more bizarre problems I’ve seen on Stack Overflow:

A developer went to install his app on a particular device, and it failed to install, due to a duplicate permission error. On Android 5.0+, two apps cannot have the same <permission> element in the manifest, for the same name, unless both apps are signed by the same signing key. So, the developer’s app collided with an app that was already installed, both defining the same custom permission.

Except the custom permission was one that the developer had created for his own app… and the collision was with Samsung’s Bixby Voice app.

Bixby, as you’re probably aware, is Samsung’s AI agent. I’m sure that it’s nice, though I have never used it. Not only do I have grave concerns over this whole area (on privacy and security grounds), but I worry that if I make Bixby angry, it might turn into a giant green rage monster.

Bixby Voice has a lot of custom permissions in its manifest. All but one are in the namespace, as you might expect for a Samsung proprietary app. Just this one permission lies outside that namespace.

So, why does Bixby need to define a custom permission matching one from a baseball card tracking app?

The leading theory: somebody copied and pasted some code out of another Stack Overflow question, where the developer had shown this custom permission as part of that question. That question has since been edited, to help prevent this sort of copy/paste issue in the future.

Copying and pasting from Stack Overflow is so common, it has been immortalized in book cover form. However, it is incumbent upon you to adapt that pasted material to local conventions… such as keeping custom permissions inside of your namespace.

In the end, this too is not a huge problem. Yes, it sucks for users of that baseball card tracking app. Yes, it sucks for that developer, who now needs to consider changing that custom permission (or perhaps just getting rid of it). In the grand scheme of things, though, this problem is not affecting lots of people.

But, you own what you ship.

The cruft in your manifest might be fine, or it might be a problem for some people, or it might wind up someday being a problem for a lot of people. It’s cruft, so get rid of it.

Modern tools make this comparatively easy. Use the “Merged Manifest” sub-tab of the manifest editor in Android Studio, to see what is going into the manifest from all sources. Double-check that with the APK Analyzer, to see the contents of the manifest in your production APK. Does everything in there need to be there? What can you get rid of that is no longer needed? What does not match the coding style rules for this project? And, for larger projects, can you automate these checks, so that nothing leaks into the merged manifest that is unexpected?

Vetting your manifest should be a part of the product release cycle, no different than all the other checks that you do to make sure that your app is “ready for prime time”. Just because it’s not executable code does not mean that you can ignore it.

NOTE: No actual giant green rage monsters were harmed in the creation of this blog post.

Feb 06, 2018

"Exploring Android" Version 0.1 Released

Subscribers now have access to the inaugural release of Exploring Android, known as Version 0.1, in PDF, EPUB, and MOBI/Kindle formats. Just log into your Warescription page and download away, or set up an account and subscribe!

As I did in 2018, with Android’s Architecture Components and GraphQL and Android, I am continuing to release new books where the subject matter is “bigger than a breadbox”.

I have offered hands-on tutorials for building an Android app for ~9 years. Originally, they were in a book titled Android Programming Tutorials. They then got folded into The Busy Coder’s Guide to Android Development. But, in general, I have rewritten the tutorials from scratch every couple of years, to better reflect the current approaches for building Android apps.

Exploring Android is the latest rewrite, pulling the tutorials back out into their own book. Partially, they are in their own book to make The Busy Coder’s Guide to Android Development smaller. But mostly, they are in their own book because, eventually, they will incorporate the Architecture Components as well.

I say “eventually” because the app created in this 0.1 release is pure “glassware”, focusing on getting the core UI set up. Version 0.2 will start to introduce the Architecture Components, including Room for saving our data. In the end, Exploring Android will create an app that looks a lot like the Model-View-Intent to-do app introduced in Android’s Architecture Components.

Stay tuned to this blog, the @CommonsWare Twitter feed, etc. for announcements about updates.

This is a brand-new book release, and so there may be more problems than normal — contact me if you encounter any issues.

Feb 05, 2018

Think Hard About @hide

Yesterday, XDA Developers wrote about a possible upcoming change to Android, whereby attempts to use hidden APIs may be blocked. Here, by “hidden”, I mean classes and members marked with the @hide pseudo-annotation.

I am not going to dive into the technical details of the change — XDA’s post has links to the relevant Git commits to accompany their analysis. Instead, here, I want to explain a bit more about @hide and what you should be doing given this potential Android change.

“These Are Not the APIs That You Are Looking For”

Some of you may not know what @hide means. The simplest analogy is to think of the Android SDK as an iceberg: the portion that you see in the JavaDocs is merely the fraction that is visible to you.

The framework classes — Activity, AsyncTask, AlarmManager, and other classes that might not even begin with A — are ordinary Java classes. They are not magic. Hence, they are subject to the same rules as any other Java class in terms of visibility. Classes can be:

  • public, which makes up most of what you see in the JavaDocs
  • private
  • “package-private” (i.e., no particular scope notation, and so visible only to classes in the same Java package)

Members, such as fields and methods, can also be protected, meaning that they are visible to subclasses but otherwise are inaccessible.

In an ideal world, that would be all that is needed.

However, the implication is that everything that is public and protected is part of the visible API. In some cases, Android’s framework developers had classes and members that needed public or protected visibility for internal technical reasons… but where they did not want those classes and methods to be part of the visible API.

That is where @hide comes into play.

When you have compileSdkVersion 27 in your build.gradle file, what that really tells the build system is to:

  • Go into the $ANDROID_SDK/platforms/android-27/ directory (where $ANDROID_SDK is wherever your Android SDK is installed),

  • Find the android.jar file in that directory, and

  • Add that JAR to the compile-time classpath

When javac compiles your own Java code, it resolves all references to framework classes and methods based on what is in that android.jar file. However, that JAR is not packaged into your app, the way that your dependencies are. Instead, at runtime, a JAR file with the same visible API is linked into your process.

There are two key differences between the android.jar that you compile against and the replacement JAR that gets used at runtime:

  • The android.jar that you compile against does not have the real method implementations

  • The android.jar that you compile against does not have anything marked with @hide

So, in the source code for Android itself, the framework developers simply mark classes and members with an @hide string in the JavaDoc comment. The tools that package up the Android SDK strip those classes and members out of the android.jar that you compile against. This way, the framework can have public and protected things that are not part of the Android SDK.

This gets used a lot. In the Android 8.1 version of Activity, @hide appears 45 times… and that’s just one class.

@hide and Seek

On the whole, Android developers do not take “no” for an answer. So, when they are told that they cannot use certain things, some will try to find ways around the restriction. If the member marked with @hide is a constant, some developers will copy that constant into their own code. For everything else, there is reflection, such as Class.forName() and getDeclaredMethod() and so forth.

There are lots of recipes floating around that use reflection to access things that are marked with @hide, from disabling mobile data and ending phone calls to tweaking TabWidget and forcing icons to display in the overflow menu.

You May Not Like What You Find, and You May Not Find What You Like

Using these approaches has always been risky. On the whole, Google does an admirable job of keeping the visible API stable over the years. A lot of the angst that you hear about new Android versions is where Google winds up making changes that affect the visible API. However, the same protections do not hold for things marked with @hide.

As a result, problems abound:

  • The hidden API might be removed in a future Android version

  • The hidden API might be altered in a future Android version, such as changing method signatures or field types

  • Individual device manufacturers might remove or alter the hidden API, affecting that manufacturer’s devices (or some of them)

These can happen at any point, even without a full ban on accessing hidden APIs, as the XDA analysis suggests.

On the whole, I have been steering developers away from these approaches wherever I can, as the risk frequently is greater than the reward.

Looking For @hide In All the Wrong Right Places

So, with all that in mind, what should you be doing?

Bear in mind that while using hidden APIs has never been a great solution, we have only some hints at possible changes in how those hidden APIs behave. We are likely to get the first developer preview of the next major Android release in a few months, and we will get more clarity then (I hope).

However, what is worth doing in the short term is knowing where in your code you are using this sort of trick, and make sure that your test suite adequately covers those uses. That way, no matter what the reason is why the hidden API stops working, you will be able to detect it, at least in lab testing.

For your own code, simply scanning all the places where you are using Java reflection may be sufficient. Search for import statements that pull in java.lang.reflect.* classes (e.g., Method), or search for key reflection methods like Class.forName(). You can do the same for open source libraries that your app happens to use.

Then, have a rough-cut plan for what your fallback will be if the hidden API is no longer usable for whatever reason. If it makes sense, execute that plan now, as if you have a good workaround for using a hidden API, that is likely to be a better long-term solution than what you have now. But, if the hidden API is so useful that you want to continue risking it, have a plan for what you will do if and when that hidden API comes unavailable.

This is one of the reasons why I steer developers away from hidden APIs: if you become dependent upon them, their loss might affect your users. Users do not understand the nuances between hidden APIs and regular APIs. Users just know that your app no longer supports some feature, one that they had been using or they read about in a review. Your plan for dealing with the loss of the hidden API may be as much about explaining what happened to your users as it is about changing your code to avoid crashing on the missing APIs.

Again, it is entirely possible that the Android changes pointed out by XDA will have no practical impact on your apps, if those changes even make it into Android at all. But the commits that XDA shows suggest an increased risk in using hidden APIs, and so this is a fine time for you to audit your use of them.

Jan 18, 2018

Android's Architecture Components Version 0.7 Released

Subscribers now have access to an update to Android’s Architecture Components, known as Version 0.7, in PDF, EPUB, and MOBI/Kindle formats. Just log into your Warescription page and download away, or set up an account and subscribe!

This update focuses on GUI architectures, with two new chapters. One covers GUI architectures in general, reviewing the major MV* candidates. The other takes a deep dive into an implementation of Model-View-Intent (MVI) on Android.

Also, the chapter on Paging is up-to-date for the alpha4-1 release.

Jan 16, 2018

Older Posts