The Death of External Storage: What? And What Now?

UPDATE 2019-06-08: With Q Beta 4 out, the story is now much simpler. I am leaving the original post here for historical reasons.

Under the fairly innocuous title of “scoped storage”, Google has announced that external storage, as Android developers have used it, effectively is dead, for Android Q and onwards.

I have been hoping that I’m wrong about this, but so far my tests bear out what the docs say. Also, the developers involved in this issue seem to have similar concerns.

I am going to spend some time on this over the course of the week, as this change needs some attention. Today, I am going to focus on what has changed and what you are now expected to do.

READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE are not merely deprecated, but are ineffective:

• Apps targeting Q cannot request them — those permissions will be treated as if the user previously rejected them with “don’t ask again”, and the app will be unable to access arbitrary locations on external or removable storage

• Apps targeting previous versions can request them, but they do not actually grant any rights, and the app will be unable to access arbitrary locations on external or removable storage

Probably.

I say “probably” in part because Google, in its infinite wisdom, decided to hide this behavior in Q Beta 1 devices and emulators. Instead, out of the box, things work as they have for the past few releases. You have to run an adb command to get Android to behave in the documented fashion:

adb shell sm set-isolated-storage on

Perhaps this is because Google is still debating the wisdom of all of this and might elect to punt on this feature until Android R, or Q MR1, or something.

Perhaps this is because Google is considering implementing the restriction, but limiting it to apps that target Q. This does not excuse hiding the behavior as they did, but it would blunt the impact, as many developers would have a bit over a year to adapt their apps. Certainly, that would seem to be the hope of the folk starring this issue.

My fear is that perhaps this is some sort of epic-level trolling by Google. Every week that passes where developers do not realize that they need to adapt their apps is one less week they have to do that work, before Q ships as Android 10.0.

I also say “probably” because I found a loophole big enough to drive a truck through. I assume that it is a bug, and so I filed a security issue for it. We will see what happens. If it turns out that this loophole actually is intended behavior, then the situation improves a fair bit. Once I get clarity on this — assuming somebody actually looks at the issue — I will write about it.

Note that there is one class of apps that is temporarily immune to these effects: apps that are already installed on the device at the point when the device gets upgraded to Android Q. Those apps get the same basic behavior as before. If the app is uninstalled and reinstalled, it is treated as a fresh install. And, I suspect that if the app is updated to one that targets Q, the app will then be subject to normal Q behavior.

On the surface, this may sound great. After all, it means that even if you do nothing, on Day 1 of the Android 10.0 rollout, all your users will be unaffected.

But if you are getting a steady stream (or flood) of new users, some of them will have Q devices, and those fresh installs do not get “grandfathered” in the way existing installs do. So, you still will have unhappy Q users, just perhaps fewer of them.

There are three major ways that you can work with external and removable storage given these limitations.

First, you can still use all the external-storage methods on Context, such as getExternalFilesDir() and getExternalCacheDir(). These work as they have since their introduction. They do not require any permissions, and you have ordinary filesystem access. However, you only have access to a few specific directories.

Second, you can use the Storage Access Framework. Primarily, that’s ACTION_OPEN_DOCUMENT, ACTION_CREATE_DOCUMENT, and ACTION_OPEN_DOCUMENT_TREE. These give you the Android equivalent of “open-file”, “new-file”, and “choose-directory” dialogs that you may be used to from other programming environments. These have the advantage of not only supporting external and removable storage, but also supporting third-party document providers, such as Google Drive. The big disadvantage is that you are not working with files anymore — working with Uri and ContentResolver limits your flexibility. I have written a lot about the Storage Access Framework over the years — here is a FAQ-style blog post on the subject.

Third, you may be able to use the MediaStore. What works there depends on a number of factors:

• If your targetSdkVersion is 28 or below, you can query the MediaStore without issue and can find whatever content it has indexed.

• If your targetSdkVersion is Q, by default, your app can only see things in the MediaStore that your app put there in the first place, such as by manually insert()-ing entries into the MediaStore and writing content to the returned Uri. However, your app cannot see content that the user put on the device by other means, whether that is another app, a desktop file manager, or anything else.

• If your targetSdkVersion is Q, and you hold ROLE_MUSIC or ROLE_GALLERY in the new roles system, your app can work with the MediaStore as if its targetSdkVersion is lower, but only for the type of content associated with the role. For other types of content, the role does not help, and you are limited to only having access to your own content. However, only one app can hold a role at a time. Unless your app is extremely important to the user, or legitimately is a full local music player or gallery-style app, the role solution is unlikely to help.

UPDATE 2019-04-05: The documentation no longer cites roles, so it is unclear if those are going away. However, there are READ_MEDIA_AUDIO, READ_MEDIA_IMAGES, and READ_MEDIA_VIDEO permissions that you can request. These are dangerous permissions, so you request them the same way you might have requested READ_EXTERNAL_STORAGE. However, they only grant you access to the MediaStore for the associated content type; you still do not get filesystem access.

And, AFAIK, that’s it.

Tomorrow, I’ll explore a bit more about what your options are if you absolutely, positively, have to have access to a file on external storage.