The Storage Situation: External Storage

UPDATE 2017-11-14: This is the older edition of this blog post — I suggest that you click through and read the newer one.

There is a lot of confusion regarding Android’s storage model. That confusion has only increased with Android 4.4’s changes to that storage model. There are countless StackOverflow questions and the like where they clearly do not quite grok the various pieces of Android’s storage model.

This is the second post in a five-part series covering this storage model, to help clarify what is going on. Today, we will look at “external storage”… including its various definitions.

What Your Users Think “External Storage” Means

Many Android device users will have no idea what “external storage” means. There is nothing in the device UI that will necessarily use that term.

Users who visit Settings > Storage may see only “Internal Storage”:

Or, they may see both “Internal Storage” and “SD Card”:

Hence, at best, your users will think that an SD card is external storage. That’s not quite right. Except on Android 4.4+, in which case it is only sorta right.

(yes, this is all rather confusing)

What Google Thinks “External Storage” Means

The Android SDK documentation has this to say in terms of a definition of “external storage”:

Every Android-compatible device supports a shared “external storage” that you can use to save files. This can be a removable storage media (such as an SD card) or an internal (non-removable) storage. Files saved to the external storage are world-readable and can be modified by the user when they enable USB mass storage to transfer files on a computer.

The vast majority of what has been written so far, in the documentation and elsewhere, about external storage was for the pre-4.4 world. In those halcyon days of yesteryear, there was a single volume known as “external storage”, and it was effectively defined as “the stuff that shows up when the user plugs their device into a computer using a USB cable”. Even that wasn’t completely accurate, as some manufacturers would also allow access to their devices’ removable media via the USB cable as well. And Android 4.4 has added yet more wrinkles in terms of removable media… which is why removable media gets its own blog post tomorrow.

For the purposes of this blog post – and to line up with what most other written material will refer to – “external storage” is defined as the directory tree returned by Environment.getExternalStorageDirectory().

The Many Places Where External Storage Is Stored

External storage, like a tumbleweed, goes where the wind blows it.

In Android 1.x and most 2.x devices, external storage was generally some form of removable media, typically a micro SD card. More importantly, for all Android 1.x and 2.x devices, external storage was a separate partition with a separate filesystem. While a few Android 2.3 devices elected to use on-board flash for external storage, that was still a separate partition from the partition that held internal storage. As a result, we wound up with devices that might advertise having several GB worth of storage, but that storage tended to be mostly for external storage, as the partitions could not be resized by ordinary users.

Android 3.0 changed this around, allowing internal and external storage to each be on the same partition, just in separate directory trees. This provided a lot more flexibility for users, as now there was no artificial hard distinction between space for internal storage and space for external storage. Device manufacturers still could elect to have external storage be a separate partition, or even be on removable media, but typically they did not.

The Many Paths Under Which External Storage is Stored

For us as developers, the actual path under which this magical “external storage” was accessed varied over the years, from /sdcard, to locations under /storage, to eventually the current /mnt/shell/emulated/0. And just as secondary users of an Android 4.2+ tablet get their own internal storage, they get their own external storage, with its own root directory.

Hence, as I mentioned yesterday:

NEVER HARDCODE PATHS

Use various methods to get the base directory to work from… though this too has gotten more complicated over the years.

In the beginning, everyone used Environment.getExternalStorageDirectory(), which pointed to the root of external storage. This led to external storage being just a big basket of random content.

Later, Google offered more organization:

• getExternalFilesDir() and getExternalCacheDir() on Context, pointing to an application-specific directory on external storage, one that would be deleted when the app is uninstalled

• Environment.getExternalStoragePublicDirectory(), for centralized places to store well-known file types, like photos and movies

Note that the Context methods have plural forms on Android 4.4+ (getExternalFilesDirs() and getExternalCacheDirs()), which ties into removable media, which we will get into more tomorrow.

External Storage and Permissions

Just as the location – physically and logically – of external storage keeps changing, so does our ability to work with it.

Originally, apps could do whatever they wanted.

Android 1.5 added the WRITE_EXTERNAL_STORAGE permission, which apps had to hold to be able to write files to external storage. That way, the user would be informed at install time that the app intended to modify external storage contents. Any app could still read from external storage, though.

Android 4.4 is now enforcing a READ_EXTERNAL_STORAGE permission, so you cannot even read from external storage if you do not hold some permission. Note that WRITE_EXTERNAL_STORAGE implies READ_EXTERNAL_STORAGE, so you only need one of those, not both. And, for getExternalFilesDir() and getExternalCacheDir(), you do not need either of those permissions – you can read and write in those directories with impunity. Android now has an android:maxSdkVersion attribute on <uses-permission>, specifically so that you can drop WRITE_EXTERNAL_STORAGE if you no longer need it, because you are only working with getExternalFilesDir() and getExternalCacheDir().

Poking Around External Storage

As a developer, assuming that you can find where external storage really resides for your version of Android, you have unfettered access to it from DDMS, for both emulators and production devices.

You can also use a USB cable, much like your users will use. However, bear in mind that what is presented to the USB interface is not what is on external storage… but, instead, is what has been indexed on external storage in the MediaStore. Hence, unless you take steps to ensure that new files that you create get indexed, they may not be immediately visible.

Under the covers, Android is using the Media Transfer Protocol for its USB communications. This enables a lot more flexibility than does Mass Storage Mode (a.k.a., what thumb drives use) that Android used originally. However:

• some MTP clients may cache directory listings, so even after you get your file indexed by MediaStore, an already-attached client may need to be refreshed

• I eagerly await a decent MTP solution for Linux (while Ubuntu’s support is better than it used to be, I still get squirrelly results, particularly for advanced I/O operations like “delete”)

Frequently-Asked Questions

Here are some FAQs about what the SDK refers to as external storage:

How Do I Secure My Files on External Storage?

In general, you don’t. The user has complete access to external storage, and other apps can get to external storage if they hold the right permissions.

One thing that you can do is use something like Facebook’s Conceal. This encrypts your files on external storage, but uses a generated encryption key kept on internal storage. From a security standpoint, the net effect is to make external storage closer to internal storage in terms of read access. Note, though, that Conceal cannot prevent other apps, or the user, from deleting your files on external storage, or perhaps trying to write to them and corrupting the files as a result.

What About Apps-On-SD?

I mentioned yesterday that internal storage used to be as little as 70MB.

(and that’s still not a typo)

There was a hue and a cry to allow applications to be installed to external storage, as a way of providing more space for those apps. Android 2.2 added such a capability, via the android:installLocation attribute on the root <manifest> element in your manifest file.

However, once Android 3.0 shipped, this became less useful, because internal and external storage share a common partition.

My understanding is that you can only move an app to external storage nowadays if you have a device for which external storage is still a separate partition. Personally, out of the 50+ devices in my Secret Mountain Lair, I have zero that are on Android 3.0+ that have external storage as a separate partition, so I cannot confirm this behavior.

Now, of course, the hue and cry have moved to putting apps on removable media, as power users are running out of room on 32GB devices for all their apps. It is conceivable that Google will extend android:installLocation to support removable media, though I am not exactly holding my breath.

The Rest of the Posts

The entire blog post series covers:

• Internal storage, as defined by users and Google
• External storage, as defined by users and Google
• Removable storage, and where things have gone off the rails
• Where Google went wrong with all of this
• Where developers went wrong with all of this