The Storage Situation: Where Developers Go Astray

There is a lot of confusion regarding Android’s storage model. That confusion has only increased with Android 4.4’s changes to that storage model. There are countless StackOverflow questions and the like where they clearly do not quite grok the various pieces of Android’s storage model.

This is the final post in a five-part series covering this storage model, to help clarify what is going on. Today, we will look at how developers have exacerbated the problem with Android 4.4’s changes to removable storage.

If your app uses risky code, you must communicate those risks to users

Here, by “risky code” I mean stuff that has above-average chance of breaking across devices or in future Android versions, because the code uses techniques that are:

Let’s go back to the quote from Dianne Hackborn:

…keep in mind: until Android 4.4, the official Android platform has not supported SD cards at all except for two special cases: the old school storage layout where external storage is an SD card (which is still supported by the platform today), and a small feature added to Android 3.0 where it would scan additional SD cards and add them to the media provider and give apps read-only access to their files (which is also still supported in the platform today).

Android 4.4 is the first release of the platform that has actually allowed applications to use SD cards for storage. Any access to them prior to that was through private, unsupported APIs. We now have a quite rich API in the platform that allows applications to make use of SD cards in a supported way, in better ways than they have been able to before: they can make free use of their app-specific storage area without requiring any permissions in the app, and can access any other files on the SD card as long as they go through the file picker, again without needing any special permissions.

This is not news.

While it could be called out better in the documentation (e.g., an programming anti-patterns section, to line up with the design anti-patterns), we have been told that removable media was unsupported for quite some time. I personally have pointed this out in a variety of StackOverflow answers, of which this one is the highest-regarded (and just updated to reflect these 4.4 changes).

If you find yourself doing any of these things, you are using techniques that are undocumented, unsupported, and unreliable across devices and OS releases:

  • Examining Linux system files (e.g., /proc/mounts)

  • Running Linux system commands (e.g., mount)

  • Using reflection to get at classes and methods that are marked as @hide and therefore are not part of the Android SDK

  • Hard-coding references to Google-proprietary or AOSP apps, such as package names and ContentResolver Uri values

I am not saying that you cannot use these techniques. Quite obviously, you can. I am saying that either:

  • You should not use those techniques, but if you do…

  • …you need to warn your users that certain app features that depend upon such techniques may be unreliable across devices and OS releases

Here, by “warn” I don’t mean that you need to get the user to sign in blood somewhere that they understand the ramifications about this feature. At the same time, I don’t mean that you just keep the information to yourself.

There are plenty of possible analogies in the area of consumer product labeling and similar laws. For example, let’s think about the US Securities and Exchange Commission’s (SEC) requirements about risk disclosure:

Firms that are publicly traded need to publish certain information at the time they go public, plus periodic updates, with the SEC. One such annual filing is Form 10-K. As Wikipedia phrases it:

Although similarly named, the annual report on Form 10-K is distinct from the often glossy “annual report to shareholders,” which a company must send to its shareholders when it holds an annual meeting to elect directors (though some companies combine the annual report and the 10-K into one document). The 10-K includes information such as company history, organizational structure, executive compensation, equity, subsidiaries, and audited financial statements, among other information.

Specifically, section 1A of the 10-K is where a firm outlines “risk factors”, to disclose to potential investors known areas where there may be problems. Sometimes, those problems are purely internal, such as reliance upon innovative new engineering processes. Sometimes, the problems are external, such as pending litigation or “changes in regulatory climate” (i.e., lawmakers making laws). Generally, the risks are things that are relatively specific to the firm itself, or perhaps its market segment, rather than risks that are endemic to a region (e.g., natural disasters) or are otherwise broadly applied (e.g., zombie invasion).

For example, Google’s 10-K for the end of 2013 has 10 pages of risks just to the business, ranging from “intense competition” to “new technologies could block our ads”.

The reason for these disclosures is so that investors can be informed, to a degree, about the risks involved in investing in the firms. Tactically, this can be viewed as bad for the firm, as such disclosures can depress stock price. Strategically, these disclosures help with lawsuits and future regulation, by demonstrating that the firm is a “good citizen” and has reasonable transparency about the “known unknowns”. From the standpoint of public policy, such disclosures allow investors to be well-informed (if they read 10-K statements) or ill-informed (if they do not). The SEC is not trying to force investors to be well-informed, but is trying to ensure that investors have the opportunity to be well-informed if they so choose.

If you don’t want lawsuits, and you don’t want regulation to enforce “app labeling” on par with other consumer goods labeling (or, worse, civil or criminal penalties for failing to disclose known risk factors), you need to start documenting this stuff. To think that app developers live apart from the realities of other industries is sheer folly.

This does not have to be hard.

First, write documentation. Yes, this is boring. Yes, many users will not read it. Yes, do it anyway.

Second, when describing a feature that depends upon a risky technique, simply call it out (“NOTE: this feature, while wicked cool, relies upon some stuff that may not work across all devices and may not work in future versions of Android”).

Third, make sure that the documentation is reasonably easy to access, so that users cannot complain that they cannot find the documentation. If you want to host the documentation on your Web site, provide a “Help” overflow item in your activities that leads there.

Of course, some developers will resent my suggestion that they should be honest with their users, stating that it might depress sales of the app (or in-app purchases, or usage with resulting ad impressions, etc.). It probably will depress sales, at least in the short run.

Other developers will point out that users will still complain about lost features, even if the risk of that is pointed out. This too is true, as not all users will read the relevant bits of the documentation, let alone remember it. Using risky techniques is just that: risky. You are banking on the fact that the technique will stay working for enough devices and OS releases, so that you do well enough with the app to survive any downturn if and when the technique falls apart. That’s why I counsel against these techniques in the first place.

But, to look at it another way, you not telling your users about things that might break means that you are no better than Google. Google failed to communicate that they were planning on enforcing this removable media lockdown, and you were affected. You failed to communicate to your users that your removable media support was based on a series of assumptions, and your users were affected. Blaming Google for their behavior, while not taking responsibility for your own, is hypocritical, even if the scope is different (Google’s failure to communicate affects more people than did yours, unless your app is crazy popular).

There are other things that you can do that will help matters:

  • If you do not know if a certain technique is documented and supported, ask on StackOverflow or another developer support site.

  • If you feel that a certain capability is risky, but perhaps worth the risk, try to craft an open source library that wraps up the capability behind an as-opaque-as-possible API. The objective is to be able to allow many developers to share a single implementation, so that tweaks to overcome device differences can be made in one spot. Also, it provides a spot for trying to supply workarounds in case the capability is flat-out blocked, as it was here for removable media writing. And, an organized group of developers writing and implementing a library for an unsupported capability is incrementally more likely to be able to find out about upcoming changes in Android that might affect that capability, so you can begin planning what to do.

Overall, there is little question in my mind that the changes to removable media in Android 4.4 could have been handled better. However, this isn’t the first time that Google has elected to change Android behavior in ways that break undocumented and unsupported techniques. It’s also unlikely to be the last time that they do so. All I can do is hope that:

  • Google starts communicating these changes better (e.g., the SMS changes in 4.4 were handled fairly well in terms of advance notice and descriptions of what to expect)

  • Developers start paying closer attention to the ramifications of using undocumented and unsupported techniques, and if nothing else start informing their users about potential problems before they become actual feature regressions

The Rest of the Posts

The entire blog post series covers:

 

Meta


Creative Commons License