The CommonsBlog

Android O Background Limitations: Not Just for targetSdkVersion 'O'

In Monday’s post on ODP2, I wrote:

The biggest concern is “Even for apps not targeting O, users can use the Settings screen to enable background execution limits.” This implies that all developers need to concern themselves with the changes in Service behavior, the semi-ban on implicit broadcasts, and the changes in background location behavior. Previously, this was limited to developers who jumped to targetSdkVersion of O or higher. However, I cannot find where in Settings that the user can make this change. I cannot even find it in Developer Options. So, it is unclear how much of an issue this is.

Yesterday, Oasis Feng pointed out that the place in Settings where the user can apply background limitations to apps is in the Battery screen, for apps that are listed there as big battery consumers.

(of course, it’s not every app, as the Google App does not have that option)

From the standpoint of whether or not users toggling this setting will become common, this Settings location is not bad. Users will have limited ability to apply the background limitations to apps. However, it is possible, and so developers who are not planning on shipping a targetSdkVersion of O right away need to test this scenario.

However, developers cannot use the Settings app to apply this limitation, unless their app happens to be in the battery blame list, which ideally doesn’t happen. So, I updated my issue to request some instructions for how developers can toggle this by other means.

And, today, I received a response.

Apparently, developers can do the same thing that users can, but to any app, via the undocumented adb shell appops command:

adb shell appops set RUN_IN_BACKGROUND ignore

(where is your app’s applicationId)

I say “apparently”, as I have no good way of confirming that this “appop” is the same as what users can do from the battery blame list, but I’ll take it on faith that it is the same thing.

Based on my testing, the behavior of an app with targetSdkVersion of O and an app with a lower targetSdkVersion but affected by this appop is slightly different:

  • In either case, you cannot receive broadcasts of implicit Intent objects via manifest-registered receivers

  • In either case, a background unbound service (i.e., started via startService()), will be stopped after ~1 minute

  • In the targetSdkVersion 'O' case, you will not be able to start a service via startService() after ~1 minute (“Background start not allowed”), but in the appop case, you still can start services in this state

The argument about why this is not documented anywhere is:

We are not generally advertising this for development because it does not behave the same in many important ways as an app that is targeting O, so you need to actually target O to test your app against background check. (And generally we are not wanting to encourage developers to mess around with how broken their app may be with the background app op restriction imposed on it, but instead target O so they work within the new background limits and don’t need to worry about the user trying to do this to them.)

However, developers routinely do not move to the latest targetSdkVersion as soon as it is released. For example, here are some better-known apps from my “daily driver” device that are not on targetSdkVersion of 25 (the now-current latest version):

  • Adobe Reader: 24
  • Amazon Kindle: 24
  • Barcode Scanner: 19
  • Fastmail: 21
  • Foxit PDF: 22
  • Mozilla Firefox: 23
  • TripIt: 23
  • United Airlines: 21
  • VLC: 23
  • Vonage: 23

And so on.

Now, plenty of app developers do aim for the latest targetSdkVersion. But it is certainly not all. Google should be aware of this; I have to imagine that they have metrics on the state of apps in the Play Store.

An attitude of “we don’t want to document this because developers should always use the latest targetSdkVersion” is unrealistic and, IMHO, unfortunate.

So — once again, IMHO — here’s what you should do:

  • If your app will ship with a targetSdkVersion of 26 (or whatever O turns into) shortly after Android O ships, hopefully you are already testing how your app behaves with the background limitations, and so you know how it will affect your app.

  • If you plan on sticking with a lower targetSdkVersion for a while, and you are really really sure that your app will not show up on the battery blame list, and you want to ignore the background limitations for now, that’s your decision to make.

  • If, however, you plan on sticking with a lower targetSdkVersion and your app does tend to consume a fair bit of battery life, you should test your app with the adb shell appops command cited above. If nothing else, you can identify the likely symptoms that users will experience if they limit your background work through the Battery screen in Settings. That way, if you get customer service calls/emails/texts/Play Store comments/candygrams about those symptoms, you can better advise users about what to do.

Meanwhile, the War on Background Processing continues.

May 24, 2017

Random Musings on the O Developer Preview 2

Each time Google releases a new developer preview, I rummage through the API differences report and the high-level overviews, to see if there are things that warrant more attention from developers, with an emphasis on mainstream features that any developer might reasonably use. I also tend to emphasize things that may not get quite as much attention, because they are buried in the JavaDocs.

So, here are my notes on O Developer Preview 2 (ODP2).

What Did I Say Last Time That Is No Longer a Concern?

Two months ago, I published my random musings on ODP1.

Of note, my concerns about multi-display testing were somewhat abated by realizing that the simulated secondary displays handled it well.

Otherwise, what I had in that original post is mostly still relevant.

What’s Interesting in the Release Notes

Each developer preview has its section of the release notes. In the case of ODP2:

  • The biggest concern is “Even for apps not targeting O, users can use the Settings screen to enable background execution limits.” This implies that all developers need to concern themselves with the changes in Service behavior, the semi-ban on implicit broadcasts, and the changes in background location behavior. Previously, this was limited to developers who jumped to targetSdkVersion of O or higher. However, I cannot find where in Settings that the user can make this change. I cannot even find it in Developer Options. So, it is unclear how much of an issue this is.

  • The release notes state that “Developers using FCM/GCM with Android O should update their Google Play services SDK to version ≥ 10.2.1, which is available from the SDK manager in Android Studio.” That is curious, as IIRC, this raises the minSdkVersion to 14, meaning that it is even firmer official Google policy to leave Android 2.x devices in the dust. This is in the “Known Issues” portion of the release notes, so it may be that this is a temporary requirement.

  • The remaining findViewById() implementations (Activity, Dialog, etc.) have now been updated to return T rather than View, for consistency.

  • ODP1’s startServiceInForeground() on NotificationManager has been moved to startForegroundService() on Context, which is a nice simplification.

  • As was discussed at Google I|O, there are new APIs for downloading fonts on the fly from a font provider.

What’s Interesting in the “Features and APIs” Page?

For some reason, there are changes to Android that show up on the “Android O Features and APIs” page but are not mentioned in the release notes.

Of note:

  • JobScheduler has a new set of behaviors. A JobService can now have a work queue, with an eye towards replacing IntentService. And you can attach a ClipData object to a JobInfo, to be able to not only pass a Uri with the job, but to pass along any permissions you have for the content identified by the Uri (e.g., a Uri that you received from ACTION_GET_CONTENT or ACTION_OPEN_DOCUMENT). Also, you have new limits that you can place on jobs via JobInfo.Builder, including setRequiresBatteryNotLow() and setRequiresStorageNotLow().

  • VolumeShaper, rather than being a tool for managing 1980’s “big hair”, is an option for performing fade-in/fade-out/cross-fade sorts of audio transitions.

What’s Interesting in the API Diffs Report?

Where most of the “under the covers” stuff shows up is in seeing the diffs between the ODP1 and ODP2 APIs. There are more changes here than I would have expected.

Of note:

  • ActivityManager getRunningServices() is deprecated, and it will return only an app’s own services. This is part of the continued crackdown on apps knowing what other apps are doing.

  • Your ServiceConnection implementations will now need to add an onBindingDied() method. This will be called if the binding is permanently dead, requiring a fresh bindService() call. Now, onServiceDisconnected() represents a temporary suspension of the connection, but you may get called with onServiceConnected() again later on, when the service starts back up. AFAIK, this represents a change in onServiceDisconnected() behavior.

  • PendingIntent now has a getForegroundService() method. This works like getService(), but apparently it will use startForegroundService() instead of startService().

  • The Activity and Fragment onMultiWindowModeChanged() and onPictureInPictureModeChanged() methods now take a Configuration object as a parameter.

  • The multi-display APIs now offer an onMovedToDisplay() method on Activity, to find out when it is moved to some other display.

  • Fragment now has getLayoutInflater(), saving you from having to get one from the hosting Activity.

  • The autofill classes and methods switched from a capital F (AutoFill...) to a lowercase f (Autofill...). I eagerly await ODP3, where the lowercase f will be replaced by some other Unicode glyph that happens to resemble an f.

  • Beyond the “f”-ing changes, autofill has had other overhauls in its API. Apps can now provide hints as to the roles of views, and there are a bunch of new classes (FilLContext, FillEventHistory, SaveInfo, etc.). I suspect that my sample apps will need a significant overhaul.

  • `Bundle` now offers to store `UUID` values, though I argue that this should be in `BaseBundle`

    (UPDATE 2017-05-23: never mind, it’s being pulled from the API, apparently)

  • There is a new suite of classes for IPsec, starting with IpSecManager.

  • Chronometer now offers isTheFinalCountDown(), possibly in homage to a GEICO commercial.

What’s Interesting When You Try to Use ODP2?

The current release build of Android Studio (2.3.2) does not support ODP2, despite the fact that it had supported ODP1, and despite the fact that the current release build of Android Studio has supported then-current developer previews in the past. I find it curious that Google thinks so little about release builds of their IDE. Your code will still build and run just fine with Android Studio 2.3.2, but you will get lots of unrecognized symbol errors in the Java editor, along with related limitations (e.g., code completion).

May 22, 2017

The Trouble with Treble

As most of you are probably aware, this past weekend the world was attacked by worms.

Alas, this is not a reference to “Tremors”.

Rather, this refers to the WannaCry worm, powered by an NSA-developed Windows exploit that was released by the Shadow Brokers. While WannaCry is distinctive in terms of its reach and attack vector, in the end, it’s “just” a piece of ransomware, demanding some Bitcoin in exchange for unlocking files that the ransomware encrypted. Many organizations were affected, including some hospital chains, leaving open the possibility that WannaCry has directly caused some deaths.

There, but for the grace of $DEITY, goes Android.

After all, Android powers more devices than does Windows. While this specific worm is Windows-specific, someday, somebody is going to launch a mass attack against Android. The good news is that Android has been a huge success. The bad news is that Android has been a huge success and as a result has a target painted on its back.

Matt Blaze’s security advice has not changed after this ransomware attack: backup, patch, and turn off unneeded features. Unfortunately, Android suffers on all three:

  • Android itself has no user-accessible backup mechanism (what it claims is “backup” is really disaster recovery, and ideally ransomware would not qualify as a disaster)

  • Neither Google nor device manufacturers want to make it easy for you to turn off unneeded features, as while you might not need them, they want them on, as part of monitoring your actions

  • Android overall has had a terrible track record of devices getting any sort of security patch, let alone be able to get one 10-15 years after an OS release (akin to Microsoft releasing patches for XP, as they did this weekend)

The latter issue is where Treble comes into play.

The Base of Treble

On Friday, as WannaCry was starting its wormly wave of woe, Google announced Project Treble.

In a nutshell, Android is stealing a page from the late Firefox OS, going with a three-tier architecture:

  • Vendor code, adhering to a specification and Vendor Test Suite, powers…

  • The Android OS framework code, which adheres to the CDD specification and Compatibility Test Suite, which powers…

  • Ordinary Android apps, which rely on those specifications and test suites so that apps can run successfully on semi-arbitrary Android hardware

The idea is that isolating vendor-specific code from the core OS framework means that the framework can be updated separately.

Android O implements Treble, so all Android O devices (and beyond) should work this way.

This is clearly an improvement in terms of the “patch” step of security. With Treble, it will now be easier for manufacturers to ship updated Android bits that fix bugs, including security bugs.

I Know Not This “Vendor” Of Which You Speak

The Google blog post does not say, exactly, who the “vendor” is with respect to the Vendor Test Suite and the “vendor implementation”. One might think that a vendor is a device manufacturer.

This quote from the post, though, suggests otherwise:

With a stable vendor interface providing access to the hardware-specific parts of Android, device makers can choose to deliver a new Android release to consumers by just updating the Android OS framework without any additional work required from the silicon manufacturers:

The phrase “silicon manufacturers” usually refers to those making the chipsets that power Android devices. In other words, it usually refers to Qualcomm, though other chipset manufacturers exist. Sometimes, a device manufacturer develops their own chipsets. Samsung, for example, uses both Qualcomm chipset and develops their own (Exynos).

If a Treble Fell in a Forest, and Nobody Distributed It, Does It Really Exist?

Treble seems to assume that the “Android OS framework” — everything between the apps and the “vendor implementation” — is unchanged and can be distributed without modifications.

That is not especially realistic. Device manufacturers certainly have changed what would go in the “Android OS framework”, for everything from legacy multi-window to having custom omnipresent sliding side-panels. If anything, the silicon manufacturer code tends to be more stable, though it too can have security flaws.

If the “Android OS framework” needs customization by device manufacturers, then we are back where we started, albeit with perhaps a little less work on behalf of those manufacturers.

And, as Ron Amodeo of Ars Technica points out, that’s not especially realistic either:

Updating Android will still be costly because OEMs and carriers will still be in the loop, and, because updating a device has a negative effect on companies’ bottom lines, they’re not motivated to actually do it.

I am Altering the Deal. Pray I Don’t Alter It Any Further.

One way that Google could address this is by contract. Perhaps not for Android O, but some down-the-road version (e.g., Android R), the contract for licensing the Play Store and other Google proprietary apps might require that manufacturers not modify the “Android OS framework”. At that point, Google might have a way of shipping updates to that framework without involving device manufacturers or carriers.

Then, the question is: how well will that work?

On the one hand, we have plenty of experience in desktop OSes where a software patch breaks things. What happens if the “Android OS framework” rolls out and bricks millions of devices? Or, what happens if the “Android OS framework” bricks certain enterprise-developed apps?

On the other hand, if contract terms prohibit the likes of Samsung from shipping custom Android OS versions, will they continue to support Android? Might they support Android, but instead form some other coalition of manufacturers that eschews the Google proprietary code, enlisting existing manufacturers who do just that (e.g., Amazon).

From the standpoint of app developers, how much will this cause us to have to deal with yet more combinations of possible devices to have to work with? “OK, so this device runs Android 8.0.1 with Framework 14.2, and that particular combination needs Workaround X…”?

Hope Springs Eternal (Begging the Questions: Who is Hope, and Why Was Eternal In Jail?)

While I am being pessimistic, I do not want to discount Treble. A lot of the Treble details have yet to be revealed. Nothing of what I have written here is especially insightful — I have little doubt that the engineers who built Treble have been wrestling with these and other issues. With luck, when the full Treble details are released, some of the concerns here will be addressed in one form or fashion.

Treble is a step forward. How far of a step has yet to be determined.

May 15, 2017

RecoverableSecurityException, RIP

The O Developer Preview 1 added RecoverableSecurityException. This was an interesting class, where the exception could contain information about how to lead the user to an activity to recover from whatever the problem was. Your code catching the exception could then:

  • Display a dialog, with a button leading to the recovery activity
  • Raise a Notification, with an action leading to the recovery activity
  • Do whatever you want yourself (e.g., display a Snackbar)

However, comments on this issue and this issue indicate that RecoverableSecurityException is being removed from the Android SDK.

The specific phrasing of the comments is interesting:

The engineering team will no longer include support for this Exception in the upcoming Android O releases.

This might mean that the class will exist but be marked as @hide (“no longer include support”). Or, it could be discontinued outright. We’ll find out when the source code is released.

Regardless, I would not spend much time on RecoverableSecurityException until its final status is determined.

May 12, 2017

SYSTEM_ALERT_WINDOW, Android O, and Disappointment

Two days ago, Check Point researchers breathlessly reported on something that I had pointed out well over a year ago: apps requesting the SYSTEM_ALERT_WINDOW permission automatically get it. The fact that Check Point thinks that this is news is… disappointing.

Check Point then goes on and claims that Android O’s TYPE_APPLICATION_OVERLAY is “a new restrictive permission”. Since TYPE_APPLICATION_OVERLAY is not a permission, Check Point’s claim is… disappointing.

So, let’s get caught up on the whole SYSTEM_ALERT_WINDOW situation.

Apps requesting that permission have the right to float windows over other apps. Any app can do this on a temporary basis — a Toast is a window floating over whatever is on the screen. To do this with custom windows and arbitrary duration, you need SYSTEM_ALERT_WINDOW. Anything you install in Android that pops stuff up out of nowhere (e.g., Facebook’s legendary “chat heads”) or has some form of always-available UI (e.g., drawers that you can slide open from the screen edge) are using this permission.

Originally, this permission was dangerous, so the user would be notified about it at install time. On Android 6.0, it was converted into an appop-style permission, one that the user has to grant manually in Settings. But, starting with Android 6.0.1, apps requesting that permission would get it automatically, without any notice to the user.

IMHO, this was a very poor decision from a security standpoint. On that, Check Point and I agree.

The change that Check Point is alluding to is that the particular window “types” that you can use via SYSTEM_ALERT_WINDOW has been reduced to a single TYPE_APPLICATION_OVERLAY, and this window type cannot float over core system UI (e.g., navigation bar, status bar). Of note, this allows the user to get into Settings, where they can then be blocked from making any changes by a floating window. The fact that Check Point thinks that this is a significant improvement is... disappointing.

This minor improvement does not really address the core security issues with `SYSTEM_ALERT_WINDOW`, though other changes that Google has been making help, such as blocking apps from knowing what app is in the foreground. That *might* be just about enough to allow the access-Settings-from-the-shade approach to work, though I am skeptical. Malware authors are very creative.

{UPDATE: Mishaal Rahman of XDA-Developers pointed out that Android O apparently now has a Notification that appears when an overlay is active. On the plus side, this provides real-time information about overlays, and if overlays cannot interfere with the notification shade, it provides rapid access for turning it off. It remains to be seen how effective this will be — for example, users may ignore the Notification if they regularly use apps that employ overlays. However, it is definitely a step in the right direction, and if it works as advertised, I will be very grateful.}

And, in other respects, Android O makes the SYSTEM_ALERT_WINDOW situation worse.

While most of my book examples are just examples, one that I run on my personal device is SAWMonitor. This listens for ACTION_PACKAGE_ADDED and ACTION_PACKAGE_REPLACED broadcasts, then raises a Notification if the app in question requests SYSTEM_ALERT_WINDOW. This way, I am notified in real time if an app that I just installed has the rights to draw over other apps, so I can go into Settings and disable that “feature”.

However, I have to modify SAWMonitor to deal with the implicit broadcast ban. We can no longer listen for ACTION_PACKAGE_ADDED and ACTION_PACKAGE_REPLACED broadcasts via the manifest, and Google is fairly forceful in stating that this is intended. So, instead, I will have to have SAWMonitor use alarms or jobs to find out which new/changed apps since the last check have requested SYSTEM_ALERT_WINDOW. So now, rather than having my process run only when I happen to install or update an app, I have to have my process run rather frequently, so I can find out about SYSTEM_ALERT_WINDOW changes in near-real-time. Android’s changes to reduce process churn, in this case, will dramatically increase process churn (and battery usage), and that is… disappointing.

And, of course, ODP1 continues to automatically grant SYSTEM_ALERT_WINDOW privileges to apps, without warning the user. “Disappointing” does not even begin to do justice to my sentiments on that subject. {UPDATE: Though the real-time Notification mentioned in the update from above certainly helps a fair bit.}

May 11, 2017

Older Posts