Using Repository Artifact Safelists in Gradle

With JCenter going away, we are going to be peeking more at our repositories and artifacts. After all, we need to make sure that we will continue to get the libraries that we need from their new homes, for any that were published purely to JCenter.

The timing is interesting, as “supply-chain” attacks are on the rise. It is far too easy for somebody to publish a malware-laden library and have it be picked up automatically by developers. This can even affect private artifacts in private repositories.

Ideally, while we are cleaning up our Gradle scripts, we would lock down where we get our artifacts from. The good news is that modern versions of Gradle give us somewhat better options for this. The bad news is that implementing those options is rather painful.

The problem with the default way that we declare dependencies in an Android project is that we do not say where each dependency comes from. Gradle largely disassociates artifacts (e.g., all those implementation lines) from repositories. We do not say “get this artifact from this repository”. We just list the supported repositories and desired artifacts, and Gradle fulfills our requests on its own.

And, by default, all Gradle does is use a top-down search against each repository. In other words, it uses this algorithm:

For each artifact
  For each repository in the order they are listed
    If the repository offers this artifact, download it
  End
End

There is little stopping an artifact from being offered in more than one repository, which is at the root of some of these supply-chain attacks. Depending on the order of the repositories in your Gradle script, you might get different actual artifacts than another project requesting the same artifacts but using a different repository order.

Ideally, we would say “get this artifact from this repository”. We can do that, to an extent.

Starting with Gradle 5.1, we can safelist what artifacts we get from a given repository. To do this, we add a content {} closure to the repository declaration, and in there use include...() functions to stipulate what artifacts to obtain from that repository:

jcenter {
    content {
        includeModule("org.jetbrains.trove4j", "trove4j")
    }
}

Here, we are saying that the only thing that we want to obtain from JCenter is org.jetbrains.trove4j:trove4j. JCenter will not be used for other artifacts.

The three main safelist functions are:

includeModule(), where you provide the artifact group (org.jetbrains.trove4j) and artifact ID (trove4j)
includeGroup(), to support any artifact from a specified artifact group
includeGroupByRegex(), which allows you to specify a regular expression and support any artifact group that matches that expression (e.g., includeGroupByRegex("org\\.jetbrains\\..*"))

If all of your repository declarations include one or more include...() functions, then the build should work purely off of those safelists:

The artifacts that you are using will obtained from their associated repositories and nowhere else
No artifacts that are not on the safelist will be used in your build

See this blog post for a bit more on the options.

The problem is that the safelists not only need to handle your direct dependencies, but also all of the transitive dependencies.

That can get rather lengthy.

This sample project is based on the tutorial project that we build in Exploring Android. The project requests 33 artifacts for the module, plus three classpath entries for Gradle plugins.

After adding all of the necessary include...() functions, the top-level build.gradle file is over 150 lines long, mostly involving those functions. And that is with cheating and using includeGroup() and includeGroupByRegex(), both of which are a bit less secure than includeModule().

Basically, what happens is that you add include...() calls for all of your direct dependencies, then try doing a build. In particular, adding --refresh-dependencies to a command-line build (e.g., gradle --refresh-dependencies app:assembleDebug) will confirm that Gradle can download all of your dependencies. You will wind up with a bunch of errors:

* What went wrong:
A problem occurred configuring root project 'GradleSafelist'.
> Could not resolve all artifacts for configuration ':classpath'.
   > Could not resolve org.glassfish.jaxb:jaxb-runtime:2.3.1.
     Required by:
         project : > com.android.tools.build:gradle:4.1.2 > androidx.databinding:databinding-compiler-common:4.1.2
         project : > com.android.tools.build:gradle:4.1.2 > com.android.tools.build:builder:4.1.2 > com.android.tools:sdklib:27.1.2 > com.android.tools:repository:27.1.2
      > Could not resolve org.glassfish.jaxb:jaxb-runtime:2.3.1.
         > Could not parse POM https://repo.maven.apache.org/maven2/org/glassfish/jaxb/jaxb-runtime/2.3.1/jaxb-runtime-2.3.1.pom
            > Could not resolve com.sun.xml.bind.mvn:jaxb-runtime-parent:2.3.1.
               > Could not resolve com.sun.xml.bind.mvn:jaxb-runtime-parent:2.3.1.
                  > Could not parse POM https://repo.maven.apache.org/maven2/com/sun/xml/bind/mvn/jaxb-runtime-parent/2.3.1/jaxb-runtime-parent-2.3.1.pom
                     > Could not resolve com.sun.xml.bind.mvn:jaxb-parent:2.3.1.
                        > Could not resolve com.sun.xml.bind.mvn:jaxb-parent:2.3.1.
                           > Could not parse POM https://repo.maven.apache.org/maven2/com/sun/xml/bind/mvn/jaxb-parent/2.3.1/jaxb-parent-2.3.1.pom
                              > Could not find com.sun.xml.bind:jaxb-bom-ext:2.3.1.
   > Could not resolve com.google.auto.value:auto-value-annotations:1.6.2.
     Required by:
         project : > com.android.tools.build:gradle:4.1.2 > com.android.tools.build:bundletool:0.14.0
      > Could not resolve com.google.auto.value:auto-value-annotations:1.6.2.
         > Could not parse POM https://repo.maven.apache.org/maven2/com/google/auto/value/auto-value-annotations/1.6.2/auto-value-annotations-1.6.2.pom
            > Could not resolve com.google.auto.value:auto-value-parent:1.6.2.
               > Could not resolve com.google.auto.value:auto-value-parent:1.6.2.
                  > Could not parse POM https://repo.maven.apache.org/maven2/com/google/auto/value/auto-value-parent/1.6.2/auto-value-parent-1.6.2.pom
                     > Could not find com.google.auto:auto-parent:6.

The end leaf of each branch shows a transitive dependency that is not covered by your include...() functions — in this case, com.sun.xml.bind:jaxb-bom-ext:2.3.1 and com.google.auto:auto-parent:6. After adding those, you run the test again and get a bunch of fresh errors from the transitive dependencies of the transitive dependencies. And, as the shampoo instructions state, lather, rinse, repeat, until eventually you get no more errors.

This process sucked… and this is not a big project. With luck, we can create some tooling to help make generating these safelists easier.

But, this is a more secure build than what it started with. The more important your project, the more likely it is that you are going to want to explore options like this for ensuring that your artifacts come from where you expect them to.

— Feb 20, 2021

Using Repository Artifact Safelists in Gradle

Meta