How androidx.tech Worked

I have received inquiries as to how my former androidx.tech site worked. It offered a catalog of every Jetpack artifact version, including the source code. It was designed to be a workaround for the fact that Google did not offer this sort of stuff back in the day. Nowadays, maven.google.com kinda has this, insofar as it offers “source” links (and you need to inspect for yourself which is the actual artifact source and which is sample code).

So, here is how androidx.tech worked.

Step 1: Find the POMs

Google’s Maven publications have long had links to source JARs for their artifact versions in their POM files. The POM files also had information about dependencies that I wanted to display. So, I needed the POM files for all the artifacts and their versions. And, to save me having to re-download them, I wanted to store each of those POMs locally.

The simplest solution, given this need plus the need for the source code, would be to mirror maven.google.com. I could not figure out how to do this five years ago, and I may also have been concerned about how much disk space would be needed by all of this.

I created a Kotlin project, reporeader, that would traverse the artifacts in a Maven repository and pull their POMs. This involved:

Downloading https://dl.google.com/dl/android/maven2/master-index.xml and parsing the XML to get the list of artifact groups
Filtering that list to only worry about androidx. groups, as Google’s Maven repository has a lot of other stuff as well, such as Play Services
Generating the URL for the artifact group index, which involves replacing all the . characters in the artifact group with / and using that to get a group index XML URL, such as https://dl.google.com/dl/android/maven2/androidx/activity/group-index.xml for androidx.activity
Downloading and parsing that XML to get the artifacts and versions that are in that group
Generating the URL for the POM, which replaces group-index.xml in the group index URL with /$artifact/$version/$artifact-$version.pom for a given artifact name and artifact version, giving you something like https://dl.google.com/dl/android/maven2/androidx/activity/activity-compose/1.10.0-alpha03/activity-compose-1.10.0-alpha03.pom
Downloading that POM

This tool also knew to emit a roster of what was new: new artifact groups, new artifact IDs within a group, and new artifact versions. I still use this tool today, to generate the list of changes that go into posts like this one.

Step 2: Download the Sources

A POM file is an XML file with lots of interesting info:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <!-- This module was also published with a richer model, Gradle metadata,  -->
  <!-- which should be used instead. Do not delete the following line which  -->
  <!-- is to indicate to Gradle or any Gradle module metadata file consumer  -->
  <!-- that they should prefer consuming it instead. -->
  <!-- do_not_remove: published-with-gradle-metadata -->
  <modelVersion>4.0.0</modelVersion>
  <groupId>androidx.activity</groupId>
  <artifactId>activity-compose</artifactId>
  <version>1.10.0-alpha03</version>
  <packaging>aar</packaging>
  <name>Activity Compose</name>
  <description>Compose integration with Activity</description>
  <url>https://developer.android.com/jetpack/androidx/releases/activity#1.10.0-alpha03</url>
  <inceptionYear>2020</inceptionYear>
  <organization>
    <name>The Android Open Source Project</name>
  </organization>
  <licenses>
    <license>
      <name>The Apache Software License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
      <distribution>repo</distribution>
    </license>
  </licenses>
  <developers>
    <developer>
      <name>The Android Open Source Project</name>
    </developer>
  </developers>
  <scm>
    <connection>scm:git:https://android.googlesource.com/platform/frameworks/support</connection>
    <url>https://cs.android.com/androidx/platform/frameworks/support</url>
  </scm>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>androidx.activity</groupId>
        <artifactId>activity</artifactId>
        <version>1.10.0-alpha03</version>
      </dependency>
      <dependency>
        <groupId>androidx.activity</groupId>
        <artifactId>activity-ktx</artifactId>
        <version>1.10.0-alpha03</version>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>androidx.compose.runtime</groupId>
      <artifactId>runtime-saveable</artifactId>
      <version>1.7.0</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>androidx.activity</groupId>
      <artifactId>activity-ktx</artifactId>
      <version>[1.10.0-alpha03]</version>
      <scope>compile</scope>
      <type>aar</type>
    </dependency>
    <dependency>
      <groupId>androidx.compose.runtime</groupId>
      <artifactId>runtime</artifactId>
      <version>1.7.0</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>androidx.compose.ui</groupId>
      <artifactId>ui</artifactId>
      <version>1.0.1</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>androidx.core</groupId>
      <artifactId>core-ktx</artifactId>
      <version>1.13.0</version>
      <scope>compile</scope>
      <type>aar</type>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlinx</groupId>
      <artifactId>kotlinx-coroutines-core</artifactId>
      <version>1.7.3</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>androidx.lifecycle</groupId>
      <artifactId>lifecycle-runtime</artifactId>
      <version>2.6.1</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>androidx.savedstate</groupId>
      <artifactId>savedstate</artifactId>
      <version>1.2.1</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>androidx.lifecycle</groupId>
      <artifactId>lifecycle-viewmodel</artifactId>
      <version>2.6.1</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-stdlib</artifactId>
      <version>1.8.22</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>androidx.lifecycle</groupId>
      <artifactId>lifecycle-common</artifactId>
      <version>2.6.1</version>
      <scope>runtime</scope>
    </dependency>
  </dependencies>
</project>

However, it does not contain a link to the sources JAR for the artifact. Fortunately, Google tended to use a consistent naming convention of $artifact-$version-sources.jar. So, if I did not already have the sources for that artifact version, I generated the URL for it (e.g., https://dl.google.com/dl/android/maven2/androidx/activity/activity-compose/1.10.0-alpha03/activity-compose-1.10.0-alpha03-sources.jar) and downloaded it. Since JAR files are ZIP files, I could then use ordinary unZIP code to convert the JAR into a directory tree of source code.

Step 3: Generate the Site

androidx.tech was a static site, generated by Jekyll. Jekyll is Ruby code and is not designed for a site of this size, with hundreds of thousands of pages. Still, given a big enough Amazon EC2 instance, I could plow through it.

I had a separate Kotlin project, gen2, that would generate the Markdown source for the site, given the POM files and source trees. My publish script would then set Jekyll loose to generate the site, and from there it was a matter of rsync to push the changes up to the actual host.

— Nov 10, 2024