Reading, Writing, and Debugging Storage

Once you have a File object that points to a location of interest to you, everything else works like standard Java/Kotlin disk I/O. You can use streams (e.g., FileInputStream), readers/writers (e.g., OutputStreamWriter), and so on. However, disk I/O may be slow, so you want to do that I/O on a background thread of some form.

For debuggable apps — the sort that you normally run from Android Studio — the IDE will give you somewhat greater ability to view files than ordinary users get. There is a “Device File Explorer” tool in Android Studio, by default docked on the right edge. If you have a device (or emulator) available, it will give you a file explorer for that device that will let you examine the major storage locations of interest to you.

Introducing the Sample App

In the chapter on permissions, we saw a few code snippets from the ContentEditor sample module in the Sampler and SamplerJ projects. This app implements a tiny text editor, where the user can edit text from various sources.

The app mostly is a large EditText widget for typing in some text. Above it is a TextView that we use to show a Uri representation of the content being shown in the EditText. By default, this will open up a file in the app’s portion of internal storage:

The Save button will let you save any changes that you made to the file. The overflow menu lets you switch the editor to different files or to content that you create using the Storage Access Framework.

Specifying the Location

Our MainActivity manages the options menu, including the “Save” item and the overflow. It, therefore, indicates where we should be loading our text from (or saving it to). That is triggered by onOptionsItemSelected():

  @Override
  public boolean onOptionsItemSelected(MenuItem item) {
    switch (item.getItemId()) {
      case R.id.loadInternal:
        loadFromDir(getFilesDir());
        return true;

      case R.id.loadExternal:
        loadFromDir(getExternalFilesDir(null));
        return true;

      case R.id.loadExternalRoot:
        requestPerm.launch(Manifest.permission.WRITE_EXTERNAL_STORAGE);
        return true;

      case R.id.openDoc:
        openDoc.launch(new String[] { "text/*" });
        return true;

      case R.id.newDoc:
        createDoc.launch(FILENAME);
        return true;

      case R.id.save:
        motor.write(current, binding.text.getText().toString());
        return true;
    }

    return super.onOptionsItemSelected(item);
  }

  override fun onOptionsItemSelected(item: MenuItem): Boolean {
    when (item.itemId) {
      R.id.loadInternal -> {
        loadFromDir(filesDir)
        return true
      }

      R.id.loadExternal -> {
        loadFromDir(getExternalFilesDir(null))
        return true
      }

      R.id.loadExternalRoot -> {
        requestPerm.launch(Manifest.permission.WRITE_EXTERNAL_STORAGE)
        return true
      }

      R.id.openDoc -> {
        openDoc.launch(arrayOf("text/*"))
        return true
      }

      R.id.newDoc -> {
        createDoc.launch(FILENAME)
        return true
      }

      R.id.save -> {
        current?.let { motor.write(it, binding.text.text.toString()) }
        return true
      }
    }

    return super.onOptionsItemSelected(item)
  }

In the case of the loadInternal and loadExternal items, we call a loadFromDir() function providing getFilesDir() and getExternalFilesDir(null), respectively. loadFromDir() just clears out the EditText and passes the location to our MainMotor and its read() function:

  private void loadFromDir(File dir) {
    binding.text.setText("");
    motor.read(Uri.fromFile(new File(dir, FILENAME)));
  }

  private fun loadFromDir(dir: File?) {
    binding.text.setText("")
    motor.read(Uri.fromFile(File(dir, FILENAME)))
  }

However, we pass the location to read() as a Uri, not a File. You can use Uri.fromFile() to create a Uri representation of a file. This is perfectly fine within an app. However, if you try passing such a Uri to another app, such as via an ACTION_VIEW Intent, you will get a FileUriExposedException on Android 7.0+. You can use FileProvider to get a safe Uri to pass to another app, as we will see later in this chapter.

For the loadExternalRoot item, we will wind up calling loadFromDir() providing Environment.getExternalStorageDirectory() as the location. However, this requires permission from the user. So, in addition to having the <uses-permission> element in the manifest, we need to check for this permission at runtime, asking for it if we do not have it already, as we saw in the chapter on permissions, using ActivityResultContracts.RequestPermission:

  private final ActivityResultLauncher<String> requestPerm =
    registerForActivityResult(new ActivityResultContracts.RequestPermission(),
      new ActivityResultCallback<Boolean>() {
        @Override
        public void onActivityResult(Boolean wasGranted) {
          if (wasGranted) {
            loadFromDir(Environment.getExternalStorageDirectory());
          }
          else {
            Toast.makeText(MainActivity.this, R.string.msg_sorry, Toast.LENGTH_LONG).show();
          }
        }
      });

  private val requestPerm = registerForActivityResult(ActivityResultContracts.RequestPermission()) {
    if (it) {
      loadFromDir(Environment.getExternalStorageDirectory())
    } else {
      Toast.makeText(this, R.string.msg_sorry, Toast.LENGTH_LONG).show()
    }
  }

For the openDoc and newDoc items, we use ActivityResultContracts.OpenDocument and ActivityResultContracts.CreateDocument, respectively:

  private final ActivityResultLauncher<String[]> openDoc =
    registerForActivityResult(new ActivityResultContracts.OpenDocument(),
      new ActivityResultCallback<Uri>() {
        @Override
        public void onActivityResult(Uri uri) {
          binding.text.setText("");
          motor.read(uri);
        }
      });

  private final ActivityResultLauncher<String> createDoc =
    registerForActivityResult(new ActivityResultContracts.CreateDocument(),
      new ActivityResultCallback<Uri>() {
        @Override
        public void onActivityResult(Uri uri) {
          binding.text.setText("");
          motor.read(uri);
        }
      });

  private val openDoc =
    registerForActivityResult(ActivityResultContracts.OpenDocument()) { uri ->
      binding.text.setText("")
      uri?.let { motor.read(it) }
    }

  private val createDoc =
    registerForActivityResult(ActivityResultContracts.CreateDocument()) { uri ->
      binding.text.setText("")
      uri?.let { motor.read(it) }
    }

In these cases, we get a Uri from the Storage Access Framework, so we do not need to convert anything into a Uri. And, in our case, we treat both the same: clear the EditText and pass the Uri to read():

In the end, no matter which of the options the user chooses, we pass a Uri to the MainMotor to read. Also, when the user clicks the save item, we pass that Uri and the current contents of the EditText to a write() function on the MainMotor.

Reading from the Location

Eventually, the read() call on MainMotor turns into a read() call on TextRepository.

Java

In Java, this creates and returns a LiveStreamReader implementation of LiveData that reads in the file and returns a StreamResult:

  private static class LiveStreamReader extends LiveData<StreamResult> {
    private final Uri source;
    private final ContentResolver resolver;
    private final Executor executor;

    LiveStreamReader(Uri source, ContentResolver resolver, Executor executor) {
      this.source = source;
      this.resolver = resolver;
      this.executor = executor;

      postValue(new StreamResult(true, null, null, null));
    }

    @Override
    protected void onActive() {
      super.onActive();

      executor.execute(() -> {
        try {
          postValue(new StreamResult(false, source,
            slurp(resolver.openInputStream(source)), null));
        }
        catch (FileNotFoundException e) {
          postValue(new StreamResult(false, source, "", null));
        }
        catch (Throwable t) {
          postValue(new StreamResult(false, null, null, t));
        }
      });
    }

    private String slurp(final InputStream is) throws IOException {
      final char[] buffer = new char[8192];
      final StringBuilder out = new StringBuilder();
      final Reader in = new InputStreamReader(is, StandardCharsets.UTF_8);
      int rsz = in.read(buffer, 0, buffer.length);

      while (rsz > 0) {
        out.append(buffer, 0, rsz);
        rsz = in.read(buffer, 0, buffer.length);
      }

      is.close();

      return out.toString();
    }
  }

StreamResult is just an encapsulation of the state, including the text once we have it loaded:

package com.commonsware.jetpack.contenteditor;

import android.net.Uri;

class StreamResult {
  final boolean isLoading;
  final Uri source;
  final String text;
  final Throwable error;

  StreamResult(boolean isLoading, Uri source, String text, Throwable error) {
    this.isLoading = isLoading;
    this.source = source;
    this.text = text;
    this.error = error;
  }
}

Note that we use ContentResolver and openInputStream(). This not only works for a Storage Access Framework Uri but also one that we get from Uri.fromFile(). Most of TextRepository does not care what sort of Uri this is, so long as ContentResolver can open it.

Kotlin

The Kotlin version of TextRepository uses coroutines, returning a StreamResult directly from read():

  suspend fun read(context: Context, source: Uri) =
    withContext(Dispatchers.IO) {
      val resolver: ContentResolver = context.contentResolver

      try {
        resolver.openInputStream(source)?.use { stream ->
          StreamResult.Content(source, stream.reader().readText())
        } ?: throw IllegalStateException("could not open $source")
      } catch (e: FileNotFoundException) {
        StreamResult.Content(source, "")
      } catch (t: Throwable) {
        StreamResult.Error(t)
      }
    }

Also, the Kotlin version of StreamResult is a sealed class representing the loading-content-error state:

package com.commonsware.jetpack.contenteditor

import android.net.Uri

sealed class StreamResult {
  object Loading : StreamResult()
  data class Content(val source: Uri, val text: String) : StreamResult()
  data class Error(val throwable: Throwable) : StreamResult()
}

Writing to the Location

Similarly, when we call write() on the motor, it routes to write() on the TextRepository.

Java

The Java version of TextRepository creates and returns a LiveStreamWriter that wraps up the disk I/O in a LiveData:

  private static class LiveStreamWriter extends LiveData<StreamResult> {
    private final Uri source;
    private final ContentResolver resolver;
    private final Executor executor;
    private final String text;
    private Context context;

    LiveStreamWriter(Uri source, ContentResolver resolver, Executor executor,
                     String text, Context context) {
      this.source = source;
      this.resolver = resolver;
      this.executor = executor;
      this.text = text;
      this.context = context;

      postValue(new StreamResult(true, null, null, null));
    }

    @Override
    protected void onActive() {
      super.onActive();

      executor.execute(() -> {
        try {
          OutputStream os = resolver.openOutputStream(source);
          PrintWriter out = new PrintWriter(new OutputStreamWriter(os));

          out.print(text);
          out.flush();
          out.close();

          final String externalRoot =
            Environment.getExternalStorageDirectory().getAbsolutePath();

          if (source.getScheme().equals("file") &&
            source.getPath().startsWith(externalRoot)) {
            MediaScannerConnection
              .scanFile(context,
                new String[]{source.getPath()},
                new String[]{"text/plain"},
                null);
          }

          postValue(new StreamResult(false, source, text, null));
        }
        catch (Throwable t) {
          postValue(new StreamResult(false, null, null, t));
        }
      });
    }
  }

We can just use openOutputStream() on ContentResolver, regardless of whether this is a Storage Access Framework Uri or one representing a file on the filesystem.

However, if the Uri does represent a file on the filesystem, and that file is on external storage, we want to tell the MediaStore about it. We detect this case by:

• Confirming that the scheme of the Uri is file — the scheme is the first portion of a URI, such as the https in https://commonsware.com
• Confirming that the path of the Uri starts with the root directory of external storage, to cover both our external and externalRoot scenarios

If both are true, we then call scanFile() on MediaScannerConnection, passing in:

• A Context
• An array of the paths to be indexed
• An array of the associated MIME types for each of those paths
• A callback object, or null if we do not need one (as is the case here)

Our file may not be indexed immediately, but it should be indexed much more quickly than if we did not do this bit of work.

Kotlin

The Kotlin version of TextRepository does the same work, but once again it uses coroutines:

  suspend fun write(context: Context, source: Uri, text: String) =
    withContext(Dispatchers.IO + appScope.coroutineContext) {
      try {
        val resolver = context.contentResolver

        resolver.openOutputStream(source)?.let { os ->
          PrintWriter(os.writer()).use { out ->
            out.print(text)
            out.flush()
          }
        }

        val externalRoot =
          Environment.getExternalStorageDirectory().absolutePath

        if (source.scheme == "file" &&
          source.path!!.startsWith(externalRoot)
        ) {
          MediaScannerConnection
            .scanFile(
              context,
              arrayOf(source.path),
              arrayOf("text/plain"),
              null
            )
        }

        StreamResult.Content(source, text)
      } catch (t: Throwable) {
        StreamResult.Error(t)
      }
    }

In particular, we need to worry about the possibility that the user will use back navigation to leave the activity before our disk write is complete. When the activity is completely destroyed, its viewmodel is cleared, and that will cancel our coroutine. For a read operation, this is fine — we will not need that data anyway. But it would be impolite to fail to write the data to disk (or, perhaps worse, write only part of it) just because the user left the activity. We need a separate CoroutineScope, one that will survive past the life of the viewmodel. For that, TextRepository has appScope:

  private val appScope = CoroutineScope(SupervisorJob())

This CoroutineScope is configured with a SupervisorJob, which treats each job independently — if one job fails for some reason, other jobs will not be canceled automatically. We then use appScope (and its CoroutineContext) when launching our coroutine:

    withContext(Dispatchers.IO + appScope.coroutineContext) {

This has the net effect of ensuring that our disk writes will not be interrupted by the viewmodel being cleared.

The Motor

Our motor’s job is to work with TextRepository to load and save our text. As with most things, this is incrementally easier with Kotlin.

Java

Ideally, the activity would have a stable LiveData to observe for getting the content to display in the editor. However, our TextRepository returns a LiveData for each load request. Somehow, we need to “pour” all of those individual LiveData objects into a single LiveData that the activity observes.

The solution for that is MediatorLiveData.

MediatorLiveData can observe one or several other LiveData objects. When values change in those LiveData objects, MediatorLiveData will invoke a lambda expression that you provide. There, you can convert the value to whatever you need and update the MediatorLiveData itself based on that change. You can add and remove LiveData objects from the MediatorLiveData whenever you need.

So, MainMotor uses a MediatorLiveData as the stable LiveData that the activity observes. As we change sources of text content, we swap in a new LiveData source for the MediatorLiveData:

package com.commonsware.jetpack.contenteditor;

import android.app.Application;
import android.net.Uri;
import androidx.annotation.NonNull;
import androidx.lifecycle.AndroidViewModel;
import androidx.lifecycle.LiveData;
import androidx.lifecycle.MediatorLiveData;

public class MainMotor extends AndroidViewModel {
  private final TextRepository repo;
  private final MediatorLiveData<StreamResult> results = new MediatorLiveData<>();
  private LiveData<StreamResult> lastResult;

  public MainMotor(@NonNull Application application) {
    super(application);

    repo = TextRepository.get(application);
  }

  MediatorLiveData<StreamResult> getResults() {
    return results;
  }

  void read(Uri source) {
    if (lastResult != null) {
      results.removeSource(lastResult);
    }

    lastResult = repo.read(source);
    results.addSource(lastResult, results::postValue);
  }

  void write(Uri source, String text) {
    if (lastResult != null) {
      results.removeSource(lastResult);
    }

    lastResult = repo.write(source, text);
    results.addSource(lastResult, results::postValue);
  }
}

Note that we do not have a separate view-state class — instead, we just pass StreamResult along to the activity. That works in this case because we had no conversions that we needed to perform on the data from the repository.

Kotlin

The Kotlin version is simpler, using MutableLiveData and viewModelScope for getting the coroutine results over to LiveData and from there to MainActivity:

package com.commonsware.jetpack.contenteditor

import android.app.Application
import android.net.Uri
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.LiveData
import androidx.lifecycle.MutableLiveData
import androidx.lifecycle.viewModelScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch

class MainMotor(application: Application) : AndroidViewModel(application) {
  private val _results = MutableLiveData<StreamResult>()
  val results: LiveData<StreamResult> = _results

  fun read(source: Uri) {
    _results.value = StreamResult.Loading

    viewModelScope.launch(Dispatchers.Main) {
      _results.value = TextRepository.read(getApplication(), source)
    }
  }

  fun write(source: Uri, text: String) {
    _results.value = StreamResult.Loading

    viewModelScope.launch(Dispatchers.Main) {
      _results.value = TextRepository.write(getApplication(), source, text)
    }
  }
}

The Results

The exact locations of internal, external, and removable storage may vary by device.

On a typical device, each app’s portion of internal storage can be found in /data/data/.../, where the ... is replaced by the application ID of the app. Unfortunately, this results in a very long list of apps, because all of the pre-installed apps get included:

If you type something into the app after you initially launch it, then click “Save”, then scroll to the /data/data/com.commonsware.jetpack.contenteditor/ directory in the Device File Explorer, you should see the files/ subdirectory that maps to getFilesDir(), and in there you should see a test.txt file that contains what you wrote:

If you right-click over a file, a context menu gives you a few operations that you can perform on that file, such as “Save As” to download it to your development machine and “Delete” to remove it from the device. “Synchronize” updates the entire tree to reflect the current contents of the device.

/sdcard in the Device File Explorer should give you a view of the root of external storage. This will include a test.txt file if you created it using the app by choosing “Load External Root” in the overflow menu, filling in some text, and clicking “Save”:

There will be an Android/ directory in the root of external storage, with a data/ directory inside of it. That is akin to the /data/data/ directory, except that it provides the list of app-specific external storage locations. And, fewer apps store data on external storage, so the list is more manageable, though it still can be rather long. If you use “Load External” in the app and save some text there, you will see a test.txt file in the app’s external storage location:

Dealing with Android 10+

As noted earlier, methods like Environment.getExternalStorageDirectory() and their associated locations do not work on Android 10 and higher by default.

To deal with Android 10, in the manifest, we have android:requestLegacyExternalStorage="true" on the <application> element:

<?xml version="1.0" encoding="utf-8"?>
<manifest package="com.commonsware.jetpack.contenteditor"
  xmlns:android="http://schemas.android.com/apk/res/android">

  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

  <application
    android:allowBackup="true"
    android:requestLegacyExternalStorage="true"
    android:icon="@mipmap/ic_launcher"
    android:label="@string/app_name"
    android:roundIcon="@mipmap/ic_launcher_round"
    android:supportsRtl="true"
    android:theme="@style/AppTheme">
    <activity android:name=".MainActivity">
      <intent-filter>
        <action android:name="android.intent.action.MAIN" />

        <category android:name="android.intent.category.LAUNCHER" />
      </intent-filter>
    </activity>
  </application>

</manifest>

This gives Android 10 the same sorts of file access as Android 9.0 had.

Android 11+, though, will not completely work with just this attribute. Partly, that is because eventually we will need to raise our targetSdkVersion to 30, at which point android:requestLegacyExternalStorage="true" no longer works. But, more importantly, even with that attribute, we do not have write access to the root of external storage. We have write access to many other places, but not that one. As a result, we need to block access to the “Load External Root” item in our overflow menu on those devices.

To do that, we make use of version-specific resources.

In res/values/, each project has a bools.xml file. By convention, this contains <bool> resources that define a boolean value:

<?xml version="1.0" encoding="utf-8"?>
<resources>
  <bool name="isPre11">true</bool>
</resources>

Here, we define isPre11 to be true.

Each project also has a res/values-v30/ directory. This contains resources are only relevant on API Level 30+ devices, where API Level 30 corresponds to Android 11. In there, we have another bools.xml with its own definition of isPre11, setting the value to false:

<?xml version="1.0" encoding="utf-8"?>
<resources>
  <bool name="isPre11">false</bool>
</resources>

Then, in res/menu/actions.xml, for the “Load External Root” item, we use android:enabled and point it to @bool/isPre11:

  <item
    android:id="@+id/loadExternalRoot"
    android:title="@string/menu_external_root"
    android:enabled="@bool/isPre11"
    app:showAsAction="never" />

As the name suggests, android:enabled controls whether the item is enabled or not. This, plus our dual definitions of isPre11, means that:

• On Android 11+ devices, this item will be disabled, as Android will use the res/values-v30/ edition of isPre11
• On older devices, this item will be enabled, as Android will fall back to the res/values/ edition of isPre11

Our Java/Kotlin code can remain oblivious to this distinction, simply reacting to that item if it is chosen… even if on some devices, it cannot be chosen, because it is disabled.