Paging and Room

The PagedFTS module of the book’s primary sample project demonstrates the use of the Paging library with Room.

This is our third generation of a full-text searching sample, originating with FTS and continuing with PackagedFTS. In those samples, we had an app that displayed the text of H. G. Wells’ “The Time Machine”, with individual paragraphs as rows in a RecyclerView. However, we loaded the entire book (or entire set of search results) at once. “The Time Machine” is short, but there are much longer books, where loading the entire book into memory would be impractical.

So, PagedFTS replaces the load-everything logic of FTS with load-pages logic, courtesy of the Paging library.

The Dependency

To use those classes, we need another dependency, one for the Paging library:

  implementation "androidx.paging:paging-runtime-ktx:3.0.1"

androidx.paging:paging-runtime-ktx gives us the code that we need to tell Room to produce paged query results and to populate a RecyclerView with the results. As with other dependencies, the -ktx suffix indicates that there are Kotlin-specific functions, such as a toLiveData() extension function that we will examine shortly.

The DAO

The revised BookStore is a bit different than what we had before:

package com.commonsware.room.fts

import androidx.paging.PagingSource
import androidx.room.Dao
import androidx.room.Insert
import androidx.room.Query

@Dao
abstract class BookStore {
  @Insert
  abstract suspend fun insert(paragraphs: List<ParagraphEntity>)

  @Query("SELECT COUNT(*) FROM paragraphs")
  abstract suspend fun count(): Int

  @Query("SELECT prose FROM paragraphs ORDER BY sequence")
  abstract fun all(): PagingSource<Int, String>

  @Query("SELECT snippet(paragraphsFts) FROM paragraphs JOIN paragraphsFts "+
      "ON paragraphs.id == paragraphsFts.rowid WHERE paragraphsFts.prose "+
      "MATCH :search ORDER BY sequence")
  abstract fun filtered(search: String): PagingSource<Int, String>
}

The insert() function is unchanged. There is a new count() function that returns the number of paragraphs in our database. But the more interesting changes affect all() and filtered():

• They no longer are suspend functions
• They no longer return a List of paragraphs, but instead return a PagingSource

PagingSource takes two generic types. The second is the type of data that we are paging through. In this case, both queries return simple String objects, so our second data type to PagingSource is String. We could use ParagraphEntity or anything else Room knows how to handle.

The first data type in the PagingSource declaration indicates to the Paging library how we are identifying the contents of pages. Room will do this based on position within a result set, so the first page might be rows 0-49, the second page might be rows 50-99, and so on. With Room, positions are indicated by an Int, so the first type that we provide to PagingSource is an Int. The Paging library supports other strategies, mostly designed around developers creating custom data sources (e.g., wrapped around a REST-style Web service), but Room uses positions.

We skip the suspend keyword because PagingSource does not perform any I/O immediately. Instead, the actual I/O is delayed until something starts using the factory. This is much like how LiveData, Single, and Flow work with our classic reactive return types.

The ViewModels

BookViewModel and SearchViewModel need to expose the book contents to their respective UI layers. In the original project, this was via a LiveData. And, the good news is that we can get a LiveData from a PagingSource — though while the code is terse, it is a bit complicated:

package com.commonsware.room.fts

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import androidx.paging.Pager
import androidx.paging.PagingConfig
import androidx.paging.cachedIn
import androidx.paging.liveData

class BookViewModel(repo: BookRepository) : ViewModel() {
  val paragraphs =
    Pager(PagingConfig(pageSize = 15)) { repo.all() }
      .liveData
      .cachedIn(viewModelScope)
}

package com.commonsware.room.fts

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import androidx.paging.Pager
import androidx.paging.PagingConfig
import androidx.paging.cachedIn
import androidx.paging.liveData

class SearchViewModel(search: String, repo: BookRepository) : ViewModel() {
  val paragraphs =
    Pager(PagingConfig(pageSize = 15)) { repo.filtered(search) }
      .liveData
      .cachedIn(viewModelScope)
}

As noted earlier, a Pager handles getting pages of data from our PagingSource. We can configure the Pager via the PagingConfig parameter — here, we say that we want to load 15 paragraphs at a time, via pageSize. We also provide the Pager with the PagingSource by way of a lambda expression. When we start trying to get data from the Pager, it will invoke that lambda expression, get the PagingSource, and start asking it for data.

That sets up the Pager itself. The liveData extension property on Pager gives us a LiveData wrapper around the Pager… or, more accurately, a LiveData wrapper around a Flow that is wrapped around the Pager. This LiveData will be for a PagingData of our PagingSource data type — in this case, String. The Pager, when it is requested to load different pages, will emit a fresh PagingData holding the results.

Because this LiveData is backed by a Flow, we need to teach the LiveData about the CoroutineScope to use, so that the Flow collector that it uses handles lifecycles properly within the coroutine system. This is handled by calling cachedIn() and providing a suitable scope — in this case, viewModelScope. If you forget to do this, while the code will compile, you will have a bad time when you try to run it.

The PagingDataAdapter

We are showing our book contents in a ParagraphAdapter. In the original project, this was a simple RecyclerView.Adapter. If you are using the Paging library, though, you will want to use PagingDataAdapter. You can hand a PagingDataAdapter a PagingData of data (e.g., a PagingData of paragraphs), and PagingDataAdapter knows how to use the PagingData API to handle loading additional pages as the user scrolls.

The good news is that with PagingDataAdapter, you do not need to bother with implementing getCount(), as PagingDataAdapter knows its PagingData and can get the overall size from it. The bad news is that you will need to provide a DiffUtil.ItemCallback to compare objects in the list. If you have used the RecyclerView version of ListAdapter, it uses the same DiffUtil.ItemCallback abstract class that PagedListAdapter does.

So, our revised ParagraphAdapter looks like:

package com.commonsware.room.fts

import android.view.LayoutInflater
import android.view.ViewGroup
import androidx.paging.PagingDataAdapter
import androidx.recyclerview.widget.DiffUtil

class ParagraphAdapter(private val inflater: LayoutInflater) :
  PagingDataAdapter<String, RowHolder>(STRING_DIFFER) {
  override fun onCreateViewHolder(parent: ViewGroup, viewType: Int) =
    RowHolder(inflater.inflate(R.layout.row, parent, false))

  override fun onBindViewHolder(holder: RowHolder, position: Int) {
    holder.bind(getItem(position).orEmpty())
  }
}

private val STRING_DIFFER = object : DiffUtil.ItemCallback<String>() {
  override fun areItemsTheSame(oldItem: String, newItem: String) =
    oldItem === newItem

  override fun areContentsTheSame(oldItem: String, newItem: String) =
    oldItem == newItem
}

Here, STRING_DIFFER is a DiffUtil.ItemCallback that can handle comparing simple strings, using content equality for areContentsTheSame() and object equality (=== in Kotlin) for areItemsTheSame().

There is also one other subtle difference: the data that we bind into the RowHolder. In the original app, this was a String. In this app, though, it is a String?. The Paging library uses null as the default placeholder data, if we try accessing parts of the list that have not yet been loaded. We call getItem() from PagingDataAdapter to return the data for a given position, and it will return a nullable type. We have to be able to cope with a null value. Here, we just use orEmpty() to convert it into an empty string, but a more sophisticated app might use a different mechanism to indicate a RecyclerView item that has been loading. Once the data for that position is available, onBindViewHolder() will be called again, so you can repopulate the UI for that item with the now-available data.

The Fragments

The only significant difference in the fragments is in setting up the ParagraphAdapter. In the original project, we would pass in the List<String> to the ParagraphAdapter constructor, so we had to wait to create the ParagraphAdapter until we were observing the LiveData that would provide our List. Now, we call a submitData() method on ParagraphAdapter, supplied by PagingDataAdapter, so we can set up the adapter ahead of time:

    super.onViewCreated(view, savedInstanceState)

    val rv = view as RecyclerView
    val adapter = ParagraphAdapter(layoutInflater)

    rv.adapter = adapter

    vm.paragraphs.observe(viewLifecycleOwner) {
      adapter.submitData(viewLifecycleOwner.lifecycle, it)
    }
  }

    super.onViewCreated(view, savedInstanceState)

    val rv = view as RecyclerView
    val adapter = ParagraphAdapter(layoutInflater)

    rv.adapter = adapter

    vm.paragraphs.observe(viewLifecycleOwner) {
      adapter.submitData(viewLifecycleOwner.lifecycle, it)
    }
  }
}

From the user’s standpoint, there is no real difference in behavior. The user can scroll through the list and read the book or the search results. In principle, on a slow device, the user might scroll fast enough that the data is not ready, and so a blank spot would appear at the bottom of the scrolled area (rows with the empty string) until the data got loaded. In the case of this app, the database I/O is small and fairly quick, so the user is unlikely to encounter any such visual hiccup with rapid scrolling. Yet, our app will be more efficient in its use of memory, by not necessarily loading the entire book at once.