Office Hours — Today, February 18

← Tuesday, February 16

Feb 18
7:25 PM
Mark M.
has entered the room
Mark M.
turned on guest access
Steve
has entered the room
Mark M.
hello, Steve!
how can I help you today?
Steve
Hi Mark!
I have a question about how to implement persistence in an app I'm working on.
Here's the question:
View paste (6 more lines) 
When a user logs in to an app I'm developing, the back-end sends back information about the user. I want to persist the information so it can be used in other activities and when there is no network access.

The two main persistence options I'm aware of are SharedPrefereces and SQLite, and I'm trying to determine which would be more appropriate for my use case. There are three requirements:

1. The data to store are mostly a small number of primitive data types or simple reference types, but an array or an object of a derived type may also need to be stored.
2. Reading and writing the data should be as simple as possible.
3. Data that is written must be available on any subsequent read.

Here is my attempt to compare SharedPreferences and SQLite along these dimensions:

SharedPreferences
1. Works most naturally with primitive and simple reference types, but GSON can be used to store data of any type.
2. Data can be written and read from the UI thread.
3. Due to a race condition, there is no guarantee that data written to SharedPreferences will be available on any subsequent read.

...
Mark M.
wow, you type fast :-)
give me a moment to digest that
Steve
I've been practicing
sure
7:30 PM
Mark M.
"The two main persistence options I'm aware of are SharedPrefereces and SQLite" -- all persistence options are files, at the end of the day, so you are welcome to do something else using standard Java file I/O (e.g., JSON)
Steve
ok
Mark M.
with regards to SharedPreferences, technically, your first request for a given SharedPreferences does the disk read
subsequent accesses work off of a cache
so, ideally, you load the SharedPreferences in a background thread
this runs somewhat counter to your #2 in the SharedPreferences list
Steve
Ok.
Mark M.
and I am not aware that #3 is an issue
leastways, they damn well have better put synchronization in their implementation
I haven't looked at SharedPreferencesImpl in ages, though
Steve
Ok.
Mark M.
and, if needed, putting your own synchronization around the SharedPreferences access would be doable
Steve
So if i call commit() after editing sharedpreferences and wait for it to return, then I can be sure that any subsequent reads will retrieve the value that was committed?
Mark M.
well, commit() is a blocking call
blocking on disk I/O, specifically
which is why typically you use apply(), to save you forking the background thread yourself
Steve
So I can't do that in the UI thread, though I could call apply() from the UI thread. Is that right?
Mark M.
correct
and "can't" is a strong term -- "shouldn't" is a better statement
7:35 PM
Steve
Sure. But if I use apply() then there could be a race condition: if I try to retrieve the data too soon, apply() might not have completed.
Mark M.
in terms of your "subsequent reads" bit, again, they damn well better be handling that
and that gets into implementation details
the SharedPreferences contract doesn't really get into specifics about threading
of course, to a large degree, neither does SQLiteDatabase
I know SQLiteDatabase does proper synchronization, which is why we use the singleton
Steve
Ok. What I want to do is save the user data in the login activity and start another activity right away. The new activity would then need access to the user data.
Mark M.
the way that SharedPreferences works is that when you commit() or apply() the editor, three things happen:
1. the in-memory copy of the SharedPreferences is updated
2. anyone with an OnSharedPreferenceChangeListener for this SharedPreferences is told about the change
3. the XML file that is the SharedPreferences backing store is replaced with a fresh copy containing the changes
exactly what order those happen in, I can't say
Steve
Ok. I want to store the user data in the login activity and start another activity that needs access to the user data.
Mark M.
however, I would hope that if they do #3 first (so we only update the in-memory copy if the disk I/O succeeds), that read operations block waiting on the disk I/O
which in turn runs the risk of possibly introducing delays on the main application thread
with regards to starting another activity, persistence is immaterial, largely
you're either passing the data in Intent extras or using a singleton cache manager
the second activity should only be going to the persistent store if for some reason the cache is empty (e.g., process was terminated, restarted, and control went to this activity due to where we were in the task)
7:40 PM
Steve
What is the singleton cache manager - would that be e.g. a singleton DatabaseHelper?
Mark M.
well, in terms that they are both singletons, sure
have you gone through the book's tutorials, building up EmPubLite, by any chance?
Steve
I did just take a look at it.
Mark M.
OK, I'll use that as a starting point
Steve
I have code now that implements a singleton DatabaseHelper and has been working fine.
Let me back up then.
Mark M.
back up where? :-)
Steve
The problem I'm trying to solve is getting data from one activity into other activities and also persisting it so it will be available if off-line.
Mark M.
in general, those are two separate things, with light coupling
Steve
Right now I'm using intent extras, but that doesn't solve the persistence problem.
Mark M.
correct
Steve
Ok.
Mark M.
and you are getting this data from network I/O, right?
Steve
Yes.
Mark M.
how are you doing that with respect to background threads: AsyncTask, IntentService, something else?
Steve
I'm using an IntentService.
Mark M.
OK
Steve
My thinking was that since I need to persist the data anyway, that might also solve the problem of passing the data between activities.
7:45 PM
Mark M.
well, you don't want to be blocking on disk I/O constantly
the fewer bits of disk I/O, the better, from a performance and battery standpoint
Steve
Ok. So from what you're saying, it sounds like I should treat persistence separately from passing data among activities rather than looking for a single solution for both problems?
Mark M.
I would describe persistence and data-passing to be lightly coupled concerns
Steve
Ok.
Mark M.
scrapping EmPubLite at the moment... have you used any sort of social networking client app on Android
such as, say, Twitter?
Steve
No. Should I look at it?
Mark M.
no, but I am struggling to find something that you have used before that I can use to draw comparisons
OK, let's try this
you have a login activity (Activity A) and another activity (Activity B)
Steve
ok
Mark M.
you have been phrasing your problem as wanting to pass data from A->B
but that's not really your problem
7:50 PM
Mark M.
you want B to have the data
there are two basic ways of getting B that data: push and pull
passing all of the data via Intent extras is pushing the data from A to B
Steve
right
Mark M.
so long as your data is fairly small, and you ignore persistence, this works
Steve
right
Mark M.
a second possibility is for A to put the data somewhere central, and B to pull the data from that central spot
this approach offers some advantages
Steve
ok
Mark M.
first, it is easier to scale to activities C, D, E, etc.
Steve
right, and that is one of my goals
Mark M.
because either you're not passing anything much around in extras, or you are passing around an identifier of what part of the central data you want, akin to a primary key in a database
second, done right, that central spot can *also* be responsible for persistence
in effect, this is what SharedPreferences does
Steve
right, that's sort of what i was thinking
Mark M.
however, SharedPreferences is not really designed for arbitrary data storage
for example, your point #1 suggested JSON
that, in the end, puts a JSON value in an XML file, as SharedPreferences are stored in XML files
this works
it's kinda messy, but it works
plus, SharedPreferences threading is only somewhat covered in the documentation
such as our discussion earlier, over whether reads after apply() get the old or new values
7:55 PM
Mark M.
technically, we don't know
Steve
ok
Mark M.
as I don't recall the docs saying specifically one way or the other
your other approach, using SQLite, is fine, but only for the persistence aspect
doing database I/O in your IntentService to save your data is fine
but then Activities B, C, D, etc. should not be doing their *own* database I/O to read that stuff back in, ideally
as that gets slow
Steve
great. that's very helpful.
Mark M.
SQLiteDatabase + a cache = SharedPreferences, in effect
where you set things up such that if the cache does not contain the desired data, you kick off a thread to go load in that data and deliver it to whoever needed it
now, rolling allllllll the way back to your original problem statement
you wrote "a small number of primitive data types or simple reference types"
can you give me a ballpark on the total size of that, in bytes?
10 bytes? 100 bytes? 1K? 100K? 2TB?
Steve
let me take a quick look
fewer than a dozen strings that won't be more than a 100 characters each - so maybe around 1K
8:00 PM
Mark M.
OK, that's about on the edge of where I'd be comfortable shoehorning this into SharedPreferences
Steve
ok, that's very helpful.
Mark M.
going back to my earlier equation...
Steve
how do you determine that limit?
Mark M.
um, gut instinct
Steve
ok
Mark M.
bear in mind that one drawaback to SharedPreferences is that the whole thing is cached
it's one XML file, the whole thing gets read in, and the whole thing gets written out
you change one character in the SharedPreferences, and the whole file needs to be rewritten
Steve
i see
Mark M.
for larger datasets, these characteristics suck
SQLiteDatabase + thread-safe cache = SharedPreferences + either hopes or a thread-safe access layer
the benefit for larger datasets of the left-hand side of the equation is that you can cache and update pieces of the data, not just the whole thing
now, in your case, if your writes are only happening inside that IntentService, the thread safety is less of an issue
there, you can use commit(), as you are already on a background thread
and so long as you don't start up the next activity until commit() returns, you should be fine
if, however, the various activities are all reading *and writing*, that's where things can get a bit interesting
SharedPreferences is really designed around a write-occasionally/read-often model
Steve
right, and that's one of the main issues i was concerned about
8:05 PM
Mark M.
where the "occasionally" is typically from a dedicated bit of UI (e.g., PreferenceFragments)
are you planning on having your activities be modifying this data?
Steve
yes, though to a limited extent. there are some Settings activities where individual pieces of user data can be changed.
Mark M.
OK, that's not too bad, then
a typical approach is for an activity simply to refresh anything dependent upon SharedPreferences in onStart() or onResume()
that way, when control returns to the activity from any other activity (e.g., settings) that changed the data, the activity gets the fresh data
Steve
great, that's very helpful.
Mark M.
if you are expecting your data model to expand in complexity, you might consider "biting the bullet" now and putting in something based on SQLite
if you are expecting that your data model will be what you have described so far, SharedPreferences should be manageable
Steve
ok. based on this discussion, i'm thinking that i might be better off passing the data between activities using intent extras and using a SQLite database for persistence, rather than having a single mechanism for both.
8:10 PM
Steve
that would eliminate any race conditions or waiting for disk access, for one thing.
Mark M.
that too is doable
there's a ~1MB limit as to how big your Intent can get
before you hit a wall
Steve
Great to know - I don't think it will be an issue.
Mark M.
prior to that, having fat Intents is going to eat up heap space, as you may wind up with N copies of your data
once again, ~1K of data... probably survivable to have N copies
Steve
sorry, but what do you mean by "N copies"?
Mark M.
something is creating an Intent to pass the data to Activity B
so, there's one in-memory copy of your data, in the form of that Intent
as Activity B holds onto that Intent for as long as Activity B is around (i.e., not destroyed)
now, Activity B starts Activity C, passing along the data
now you have two copies
Activity C starts Activity D
now you have three copies
Steve
ok, i see
Mark M.
and this ignores the possible copies coming from the IPC that underlies each activity invocation, though those shouldn't hang around long
Steve
it sounds like i should do some calculations to get a sense of how big the data could get given the multiple copies
Mark M.
I'd firm up your estimate, at least
8:15 PM
Steve
if i use intent extras to pass data between activities, if activity A starts B and is its hierarchical parent, how do i then pass data back from B to A?
Mark M.
you don't
Steve
ok
Mark M.
this is another problem with Intent extras: they are one-way
Steve
that was another issue i was wondering about.
Mark M.
now, another way of dealing with this is to not have separate activities for all of this
Steve
ok
Mark M.
but use fragments or other techniques, with one fat activity
Steve
ok
Mark M.
that's roughly analogous to writing a single-page Web app versus your traditional page-links-to-a-page sort of Web app
then, your cache of the data is just a field in the activity
and everybody refers to that
depending on circumstances, you load up the cache from the network call or from loading from a file or database
Steve
that's back to the centralized store, then
Mark M.
yes, though it no longer has to be a singleton
Steve
ok
8:20 PM
Steve
you mentioned EmPubLite earlier. Would that be a good example for me to look at to get a sense of how i might deal with these sorts of issues?
Mark M.
the reason I brought that up actually kinda ties into this point now
EmPubLite is an ebook reader
from the user's standpoint, the main "model data" is the portions of the book
however, from a programming standpoint, the model data we really need to worry about is a JSON file containing the roster of those portions
in the case of EmPubLite, there is only one activity that needs that model data
while there are other activities, they handle separate things (displaying help, collecting settings)
and so the model data can be managed by just the activity, in the form of a model (headless) fragment
where I was going originally is that if I needed more than one activity to have access to that model data, the model fragment wouldn't work, as fragments are tied to a specific activity instance
I'd have to move the model data out of the activity and into some sort of singleton
one that can be accessed by both activities
that singleton might also manage the threads for reading in and parsing the JSON, just as the model fragment manages those threads right now
Steve
ok
Mark M.
this is just an example of how your UI architecture (many activities vs. one activity/many fragments) has an impact on how you cache your data to avoid lots of disk reads
Steve
i see. i think i'm going to have to review the design with the issues you've raised in mind and think more about the overall architecture.
8:25 PM
Mark M.
with this sort of thing, there is no "one size fits all" solution
Steve
are there any examples in your book you would recommend i look at?
Mark M.
part of the reason I cited EmPubLite is that it is the most complex app in the book
my samples tend to be focused on demonstrating one thing
and, as a result, they are very simple
Steve
ok
Mark M.
on the plus side, that helps keep me sane
Steve
whatever it takes!
Mark M.
however, it does mean that my book doesn't get into large architectural concerns, or at least examples of that
Steve
i know your time is about up. this has been an extremely helpful discussion for me. thank you so much!
Mark M.
you are very welcome
Steve
have a good night
Mark M.
you too!
Steve
has left the room
Mark M.
turned off guest access

← Tuesday, February 16

 

Office Hours

People in this transcript

  • Mark Murphy
  • Steve