Hi Mark!

hello, Tad!

how can I help you today?

Hey I've got some questions for you around a topic you've posted some on back a couple years - it is how to properly interface with the external storage APIs to save media taken with the camera (image, video) such that it is available to other apps, not just mine.

So I've read all those posts

And I've gone over the doc.

I still have a couple of what are probably fundamental questions.

According to my reading, it seems that if my app (21+) wants to access the area returned from the API getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES) for either reading or writing, the permissions for read/write storage must be declared and user must allow. However - I notice when I turn this permission off, then attempt to open an image file the Android picker still lets me get into these directories - so clearly I am "reading" them. How is this possible?

"so clearly I am "reading" them" -- not really

ACTION_PICK, ACTION_GET_CONTENT, and ACTION_OPEN_DOCUMENT bring up system activities or activities from other apps

ok why isn't being able to see them in the picker and choose one "reading" it?

Maybe I don't understand "reading"

reading means "getting the bytes"

and in terms of external storage, it means "getting the bytes via filesystem APIs"

OK - so if I tried to do anything with the Uri returned from the picker I'd get a security exceptions

no

you are being granted access to that content by the picker

that other app -- typically a system one -- has rights to the content

it grants you temporary rights to the user-chosen content

but not to arbitrary other content

So I could open the file up and put it in imageview for the life of the app that ALSO had the picker active.

i.e. I start my app, use the picker, find a file, open and view it.

stop!

But I can't store that URI and use it tomorrow once the app has shut down today?

stop!!

let's make this a bit more concrete: what Intent are you using to bring up what you refer to as the "picker"?

Hang on

ACTION_GET_CONTENT

OK

note that this is ACTION_GET_CONTENT -- it is not called ACTION_GET_FILE

that is important

the user does not have to choose a file on the filesystem that you can access

and so you really need to be *very* judicious about where you use the word "file"

Ok point taken.

by saying things like "I start my app, use the picker, find a file, open and view it", you are putting yourself in a mental model that isn't accurate

so, you use ACTION_GET_CONTENT, and the user picks a piece of content and you get a Uri back

you have temporary rights to access the content identified by that Uri

see https://commonsware.com/blog/2016/08/10/uri-acc... for more

"But I can't store that URI and use it tomorrow once the app has shut down today?" -- correct

think of the Uri as being akin to a URL from a Web app, where the app requires authentication

so long as the user's session has not timed out, that URL is perfectly fine

once the user's session times out, the URL is ineffective

same thing here

if you use ACTION_OPEN_DOCUMENT, there is an option to get longer-term access rights

OK - I am also using the takePersistableUriPermission API on the resulting Uri to be able to store it and open it tomorrow.

that only works with ACTION_OPEN_DOCUMENT

it *might* work for *some* Uri values you get from ACTION_GET_CONTENT, but don't assume that it will

OK - right. Hadn't gotten that far yet in this implementation to bump up against that error :)

Let me start over.

I am using ACTION_GET_CONTENT in the part of my app that allows the user to choose existing content

That all has been working fine, using the takePersistableUriPermission approach

Sorry - ACTION_OPEN_DOCUMENT

ah, OK, that feels better

Then I just started implementing a feature whereby the user gets a dialog that allows them to either take a picture/video and "add it to the app", or choose from existing media they already have on their device.

hmmm... "add" may be too generic of a verb

So what I do is allow them to choose media from whatever the SAF presents to them.

if you are holding onto Uri values, "link" might reflect what you are doing better

Then I use the takePersistableUriPermission API on it, and store that Uri in my own DB

Then at runtime I use that Uri with Glide to show the image in an ImageVie

ImageView

All this works fine.

Besides the risk that the Uri might not be valid sometime in the future, which I understand.

And deal with in the app.

Now - the second part is what I'm tripping up on.

That is where I want to give the option for them to use the Camera to take a photo or video, then I will pass in the intent a Uri to the File that the Camera API will use to store the resulting media. That same Uri is what I would want to do the same thing on - takePersistableUriPermission.

But I hadn't gotten that far, because when I started to write the code to store the image, I started down the path about learning which permissions are needed for which directories, etc.

OK

And I got confused when it appeared I could select files from the public PICTURES directory even though I had turned off my storage permission for the app.

select content

"select" meaning your existing ACTION_OPEN_DOCUMENT code?

Yes

you can use the Storage Access Framework without having external storage permissions

that's because the user is always involved in the selection

external storage permissions let you access the filesystem without user involvement (after the initial permission grant)

so, what you are seeing is perfectly normal

OK - so for example, my app allows user A to send media to user B via WebRTC. Service running on both sides manage this transfer. Since a service is doing the work, I would need storage permissions granted on both sides, if I read bytes from the public Pictures directory on A and save to the public Pictures directory on B. Correct?

if you are doing it using filesystem APIs (e.g., java.io.File), yes

and note that it won't work on Android Q (by default) and Android R+ (for all apps)

vs. what?

versus using the Storage Access Framework

so if user A picks the content via ACTION_OPEN_DOCUMENT, you do not need external storage permission

and if user B picks the "directory" via ACTION_OPEN_DOCUMENT_TREE, you do not need external storage permissiosn

er, permissions

so is it the SAF or the fact that the user instigated SAF?

Sounds like it is user driven

Android doesn't know that the user "instigated" SAF -- it has no way to know that because the user clicked something-or-another that you brought up ACTION_OPEN_DOCUMENT

it is the Storage Access Framework itself -- allowing direct user involvement -- that eliminates the need for the storage permissions

"allowing direct user involvement" - I haven't looked at doing this, but is it possible to call the SAF API's directly from a service with the info they need such that no UI involvement is done at all?

no

ok I think I understand then.

there are no SAF APIs other than ACTION_OPEN_DOCUMENT, ACTION_CREATE_DOCUMENT, and ACTION_OPEN_DOCUMENT_TREE

I will need to be sure permission has been granted for read/write to access this storage via my service approach.

Which is fine.

Next question :)

My strategy has been to check and see whether those permissions have been granted, and if so utilize the public storage area from getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), and DIRECTORY_MOVIES.

If the permissions have not been granted, I will fall back to directories under getFilesDir()

you could use getExternalFilesDir() if you wanted

it's external storage (so user-accessible) but no permissions are required

right

I saw that also.

And it gets deleted when the app is uninstalled, apparently - so user accessible, but not persistent across apps.

Anyway -

I would describe it as "not persistent across installs"

same as getFilesDir()

Right

When I was reading about how Android wants me to set this up using a FileProvider, I got a bit confused.

What is confusing to me is that the API description for FileProvider states that you must explicitly tell it what paths it should be using.

relative to select root locations, yes

But I don't see a good generic way to indicate access to that same directory root structure returned by getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES) (for example)

Because the return from that call will be different on different devices I presume.

your root location would be <external-path>

Yes I have that:

But I was nervous about hard-coding "Pictures" and "Movies"

What if those change?

in external-path

agreed, at least for <external-path>

so, use <external-path name="tad-stuff" path="/" />

(substituting in a more appropriate value for the "name" attribute :-)

Is the ramification of that then that all directories from the root are available to be browsed?

no

I'm confused then about what path is doing

no other app has access to your FileProvider content *except* what you grant manually in code

path="/" says "I am willing to allow my FileProvider to serve from any location on external storage"

But if path is just "/", then I would (on my Samsung S7 phone) be allowing FileProvider to browse: /storage/emulated/0/

whereas path="Pictures? says "I am willing to allow my fileProvider to serve from any location off of Pictures/ on external storage"

FileProvider doesn't really "browse", so I don't know how to interpret that verb

ok - "serve from any location"

right

however, it only serves what you tell it to serve at runtime

point is - it serve up content from the root "directory"

Yeah, ok

by mimetype, yes

?

so, don't screw up your FileProvider.getUriForFile() calls

no

by Uri

with FileProvider, at runtime, you do three key things:

1. you use FileProvider.getUriForFile() to get a Uri that points to the desired file

2. you put that in an Intent somewhere (e.g., EXTRA_OUTPUT for ACTION_IMAGE_CAPTURE)

3. you call addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION)

(and/or FLAG_GRANT_READ_URI_PERMISSION)

that Intent, in turn, winds up in the hands of another app, such as a camera app

that app has rights to work with the content identified by that Uri, for whichever rights you put in the addFlags() call

however, that app has no rights to any other content that the FileProvider might be capable of serving

and no other app has rights to that Uri's content, just because this one other specific app has rights

and, the granted rights are temporary -- you saw the reverse of this before you started using takePersistableUriPermissions() (and, since this isn't ACTION_OPEN_DOCUMENT, that's not an option for the camera app in this scenario)

OK - yes that all makes sense, which is why the Camera app can write bytes to that one file.

right

In terms of the appropriate API to use to get to the public directory, the docs here: https://developer.android.com/training/data-sto... suggest normal best practice is to use getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES) to save a photo taken with the camera as the user will expect it to be available to other apps, not just mine. That is fine, however I notice now that this API is deprecated as of API level 29

yup

The blurb there says "To improve user privacy, direct access to shared/external storage devices is deprecated. When an app targets Build.VERSION_CODES.Q, the path returned from this method is no longer directly accessible to apps. Apps can continue to access content stored on shared/external storage by migrating to alternatives such as Context#getExternalFilesDir(String), MediaStore, or Intent#ACTION_OPEN_DOCUMENT."

yup

see https://commonsware.com/blog/2019/06/07/death-e...

and see the first few chapters of *Elements of Android Q*, available in a Warescription near you :-)

in short, I wrote a *lot* about these changes earlier this year

I guess I don't understand this, or am implying too much by it. It seems that this implies that a central area where photos would be stored that would be available to all apps is going away?

filesystem-level access to it is going away

so, for example, in *Elements of Android Q*, I have a sample app that downloads videos from my Web site to the device

I have code in there for an Android Q approach and a pre-Q approach

So I feel like I'm going around in circles a little bit. The API docs for how to handle letting the Camera know that you want it to save the media to the file system require that you create an empty file, then use the FileProvider API to generate a Uri which you pass in the intent to the camera.

(warning: contains Kotlin)

But at the root of all this - my code has to create an empty FILE.

So that means I need file access.

technically, you need a Uri that the other app can write to

Where am I supposed to create this File if the API is now deprecated?

well, again, technically, you need a Uri that the other app can write to

that could still be FileProvider, but you will need to use filesystem locations that you can write to yourself, such as getFilesDir() or getExternalFilesDir()

OK - but if the API is deprecated that knows where this public area is, what do I do to tell the Camera app to store the image into that public PICTURES directory?

or, you could use MediaStore, get a Uri that you want to use from there, and hand that to the camera app, using FLAG_GRANT_WRITE_URI_PERMISSION so you allow that app to write to that Uri as well

I think I'm missing something fundamental - does MediaStore have an API that allows me to define a new Uri that doesn't yet exist?

yes

My understanding was that I need to hand Camera the Uri for where to place the new image.

correct

And one way to do this was create an empty file, then a Uri from that and hand it to Camera

correct

But you are saying file creation is verbotin now in the public area?

correct

So.....

you have three options, as is outlined (opaquely) by that documentation that you quoted:

1. continue to use FileProvider, for a location that you still have write access to

2. use MediaStore

3. use ACTION_CREATE_DOCUMENT

option #1 is easy, but the image doesn't necessarily wind up in a useful spot for the user

option #2 is a bit odd but not that bad, but you only have general control over the location (e.g., "it's an image")

option #3 lets the user choose the actual location

How do you use MediaStore to create a new Uri that doesn't yet exist? Is there an API for this?

you call insert() on a ContentResolver

see https://gitlab.com/commonsguy/cw-android-q/blob...

Is this in your code examples?

ok I'll check those out.

Btw - how was your transition from Java to Kotlin?

in my case, I am downloading a video, but the process for an image will be the same, except for the base Uri -- use MediaStore.Image where I have MediaStore.Video

well, I'm old, so I've seen lots of programming languages :-)

I just bought a course from coursera to learn it.

picking up Kotlin wasn't too bad, which is why I have a book on it (and a follow-up coming out on Monday)

see also *Elements of Kotlin*, also available in a Warescription near you

Well, I'm old too but still having fun developing apps

OK

actually, going back to my earlier list, there is a fourth option:

4. use ACTION_OPEN_DOCUMENT_TREE, then use DocumentFile to get a Uri for some new piece of content inside of that tree

Do you have a code example for this?

that's getting more complicated, but it allows the user to give you access and a target "directory" once, whereas the ACTION_CREATE_DOCUMENT approach requires user interaction each time

of that option, I don't think so, but hold on...

yeah, as I thought, I don't

ok

Well I will take a look at the various links you posted, thanks very much.

Sure seems complicated just to allow my app to launch the camera and get to the resulting image!

it's only complicated because of where you want the image to reside in the end

if only your app needed the image, FileProvider and getFilesDir() handles your case neatly

in your case, you not only want the image, but you want to allow the user to keep the image somewhere beyond your app

You mean in that public area? I guess I just want my app "to behave" properly - i.e. it launches the Camera and user takes a picture, the docs seem to suggest (and I would agree) that the user would expect this picture to be available in the Gallery, etc.

I disagree

That's interesting.

Why?

as a user, if I personally launch the camera app, I would expect the image to wind up where the camera app normally stores pictures

as a user, if I launch a barcode scanner, I would not expect the barcode to wind up where the camera app normally stores pictures

as a user, if I launch Instagram or something, I would not necessarily assume pictures that I take in there to wind up where the camera app normally stores pictures

Let me put it to you this way - I had previously implemented support for when the user is using the camera (not my app yet), takes a picture, then decides they want to share that picture, for my app to be one of the choices they can make. If they do, the app launches, I use the Uri to get at the picture, my app does its thing, and then we are back to the Camera.

(note: I am not an Instagram user)

"Does its thing" in my case means adds the picture to a playlist or to an internal DB that can be used to create playlists.

So now the scenario is the user launches my app, and decides they want to create a new playlist.

They can opt to choose from those that my app knows about (the DB)

or the Picker

or the Gallery

or whatever.

Or

they can choose to take a new picture

and that's fine, but that last option is a bit unusual and is not well-covered in the Android APIs

So if they are taking a new picture, particularly if they just finished a session whereby they shared a pic with my app, I can see how it might confuse them if in the first case the picture was available to them but the second was not.

and in your case, that's understandable

however, you are swimming against the Android security tide

and you will need to decide whether that particular feature is worth the headache

and that's all the time that I have for today

Oops - holy smoke you are right.

as usual, the transcript will appear on https://commonsware.com/office-hours/ shortly

Thanks for your time!

yeah, that hour blew by

next chat is Monday at 9am US Eastern

Bye now

have a pleasant day!

whoops, sorry -- next chat is tomorrow at 7:30pm US Eastern