Android 10 and Scoped Storage (missed features in Unity 2019)

Hello! Android 10 is a big issue for all developers who make app that needs to list files and work with them (file browsers, photo editors and so on). You can read more here: https://www.androidcentral.com/what-scoped-storage-android-q

My problem is that it seems I can no longer access DCIM folder for my app (pixel art editor). The app works fine with Android 9 but it gets UnauthorizedAccessException on Android 10. So my question is how Unity can handle access to DCIM and work with Storage Access Framework APIs in general?

Is the only solution I see now is to wait until someone will write a native android plugin? Unfortunately, I'm not a Java-developer at all.

 

Same issue here, a plugin seems like the way to go

Currently you can avoid this temporarily by adding this to the manifest

Code (CSharp):

1. <manifest ... >
2.   <!-- This attribute is "false" by default on apps targeting
3.        Android 10 or higher. -->
4.   <application android:requestLegacyExternalStorage="true" ... >
5.     ...
6.   </application>
7. </manifest>

source: https://developer.android.com/training/data-storage/files/external-scoped

 

We got the same issue, but adding the

android:requestLegacyExternalStorage="true"

to the AndroidManifest.xml did not fix the issue.

We are trying to access the file

file:///storage/emulated/0/DCIM/Camera/file.mp4

via the Unity VideoPlayer

This related post confirms that others have the issue too with Unity although the requestLegacyExternalStorage key is defined:
https://developercommunity.visualst...droid-files-using-c-filestream-with-unit.html

@hippogames : thanks to your post, the article mentions an even worse thing: Android 11 even removes/ignores the requestLegacyExternalStorage for security reasons and only allows file access via Storage Access Framework (SAF) / Scoped Storage.

I already filed an issue at Unity, because they have to get around it directly.

 

We have the same issue. Do you get any feedback so far?

 

I've had some experience with Storage Access Framework (1, 2, 3) and in my opinion, Unity can't do anything about this issue. File API is restricted by the Android OS itself for security reasons. When using SAF to access files that are not owned by the app, a confirmation dialog is presented to the user. Only then access to the file is granted (this is why SAF is more secure).

You might ask then why Unity doesn't display this confirmation dialog automatically whenever we are trying to access a file in external storage. One possible reason is that SAF doesn't use raw file paths, here's an example SAF path:

content://com.android.externalstorage.documents/tree/primary%3A/document/primary%3APictures

. The issue here is that, Android doesn't have an API to convert a file path to SAF path or vice versa (you can find workaround solutions on stackoverflow but they won't work in all circumstances). So, Unity can't reliably convert a file path to SAF path, which is needed in Storage Access Framework.

P.S. You might find these upcoming changes interesting, though:

https://developer.android.com/preview/privacy/storage#media-direct-file-native
https://developer.android.com/preview/privacy/storage#all-files-access

 

Thank you for the info.

We are saving and loading only files from within the application.persistentDataPath, other files are working but only the videoplayer url will not work for this.

Do you know a way of handling that, e.g. convert a local path to an SAF compatible path (only for the application.persistentDataPath). Maybe the videoplayer will work with that?

 

AFAIK, Android doesn't have an API to convert a file path to SAF path. If there is a standard SAF format for Application.persistentDataPath, I don't know about it.

Since Application.persistentDataPath can be accessed via File API, I don't think this VideoPlayer issue is caused by SAF. You can try starting your path with "file://" prefix. If you are using the StreamingAssets folder, then you are working with Application.streamingAssetsPath, not Application.persistentDataPath. Let me know if this is the case.

 

Has anybody solve this? my phone automatically upgraded to Android 10 last night, now my app cannot access

/storage/emulated/0/DCIM/Camera/20200704_230148.mp4

anymore so the upgrade broke my app in the play store. I cant see how google justifies doing this, should they be sued? Even if i fix it somehow, i can expect it will break again for android 11, and 12, and 13 etc, totally ridiculous, WTF

 

Looking at this topic, I'd say that you'll again be able to access that path starting with Android 11. To support Android 10 as well (i.e. support all Android versions), you should set requestLegacyExternalStorage to true in AndroidManifest.xml as suggested in the topic.

 

I tested requestLegacyExternalStorage, but it does not seem to work correctly. Does anybody have luck with this? This property is also not available if targeting other SDK levels than 9 or 10.

 

Hey @yasirkula, do you know any updates on this? Any info or plugin on how to use Scoped Storage on Unity?

I use some of your plugins so I trust you have much more knowledge on this matter than most Unity users

 

You can access images and videos in the Gallery using File API as stated here. To browse other files, you have to either create a native plugin or use an existing plugin like SimpleFileBrowser or NativeFilePicker.

 

Thanks for the reply! Regarding images/videos in the Gallery, are you aware of a Unity solution?

I currently use your UnityNativeGallery* and it stopped working for Samsung devices with Android 10. Now I have just rolled a test update with that AndroidManifest flag and it's working again, but it seems (from your message) that this flag is just a temporary hack, on Android 11 I will have to find a definitive fix. If you have any ideas let me know. Thanks!!

* EDIT: I use an older version of your plugin, I dont update it for over a year, maybe the current version has a fix for this already?

 

Yes I'd recommend you to update the plugin and give it another try.

What do you mean by a Unity solution for Gallery? Like NativeGallery but made by Unity? If so, AFAIK it doesn't exist.

 

No I mean a C#/Unity solution instead of a Java/"core Android" solution. I saw that the link you posted is for the Android docs, and usually the solutions there dont apply for the Unity world. Is the current solution for this problem doable in Unity-only world (without the need to write Java code)?

To be honest I took a quick look on the Android docs link but I didnt understand much what I need to do to use this scoped storage thing to access the album photos.

 

You need to call Environment functions using AndroidJavaClass to retrieve special folders' paths but then, you can use System.IO.File API to browse images and videos inside these folders. You need to keep the requestLegacyExternalStorage flag in AndroidManifest for Android 10 compatibility because File API compatibility was introduced in Android 11 and is otherwise not backwards-compatible.

 

Well, a lot of apps will stop working... Google loves making our life worse...
Probably Unity can make something to help, creating a function that returns that folder where all other apps can have access to. My game is heavly dependent on user mods (vehicles, repaints, sounds, maps...). It is already hurting.
Following the thread to see any news about it, I would love to see a solution asap since this is very critical.

 

Why do other apps need to have access to your mod files, can't you simply use Application.persistentDataPath? If you really need to interact with public directories, you'll need to use Storage Access Framework; I'm not aware of any public directories that you can read/write generic files using File API.

The plugins I've mentioned here use Storage Access Framework, so you can check them out if you want: https://forum.unity.com/threads/and...d-features-in-unity-2019.749333/#post-6574336

 

My users are used to use file managers, text editors, image and audio editors to change everything on the vehicles and maps. So it is convenient that the folder is freely available. They download mods via browser and extract to a folder, being able to change almost anything there.

If I have to make an UI to manage it, the cost of development will increase a lot, since every sound, txt and image must be replaced one by one, it would be like creating a file manager... Also for them to create backups of installed mods will be much harder or almost impossible. I can't believe that it is happening.

 

Wouldn't my SimpleFileBrowser plugin be useful in this case? Your users would just select the mod's folder with the plugin and then you'd use FileBrowserHelpers functions (similar to File API) to load these files.

 

I'll try that and comment later if it works, thanks!

 

I was reading the docs of your FileBrowserHelpers, if I didn't get it wrong...
So are the FileBrowserHelpers functions multiplatform? Like I may use it regardless of being for PC or Android, and it will manage it internally... Or should I make a different approach to manage the files by myself? By accessing on Windows with File... and on Android with FileBrowserHelpers...

Btw thanks a lot for your work on this plugin, it seems to be perfect for the moment.

I was reading that there will be a permission for some types of apps to continue using the normal File, but that must be requested, I don't think they would approve that for a game... Most mobile games do not have modding support, only PC ones. If I don't give up on the project I'll try it later, this is sooooo frustrating. I'm already facing refunds of users that have upgraded to Android 11. Unbelievable. This is not happening...

https://developer.android.com/training/data-storage/manage-all-files

 

I also don't think that MANAGE_EXTERNAL_STORAGE permission will be approved on a game.

FileBrowserHelpers is multiplatform, you can see in the source code that it uses File API when the platform isn't Android or when Storage Access Framework isn't needed (Android 9 or earlier). You just have to make sure that the path you pass to FileBrowserHelpers is returned by either the file browser or another FileBrowserHelpers function. If you pass a hardcoded filepath to FileBrowserHelpers on Android 11+, it won't work.

 

Perfect!
I made here a prototype to install the mod directly from a zip file, so users select the zip with your dialog, the game copies it to a temporary folder and then extract it. Thanks a lot for making this asset!

Do you think that it would be too hard creating a function to recursively copy an entire folder structure, the folder selected on the dialog, and also the opposite way? That would be necessary to make backups and restorations easily. Something like:

FileBrowserHelpers.CopyDirectoryRecursivelyToInternal(string SAFsourcePath, string destPath, bool overwrite)

FileBrowserHelpers.CopyDirectoryRecursivelyToExternal(string sourcePath, string SAFdestPath, bool overwrite)

I will try to make it with proper time later. It could be a nice addition to the plugin.

 

It wouldn't be too hard. It just requires a combination of GetEntriesInDirectory, CreateFileInDirectory, CreateFolderInDirectory, WriteCopyToFile (copies normal file to SAF file) and ReadCopyFromFile (copies SAF file to normal file). I may add a built-in function later.

P.S. Added FileBrowserHelpers.CopyDirectory function with the latest update.

 

Might not be related to the post, but is it possible to access OBB files and without MANAGE_EXTERNAL_STORAGE permission? My game is about 200MB large and the obb file is stored separately which had been functioning as it should when the permission is given. I need to do this as I'm trying to comply with the latest policy update from Google and I have no experience with Storage Access Framework or the Media Store API suggested in the google docs.

https://support.google.com/googleplay/android-developer/answer/10467955#zippy=%2Cpermitted-uses-of-the-all-files-access-permission%2Cexceptions

From reading the exceptions in the new policy, it does not seem that games larger than 100mb or require to access obb file is listed in there.

 

Aren't OOBs stored in your app's local directory? You should be able to access them without any permissions.

 

I think, I might be having the same problem with a game on Google Play. Due to its size the game use OBB files and Google is asking to put a MANAGE_EXTERNAL_STORAGE permission in the manifest file or it will be removed form the store.

 

You might have gotten it the wrong way around. Is it possible that Google is asking you to remove MANAGE_EXTERNAL_STORAGE permission from the manifest, if exists?

 

I think you're right, that's what the message in the google paly console said :
"Update your app to declare the All files access (MANAGE_EXTERNAL_STORAGE) permission in the manifest file, and complete the All files access permission declaration in Play Console from May 5th
Remove the All files access permission from your app entirely"

However, I'm a bit confused, I don't know how to remove MANAGE_EXTERNAL_STORAGE permission from the manifest, and I'm starting to think I need to add the permission for the PlayerPref the game uses.

 

After building the game, open PROJECT_PATH/Temp/StagingArea/AndroidManifest.xml and see if MANAGE_EXTERNAL_STORAGE exists there. If not, you don't need to worry about anything. Otherwise, check Plugins/Android/AndroidManifest.xml or your plugins to find where it comes from.

 

https://stackoverflow.com/a/67864052/11239885

 

I tried to add android:requestLegacyExternalStorage in AndriodManifest.xml file. It worked!