Is Google blocking apps writing to SD cards ?

Ok, so the subject is a little misleading. A more complete question would be:




At the moment, it does appear so, but it may also be a bug or manufacturers incorrectly implementing a "feature". First, let me clarify the question, for starters to explain which storage I actually mean.

Various Android devices released the past few years have come with "internal" flash storage, as well as having the capability to extend storage by using SD cards.

In Android terms, that "internal" flash storage is also called "external storage". It does not matter that the storage is physically inside the device, it is outside the normal app storage location.

Unfortunately, even though having these two types of external storage has been fairly common, Android still does not provide proper access to these cards through the API. You may think this article as about that - it isn't, this goes a bit deeper.

So, if you have purchased an Android device that came preloaded with 16gb (or 8, or 32, etc) storage and it can also use an SD card, your device has two external storages already.

The internal flash memory would usually be considered the primary external storage location, while the SD card would be considered the secondary external storage location (if internal flash is present). A USB stick or harddrive you can potentially connect would also be secondary external storage.

It appears that in newer Android builds, Google is making it impossible for third party apps (apps that were not preloaded on the device, but you manually installed or downloaded from Android Market / Google Play) to gain write access to the external SD card.

This would make the SD card (and USB devices) completely useless as anything other than read-only storage. Unless you manually copy files there using the built-in file explorer (if your device even comes with one), that is.

I ran into this issue earlier today when working on photo-backup functionality for DSLR Controller and testing it on a Samsung Galaxy Tab 7.7 running Android 3.2. After some time researching the problem, it appears there are a number of other devices also suffering from this "feature" - if you know where to look, complaints about this are abundant. In case of the DSLR Controller example, writing to the internal flash only instead of to SD cards makes the backup feature essentially useless, as will be the case for many other apps and their features. This problem was not present in Android 2.x, though I'm not currently sure in exactly which 3.x build it was introduced.

In the past, an app would request the "WRITE_EXTERNAL_STORAGE" permission, which would grant write access to all external storages (user/group "sdcard_rw"). This has apparently been changed to only grant write access to the primary external storage. A second permission has been introduced called "WRITE_MEDIA_STORAGE", which would grant access to the other external storages (user/group "media_rw").

The problem is, a third party will not actually be granted this permission, only system apps and apps provided by the device manufacturer will normally be granted this permission. There are exceptions, apparently on some devices third party apps will be granted this permission, but according to the AOSP sources, they're certainly not supposed to.

So, what is going on here? Is this a bug? Is this a feature? Is this something meant to be used differently but wrongly applied in firmwares by the OEM? I would sure like to know!

Sources
Of course, you're not expected to just assume whatever I say about this to be correct. Here are some pastes from ICS sources.

system/core/include/private/android_filesystem_config.h:

This shows the AID_MEDIA_RW definition, the group an app must have to have access to secondary external storages.

frameworks/base/core/res/AndroidManifest.xml:

This shows how the "WRITE_MEDIA_STORAGE" permission has "signatureOrSystem" protection. This means it will only be granted to system apps, or apps signed with the same key as the platform (apps provided by the OEM). Note the @hide marker. The description mentions internal media storage, which may hint at external SD cards being marked this way actually being a bug, instead of a feature.

frameworks/base/data/etc/platform.xml:

This show that the WRITE_MEDIA_STORAGE permission grants the "media_rw" group to an app.

system/vold/Volume.cpp @ Volume::mountVol():

This shows that whichever external storage's mount point is equal to the "EXTERNAL_STORAGE" environment variable will get the normal AID_SDCARD_RW group, and apps can get access to this storage with the "WRITE_EXTERNAL_STORAGE" permission. Any other external storage (this includes USB sticks and whatnot) will get the AID_MEDIA_RW group, and thus cannot be written to by third party apps.

The comments are somewhat confusing, as they do mention "primary SD card". The EXTERNAL_STORAGE environment variable often refers to the internal flash instead of the primary SD card. So again, maybe this is not what was intended to happen.

It is not uncommon (but also not a standard) for the internal flash to be mounted in one place and referred to as "the" external storage, with all other external storages mounted in subfolders of that mountpoint. This is a popular OEM workaround to the problem of the Android API not properly supporting multiple external storages (and Google seems to be refusing to fix this situation). This way, a third party app only needs to know "the" external storage location, and due to the subfolder structure, it can still access all of the external storages. This specific way of doing things obviously causes problems with the code posted above.

Hopefully, somebody at Google can shed some light on this situation.

Update #1
This has been discussed on Android Developers' weekly Google Plus hangout, as well as some other chats with Google engineers.

None of them had any idea what is going on here - yet. But we have gotten a promise that this specific issue will be looked into, and they will get back to me on this - I hope it will be in dialog!

I also wanted to share with you that even devices like the SGS2 (international) suffer from this issue on ICS, though Samsung has worked around it using a very ugly permission hack. The problem is still there, even if it doesn't show up.

Update #2
By request, here's the way Samsung has worked around the problem in their ICS builds for the SGS2:

/system/etc/permissions/platform.xml:

Well, this is certainly an interesting way to go about working around the problem. What is happening here is that if you ask for the "WRITE_EXTERNAL_STORAGE" permission, you also get the "WRITE_MEDIA_STORAGE" permission - at no extra charge! It should be obvious that this is not in any way a proper solution to the issue.

What makes this even more interesting is that this means at least Samsung engineers are apparently well aware of the problem.

Still looking forward to hear what Google has to say about all this. It worries me that they added code that - based on the code comments - is meant to restrict even more access, while not providing any clear reason why this should happen.

Update #3
An observant commenter on the XDA portal article stated that this issue is patched in CyanogenMod:


That's nice for CyanogenMod users, but doesn't help stock users. The question remains - WHY ?! There doesn't seem to be any reason for this stuff, and it's causing real-world issues to real-world devices for real-world people!