This may be by-design in order to allow for OBB files, or it may be a bug, I'm not sure. Exporting to ABB instead of APK does not ignore the 'split by architecture' checkbox. I build my project with ABB + split by architecture, and the ABB was missing textures, etc. Would be nice if the Unity and/or UCB interface explained this better. I was specifically looking for this detail and it wasn't clear to me, so I had to build a few times in order to figure it out.