Search Unity

UnityEngine.Windows.File vs System.IO.File

Discussion in 'Windows' started by IQpierce, Nov 1, 2018.

  1. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    My multi-platform Unity project has experienced multiple major problems on UWP that turned out to be due to how local files are access from dataPath, or streamingAssetsPath.

    The root issue was that the "System.IO.File/Directory" operations, which we've always used on other platforms, *mostly* work fine on this platform... until they don't. (For us these issues cropped up at the worst possible time - after our app was in submission to Xbox, at which time we would discover that the app froze at startup due to File operations failing - since the directories used for a store-install had different permissions than those used in a sideloaded test. These issues repeatedly jeopardized our game's Xbox submissions.) One example was that trying to read a text file using System.IO.StreamReader would fail, but only when installed on this file system *and* when the file size was 0 bytes.

    My understanding is that this is all basically because System.IO.File/Directory API is a synchronous API but that UWP/WinRT only really supports async operations... and/or because the UWP platform compiles against a unique version of mono/.NET. It took me a while to find the guidance that UnityEngine.Windows.File operations are preferred on UWP.

    The subtle bugs caused by using System.IO, which only crop up in certain situations, feels like... a trap. Even when I tried to switch all of our game's file operations over to the UnityEngine.Windows.File API, I missed some places (e.g. I replaced all our usages of File.Exists() but failed to catch a usage of FileInfo.Exists()). The fact that our cross-platform team could easily add usages of the problematic endpoints and introduce the problem again on this platform without realizing it, is very worrying.

    It seems like Unity should do one of the following
    • Make System.IO operations on UWP not work - ie throw "InvalidOperation" errors - if you ever try to use them.
    • Better yet, make System.IO operations unavailable *at compile-time*, or at least flagged with [Obsolete], when compiling code for UWP platform.
    • Even *better* yet, just make the synchronous System.IO operations call the correct/reliable async operations on this platform (but block on them so that it still serves as a synchronous API to the code using these endpoints). (This would fix the current situation where all file operations need to be surrounded by #ifdef UNITY_WSA instances to use the other API.)
    ...And Unity should *definitely* make the pitfalls of not using the correct endpoints more obvious in documentation and porting guides, as these issues were not at all obvious and rarely mentioned in the docs. The most information I found about these issues were here:
    Can a Unity rep explain why these operations fail, and why we're allowed to use System.IO.File operations on this platform if they're unreliable/unsafe? (Am I wrong in thinking that they just shouldn't be used at all on this platform?) It would be nice to hear that Unity had a better long-term solution that didn't require special-case file IO behavior for this platform.
     
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    I'm happy to comment on this.

    This is incorrect. UWP applications support synchronous I/O just fine:

    https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfile2
    https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-readfile
    https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-writefile

    Unfortunately, I have no idea. I have never seen this happen before. They _should work_. Are you using .NET or IL2CPP scripting backend? Do you know in what way they fail? I wish I had an answer to this, but it's definitely not "don't use System.IO on UWP".

    The reason UnityEngine.Windows APIs were added in the first place is complicated. Back when we first started supporting deploying to Windows Store in 2013 targeting Windows 8, you had to use Microsoft's .NET scripting backend (we didn't have IL2CPP back then). In addition to that, Microsoft's .NET for Windows Store Apps had a very limited API surface. It lacked basic stuff like System.IO. This, as you might imagine, posed great difficulties for developers porting their games to this new platform. Since System* namespace is reserved by scripting backend, and we couldn't modify Microsoft's .NET (we had to play by their rules, unlike with IL2CPP), we decided to add a couple of simple APIs to UnityEngine.Windows namespace that people could use with a couple of #ifdefs.

    Now that IL2CPP is going to be the only scripting backend moving forward, we're going to deprecate UnityEngine.Windows APIs in favour of System.IO, and remove them several releases later.
     
    ferretnt likes this.
  3. IQpierce

    IQpierce

    Joined:
    Jan 24, 2011
    Posts:
    43
    Thanks very much for the additional info. This issue was very hard to debug, since it only showed up for us on the XboxOne platform when installed from the store.

    We're using an IL2CPP UWP build.

    It sounds like the part of my fix I thought was important (the use of the UnityEngine.Windows.File API) may not have been the change that fixed this, and there's something else going on.

    I have some theories:
    • The sideloaded test builds (which worked fine) were installed at "D:/Developmentfiles/WindowsApps/" ... while the store builds were installed at "S:/Program Files/WindowsApps/". The fact that there's a space in the filename in the latter case might have caused the failure. Are there known issues with spaces in filenames in UWP paths?
    • In addition to using the alternate API, my code sanitizes the paths by converting all slash (/) characters to backslash (\) characters. Could this have mattered?
    • Some of our filenames in StreamingAssets are very long. I ran across MS documentation about the fact that very long (>255 characters) file paths are treated differently, including how it resolves/converts slash characters - see the notes about the "\\?\" prefix: https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation
    Do any of these sound like they may be our true culprit?
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    I don't think so.

    Possibly, if you constructed the paths using a + "/" + b. Not all windows file APIs like forward slashes.

    This one sounds extremely likely. How long are the paths? To be honest, you would run into the same issues on Windows Standalone, as that path limit is there everywhere.