Best Error Logging / Crash Handling Behavior for Release Build?

Boring subject I know, especially when you intend to work out all the bugs before your game releases. But we all know there's always another bug that escapes your best efforts. And even worse, run a search for any popular game alongside the word "crash" and you'll see a shocking number of results. Seems to be the nature of PC games that they will inevitably crash on someone's system, often for reasons beyond the developer's control.

With game releases on the horizon I've been trying to find the "right" way to handle this, but information isn't plentiful. I've scoured the web for this topic as it pertains to Unity. I've run dozens of tests myself for error logging and forcing crashes, both with infinite loops and UnityEngine.Diagnostics.Utils.ForceCrash. And here are the settings I'm leaning towards using, and why:


1) Disable "Use Player Log" : Having a finished game constantly working to log mountains of irrelevant information to an external file really doesn't feel good. Better to...

2) Use a custom logger : Use "Application.logMessageReceived" to log only error messages to a file of your own choosing. This catches Unity's own errors (e.g. NullReferenceException) as well as your own logged exceptions (e.g. corrupt save data). This provides something for the player to reference or send in to help with debugging if the worst should happen.

3) Don't use Cloud Diagnostics : Not interested in this for a number of reasons, primarily that I'd prefer the game handle itself locally without need of an internet connection or third party service.

4) Delete UnityCrashHandler64.exe :
- After paying to remove the splash screen, having "UnityCrashHandler" with their logo right in with your game executable feels rather cheesy.

- Not good for anyone to have the Unity logo proudly appear as your game dies a terrible death.

- It may be more common than I realize, but having a "CrashHandler" prominently sharing a folder with the main game executable doesn't seem like it would inspire the most confidence in the stability of your title.

- This has been known to run even when the game is closed without crashing; all the better to do away with it than risk the player thinking your game crashes every time they close it.

- Windows-only, so it's not even a solution for crash reporting on OS X or Linux.


Now, these settings would seem fine to me, except there's nothing here to cover crash reporting--which is arguably the hardest thing to debug.

The UnityCrashHandler doesn't seem to be documented anywhere (except here, with the insightful description of "Crash handler executable."), but what it appears to do is log a crash report to the inconvenient location of "C:\Users\Name\AppData\Local\Temp\Company\Product\Crashes\Crash..."
This is the only local crash report data Unity seems capable of generating from my findings, so it's something. However, the report itself appears to be quite cryptic and messy (and this from a forced crash), so... along with all its other cons, I'm not sure how helpful it would even be were someone to send one in.


So, my questions to anyone familiar with this stuff or who has shipped a game:

A) Is there anything I've missed to cover local crash reporting? Or do you just wing it and hope the player's report can help pin down the cause?
B) Is it standard to disable the Player Log in favor of a custom error-only log? Or do most commercial games use the Player Log in case they need extra information?
C) How do these settings strike you? Anything I'm missing, or should/shouldn't be doing?

Thanks for reading!

 

I think you're trying to shoot yourself in the foot in order to feel more confident.

"Logging mountains of information" is exactly what you want when something breaks on a machine which is running on another side of the planet using hardware you never knew existed. You need those mountains of information, you Ctrl+F them for diagnostic messages.

Likewise, if your game is collecting data and sending them to you via cloud service this is absolutely something youd' want, because it may allow you to spot issues early.

As for "feels good"/"feels bad" about logo, what kind of argument is that? It doesn't matter.

Regarding custom logger... does your logger correctly handle actual crash? This is something you'd want to test. Writing to file is not immediate, and the data is buffered for some time. In case of a hard crash the logger may fail to write piece of information, and you wouldn't want to have a log open with some error messages missing when the application crashed.

Additionally, on other platform unity logging mechanism uses different reporting service. For example, using android build you can connect to the device via adb and see what your game wrote remotely this way. Can your logger do the same?

So, I think your priorities are a bit off. No offense.

---------

It is worth mentioning that in typical use case where user plays a game delivered to them via steam or gog, the user won't even see crash handler executable in the first place. That's because the game will be launched through steam, gog, without user ever seeing contents of its folder. The contents will only be visible if your title is delivered as a zip/7z file without installer.

 

Hey neginfinity, thanks for your input.

Sorry, I think there's been a misunderstanding. I'll try to clarify...

The mountains I'm referring to are pointless things like what version of the Unity editor the game was built in, how many assets were unloaded on scene change, etc., which all get logged every time the game runs. On a finished game that has been tested thoroughly prior to release, this feels to me like a waste of processing overhead.

Whereas a custom logger will still log any actual errors that come up, without constantly logging worthless information while the game is running fine.

But perhaps I'm mistaken and most shipped games do use the Player Log anyway, hence my question.

Not sure if you're referring to the logo being in the folder or the logo appearing while your game crashes, but either way it does matter to us, and these are a couple of several major cons to the CrashHandler--not least of which is no support for OS X or Linux (that I'm aware of).

And yeah, while players likely wouldn't see it for Steam or GOG, I assume the Unity logo would still proudly display while the game dies--which is like screaming, "Hey, this happened because this game was made with Unity!" Which is not what we (or Unity, presumably) want when there's already a stigma out there surrounding games made with Unity. Furthermore, we're planning itch.io releases as well, where the player would absolutely see the executable.

No, I said as much; the custom logger only serves as a replacement for Unity's own Player Log--which also doesn't seem to handle crashes very well.

Our concerns for error logging and crash handling are just for PC, OS X and Linux. Beyond that we're only releasing for consoles, which have different considerations altogether.

 

You probably didn't find much in your searches because you weren't searching for the right things. Logging and analytics is the general area. Everyone has this now days as crashes are just one of many things you want insight into.

Your logic for not wanting a cloud solution is wrong. How do you think you are going to get at the data? At some point the player has to be connected to get it to you. Store and forward is a common approach, it doesn't require being connected to play. You don't necessarily need a cloud solution but don't discount it for the wrong reasons.

Almost all of the good solutions geared for small companies are cloud based. But most allow you to easily integrate a store and forward approach also, so you can mix and match what works best for your game. There are also a lot of local solutions but they are generally either very expensive or lower level tools.

So the right solution is more based on what can you afford. With the proper emphasis put on how important good logging and analytics is. Like if it's a mobile game not having a good solution and you might as well not even bother making the game. A more niche non mobile app a simpler solution might work if your team is really small. Like local logging and forward via http to your server via an in game menu is probably the most minimal setup.

As far as what to log, as much as you can afford. In terms of disk space, bandwidth, whatever the constraints are. When you actually need the data more is almost always better.

 

Thanks for your input, snacktime

If you're referring to crash reports generated by Cloud Diagnostics, then yeah, I don't know how to get at good crash data from Unity locally (again, one of the questions here is if it's possible).

If you're referring to log information in general, the same way games have always done it: the player asking for help via email, Steam, Reddit, or wherever, and providing error logs (and additional information) upon request.

We may yet have to use cloud reporting, but it's a very new thing in the grand scheme of game development, and we don't consider our logic for not wanting it "wrong".

We're building non-mobile titles that we don't want relying on ongoing analytical solutions and servers, and we intend to test thoroughly and extensively for a stable release--like the old days when games had to ship on cartridges.

It's always possible a "whoops" crash of the memory-leaking nature could get out, but the kind of crashes I'm most concerned about are the weird ones that require human-to-human interaction. Things that don't get patched on the game's end, but get helped on that specific player's end; e.g. I've heard of Game Maker games crashing on start due to certain printer drivers (of all things), and disabling the driver fixes it. Getting an anonymous crash report over the cloud isn't going to give that player the support they need to fix their very specific and quirky problem.
To that end, if their game keeps crashing or a problem keeps occurring, I'd imagine a player would feel better anyway about seeking direct and immediate help, rather than trust that "the developer was just sent a report about your crash and may or may not be able to do anything about it at a later date".

Additionally, our game has no online functionality (outside of Steam), and we've had test players find it shady when the game tried to connect to the internet anyway for Unity's hardware statistics. When on Steam this presumably wouldn't be a thing, but for anyone playing a direct game from Itch.io or somewhere, it's not an ideal solution.

My general understanding of the Player Log has been that it's essentially a lot of expensive Debug.Logs firing off behind the scenes. Most developers I've seen seem inclined to disable all logging entirely for a release build (and the Player Log isn't even an option for the Mac App Store). Having a custom logger for only actual errors seems a better middle-ground to me, but I gather your recommendation would be to leave the Player Log enabled on release builds?