Unity 2018.4 Editor Window lags whenever I alt+tab back to it

This started happening when I updated to Unity 2018 (right now I'm on 2018.4.27f1). Whenever I focus on some other window and then Alt+Tab or click back inside the Unity Editor window, there's a 1 second freeze wherein clicking on anything in the Editor window does not respond, whether it's clicking a button, clicking on File menu, trying to select a game object, etc.

I turned on the Editor Profiler and I saw AssetDatabase.Refresh taking up 1 second. I haven't changed anything in my assets or source code, but still when I alt+tab, that 1 second lag from AssetDatabase.Refresh still happens. Is this normal? I could've sworn this didn't happen on older versions of Unity that I used to use.

I look at the Editor.log and it seems this is the accompanying log from AssetDatabase.Refresh:

Refresh: detecting if any assets need to be imported or removed ...



Refresh Completed time: 1.040336s

    Asset Scan time: 1.034820s

    Asset Hashing: 0.000000s [0 B, 0.000000 mb/s]

    Asset Import (Scripting) time: 0.000000s (count: 0)

    Post Processs Assets (Scripting) time: 0.000000s

    Asset Rehashing: 0.000000s [0 B, 0.000000 mb/s]

    Asset Import (Non Scripting) time: 0.000000s (count: 0)

    Post Process Assets (Non Scripting) time: 0.000000s

    Dependent Assets to Import Queue time: 0.000000s

If I turn off Auto Refresh in the General Preferences, that fixes the lag, but I assume that Auto Refresh is needed?

Is this just a natural consequence of having a large project?

I've seen older threads regarding the same symptoms happening in Unity 2017, but I'm guessing those are fixed.