Are Unity projects becoming more bloated?

Noticed that my project drive is nearly full and then I checked and found some basic Unity projects were weighing in over >1 GB in size.

It looks like the Package Manager keeps a copy of each package and every old package within the projects folder.

Isn't this an ideal job for a common Unity Package Manager cache/folder?

Is anyone else noticing this and where are you finding the most bloat in your Unity projects?

Could Unity add a clean-up process that removed non-essential e.g. generated data from projects (in theory exporting as a package does this; so there are already filters/processes within Unity that select only essential data)?

 

The artifacts folder seems to be very large in some project with 763.9 MB in the artifacts/Stevedore/ folder with lots of application tools.

Again why place a 1.2 GB copy of these build tools within each project folder?

And an example of the Package Manager bloat is a project with four versions of Burst all in the Package Cache and weighing in at over 0.5 GB.

The obtuse thing about this is I've been using Tiny to try and make smaller faster WebGL games only to find that my project drive is filling up with only a few projects!

I could move to building all my projects in one big project as scenes... is this the way we are meant to use Unity now one monolithic games project with all our games in it?

 

No, because then you run into a bunch of use case scenarios like "how do we handle backups and source control."

The only obtuse thing here is you comparing something meant for reducing build sizes and runtime performance costs with project size.

You're expected to have more than 10gb of local storage and also not have fifty different concurrent projects because it is 2020.

 

It should be possible to workaround, look at how microsoft does it with nuget. But anyway. Storage is cheap, you can get a 1tb 4gen 4x pcie nvme for next to nothing these days. I remembered when PCIe disks costed thousands of USD

 

OK but my point is why keep old copies of packages, in my case 3 older versions of burst were kept?

How often does the package modification happen and is it across all packages?

What are the statistics on this and could a central package cache be used most of the time?

 

Well it looks like 1tb SSD's costs about $100 (newegg us) so in theory a 1GB unity project only costs about $0.10 in storage.

However to some people a $100 is a lot of money and Unity could be kinder to those people and drop the price of a project to a few cents instead of over a dime (10¢).

 

Then they can use something other than an SSD. For the same price you can get a 4tb harddrive.

 

So they save a few cents but pay double in loading and build times?

And remember that all this excess bloat is probably impacting project loading/updating and build times...

 

If they can't afford an SSD now then they're already dealing with those load and build times that you're dramatically overstating.

Have you run any numbers or did you just pull this out of your ass? Actual numbers. I know you like to cite completely irrelevant numbers that look like they support your case but have nothing to do with it.

 

No i meant a 1 tb nvme 4 gen 4x. Why use those old slow ones when you can get a Gigabyte Aorus NVMe Gen4 M.2 2280 SSD 1TB with 5/4.4 gig read/write. They have dropped in price like crazy since they were released.

Nuget keeps a global cache and when you import a package it can run local logic on import. Best of both worlds.

 

If you're privileged enough to be involved in game development is there really any benefit to be thinking about cents?

I understand that there's plenty of cases where money is tight. What I don't understand is why you'd be pursuing such a time consuming pursuit as game development in cases where it's that tight. But maybe there are circumstances I can't think of right now?

 

While I could spend $449 for 2TB PCIe 4 storage, I'd much rather spend $400 for 4TB PCIe 3 storage. Plus it's not like PCIe 4 performance is that impressive right now. Current PCIe 4 drives are around 5GB/sec but the actual slot is capable of at least 7GB/sec (just like previous drives hit 3.5GB/sec out of the 4GB/sec maximum).

By the time the technology improves in performance to where it should be the capacity should be about double too.

A basic workstation capable of creating mobile games can be had for as low as $50. You just have to be willing to scour auctions and/or any nearby thrift stores regularly until it happens. Below is the first example that I came across that is bootable. It has dedicated graphics too albeit a very old GeForce 8400 GS.

You'll need a monitor, keyboard, mouse, and so on but those are far easier to find since you can be far less picky.

https://www.ebay.com/itm/Dell-OptiP...GB-RAM-160GB-HDD-Win-10-Nvidia-j/293249649533

 

Booting windows, loading large scenes etc, stuff like that is faster. And the price is so low so its no biggie.

If you always wait for the next tech leap you will be stuck on very old hardware Better to upgrade often instead This build have seen 3 CPUs during its life. First the 1800x then the 2700x and now the 3950x. Same GPU though, a 1080 TI. lets hope we see more sensible prices from nvidia with the 3-series. The 2-series prices were fubar.

 

I mentioned this in another thread but I'll repeat it here. If the current rumors of the performance of the upcoming cards are accurate I wouldn't be surprised if the RTX 3080 Ti were as high as $1,500.

Spoiler: Latest Leak

 

Can be, but if then they need the performance boost to validate it. The 1080 TI to 2080 TI performance boost did not validate that price.

edit: I'm probably buying a 3080 TI regardless of price since 1080 TI is at its end of life, but it would have been nice not being robbed in the process

 

After these years its good to see you're still worrying about negligible things.

 

The GTX 1080 Ti had 3,584 shader cores. The RTX 2080 Ti has 4,352 shader cores, 68 raytracing cores, and 544 tensor cores which would mean a relatively minor increase in traditional performance because the boost that would have happened went to the new hardware.

The RTX 3080 Ti is currently rumored to have 8,192 shader cores, 256 raytracing cores, and 1,024 tensor cores. That's nearly double the traditional shader cores, nearly quadruple the raytracing cores, and nearly double the tensor cores.

All of that is ignoring any potential clock speed improvements and the die shrink from 12nm to 7nm (from the same source that AMD used to achieve the massive performance gains for their CPUs jumping from 12nm to 7nm). Be prepared to be amazed if it's accurate, but be prepared to fork over a kidney too.

 

If we see that performance gain then its more understandable. The price/perfomance ratio between 1080 TI and 2080 TI was a joke. I hope for 980 TI to 1080 TI price/perfomance ratio

 

Where are these game developers who think $100 is a lot of money for buying storage space and also have 50+ active unity projects? Just focus on releasing 1 game at a time. For test projects, learn what you need to, then upload it to github and delete it from local storage. For fun projects, I can't see people concurrently working on any more than a couple at a time.

This reminds me of a story I heard of a jetbrains developer that had a customer complaining about the editor performance. He eventually found out the guy usually has 400+ tabs open while he develops.

Also, if someone is so hard up on cash that they can't afford something as essential as a hard drive that can store their unity projects, then spending a lot of time doing game dev is probably a bad idea.

 

Early on you might have a lot of projects for trying stuff rather than finishing a game for release. Still, last drive I purchased was about $35 Aussie dollars.

Right, and even that is already at least 500x the amount of money we're talking about with regard to the per-project storage quotes given.

I'm not saying that budget conscious developers should be ignored. I'm asking if there's really any practical benefit to optimising this specific thing in the broader context of game development.

 

Jetbrains problem is that ReSharper is hosted in a 32bit program (Visual studio). They need to move the entire thing out of process and use memory pipelines to update the UI. But they are never taking the time to do this, I guess they are waiting for Microsoft to make VS 64 bit

 

OK I've done some basic analysis of my project drive using a duplicate file finder; for files > 10 MB with identical content and file names.

There are 16.4981 GB of duplicated files (>10 MB) which if cached would only take up about 5.1048 GB.

Might do another scan with all duplicates including <10 MB...

 

How many projects are there? How many of these files are contained within asset folders? What are the files specifically?

This is a lazy analysis, not basic.

 

And how many are from the Package Manager? Are projects the only thing on the "project drive"?

 

OK Full scan of all content duplication across my projects folders:

26.406 GB total of all duplicate files of all sizes matched by filename, size and content
8.226 GB total size of files if cached.​

Therefore there is a potential saving of up to 18.18 GB* in my case if Unity adopted a more Cache based system.

*Admittedly a lot of these files could be startup default files you get from creating a Unity project, but could Unity adopt a file by reference system that only stores a local copy of a file when it is altered?

 

How big is your projects folder in total? How many projects do you have there?
In isolation, those numbers mean nothing.

 

Are they truly duplicates though? Are you running a difference checking program on them?