Question Training On Windows 11 WSL2 Ubuntu is slower than Ubuntu

Hello everyone. I have two computers with the same hardware. One with Windows 11, Ubuntu 2204 over WSL2 installed. One with Ubuntu 2204 installed. I train on two machines with the same configuration. Windows 11 training takes about 1100~1200 seconds for 100000 steps. The Ubuntu machine only needs 300~400 seconds to train 100000 steps.

I notice wsl2 access windows file system is extremely slow, so all files are on wsl2's own file system. I have followed this https://docs.nvidia.com/cuda/wsl-user-guide/index.html to installed cuda correctly. GPU usage is nearly full from windows taks manager.

Does anyone know what the problem is? Or can you give me some advice on how to debug this issue?

Solved:
I really don't know what's the real problem of wsl2 Ubuntu environment. But I use nvidia docker over wsl2 solved this problem. Now the GPU usage and training speed is the same as physical installed Ubuntu machine.

I guess there're something wrong with nvidia driver or cuda installation on wsl2 Ubuntu, but I have no interest to debug this problem.

 

Linux is just the better OS. That is it. When I use a usb-stick on my Windows11, it is like 5 times slower than on the same machine with Linux. Also if you use a virtual environment it should be obious to you that it will introduce some overhead thus slowing down the process. Why dont you just compile the project for Windows and use it that way?

 

Our studio's workflow is only support windows system. Sad. I just want to fully use my windows machine during daily work.

 

I am sorry that I can't help you with your problem then. But what is your goal exactly? You can let it train on both Windows and Linux, I train with 16 environments with 32 agents each and I can still work normally.