More hands-on difference between CPU and GPU processing?

So this is the question that bugs me for some time.

Some time ago, GPU processing was introduced. It allowed users to execute computer programs using GPU. There's tons of articles which explain that CPU have fewer cores and are more sophisticated as they're meant to process very general tasks, whereas GPUs contain thousands of smaller cores which are far more simplistic in their nature and therefore can execute tons of relatively simpler tasks far faster than CPU because it can distribute it load over thousands of cores. So if you have simple divisible mathematics you generally want GPU to process it, and if equations couldn't be divided easily and require a lot of muscle, it should be passed to CPU.

But it doesn't tell me crap. Could anybody spare some time and pull off a piece of code and explain me with two examples, why example A would be better for CPU and example B would be better for GPU? I can read and remember all the cold facts I want, but I don't fully comprehend it. It really bugs me.

I can understand more and simpler cores are fantastic for Machine Learning, where there is thousands of simple computations and barely any advanced mathematics*. But how does it translate to regular programs? What parts of say, Unity C#, would work well with CPU and which with GPU? What parts of which code do benefit CPU/GPU most? Is it possible to write a CPU or GPU optimized program?

* I know, I know.

 

If you want the simple answer? Everything runs on the CPU except shaders. As a developer, you don't need to know or worry about it. We use high-level development tools so that we don't have to worry about that sort of thing.

 

Imagine you write a large book, with thousands of chapters.
If each chapter is detached from each other, but have common base, you can ask multiple people to write each, or few chapters. (GPU).

However, if your chapter are highly dependent on each other, to be able continue, or even start, or long, but fewer of them (you run fewer task), then only handful writers, or even single person is suitable. (CPU).

Yes there are task which can be shifted to CPU/GPU, but they may require more specialized code.

 

Basically if it's https://en.wikipedia.org/wiki/Embarrassingly_parallel then it's ideal.

 

The biggest problem I can think of for using the GPU to process what you might otherwise use the CPU for in a game, is there is usually a big job the GPU is already busy with...... rendering the game.

 

For example of GPU program, look up some compute shader, ideally in HLSL as it is closer to C# than other.

 

Good for GPU - highly parallel and can be split into, say, one million threads doing the same thing.
Good for CPU - hard to parallelize, complex flow control, or threads talk to each other often.

https://docs.unity3d.com/Manual/class-ComputeShader.html

For example, "multiply one billion numbers by five" --> works better on GPU.
"Process message loop on windows platform", "implement a chat client" --> works better on CPU.

 

It also important to know that GPU is optimized for high-throughput, high-latency workload. So if you need to process data immediately, CPU would be the only choice.

 

Would Intel Xeon Phi be a good middle-man solution?
https://ark.intel.com/content/www/u...-phi-processor-7295-16gb-1-5-ghz-72-core.html

It looks like a coprocessor with 72 cores at 1.5GHz. If I wanted high parallelization like GPU, but keep CPU functionality, that looks like an option, right? A solid alternative?

 

No. Unless you build highly specialized app for this kind of processor. Obviously, you are not targeting wide audience in such case, but more like private / commercial product.

You better of using array of computers, multi process hardware, servers, or in clouds services.

 

No, it isn't a solid alternative at all.

GTX 1060 has 1280 CUDA cores, which are a minimum computational unit for gpu calculations.
https://stackoverflow.com/questions/20976556/what-is-the-difference-between-cuda-core-and-cpu-core

The point of using GPU is that most people have one and it is capable of running things at higher speed than CPU would, assuming the problem being solved can be parallelized.

Additionally apparently Xeon Phi has been discontinued:
https://www.extremetech.com/extreme/290963-intel-quietly-kills-off-xeon-phi

Then there's a matter of price tag.

 

No. Performance of the Xeon Phi is nowhere near the performance of a GPU, and worse yet it's only just barely ahead of the current generation (which are scheduled to be replaced within the next six months) of processors from AMD.

Xeon Phi has a maximum GFLOPs of 1,208.

https://en.wikipedia.org/wiki/Xeon_Phi

An AMD 3950X has a maximum GFLOPs of 1,038.

https://gadgetversus.com/processor/amd-ryzen-9-3950x-specs/

A GeForce GTX 1060 has a performance of 4,375 GFLOPs.

https://www.techpowerup.com/gpu-specs/geforce-gtx-1060-6-gb.c2862

A Radeon RX 5700 XT has a performance of 9,754 GFLOPs.

https://www.techpowerup.com/gpu-specs/radeon-rx-5700-xt.c3339

A GeForce RTX 2080 Ti has a performance of 13,450 GFLOPs.

https://www.techpowerup.com/gpu-specs/geforce-rtx-2080-ti.c3305