Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

Unity Benchmark v0.1

Discussion in 'General Discussion' started by Arowx, Sep 20, 2012.

  1. bocs

    bocs

    Joined:
    May 9, 2009
    Posts:
    335
    2 years later, I still don't see the purpose of this "Benchmark"

    You seem to think it tests GPU, since it has "Quality" settings and based on FPS

    Unless someone is running an ancient or integrated GPU, the test is kinda pointless
    My 3 year old card never hits more then 25% usage, mainly limited by Unity's single threaded nature.

    Does no body else see how flawed this "Benchmark" is?
    It's a good idea, I just don't think it's executed correctly.

    At best this measures how many objects Unity can handle on the CPU per frame.
    Maybe that's the point, but then why have graphics "Quality" options.

    How about a high poly cube, so it taxes the GPU more?
    then you could then have metrics for CPU and GPU.
    *point is to max the GPU, and have the CPU waiting
     
  2. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    So you would like this benchmark to test your GPU and DirectX or OpenGL and not Unity?
     
  3. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  4. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Just made a quick version (think) alpha of Cube Mark with the Unreal game engine.

    And thought you might want to compare and contrast.



    That's right my PC can throw around 534.0 Unreal Cubes @ 30 fps. I think this test equates to the Unity versions with colliders test at the highest quality setting.

    Now Unreal uses PBR and Unity 5 will be using PBR I'm wondering if we will see a similar drop in quantity for the boost in Quality.

    Added this early alpha Unreal Cube mark to the itch.io page http://arowx.itch.io/unity-cube-mark
     
  5. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  6. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  7. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    LOL not at you with you at me. In the process of making the Unreal Cube Mark, and realised that my benchmarking loop is benchmarking the Creation of cubes on top of a scene with cubes or CPU!

    My apologies you're absolutely right. DOH!

    Coming Soon -> Cube Mark 2.0 test your CPU or GPU!

    Cheers
     
  8. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    OK Quick update built for Unity 5 FREE!

    Running 3x Fantastic @ 2560 x 1600 full screen on my PC.

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899

    Well done Unity!

    But don't take my word for it try it for yourself...

    http://arowx.itch.io/unity-cube-mark

    Oh and Webplayer on itch.io
    30 fps > 45,061
    60 fps > 34,371
     
    Last edited: Mar 3, 2015
    ZJP likes this.
  9. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  10. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158


    Unity WebGL build! (in Chrome)

    All Quality Settings x3. Not bad. :)

    And in Firefox Nightly

     
    Last edited: Mar 4, 2015
  11. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    And for comparison the WebPlayer version.

     
  12. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  13. DarkJoney

    DarkJoney

    Joined:
    Jan 31, 2013
    Posts:
    24
    Hey, Arowx, I have question. I want to create graphics benchmark with Unity, but I don't know how to make score of the test. Can u help me with this?
     
    Last edited: Apr 8, 2015
  14. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Most benchmarks record the fps (there are free scripts to do this on the asset store) or count the number of frames the machine can produce in a given scene, which you can do by adding one to your score each Update().

    Check the scripting forum if you need more help.
     
  15. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  16. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    1920 x 1080 / Fantastic
    Capture.PNG

    Will have to run again, had a better score on 60fps earlier. :)

    Intel 5930k
    NVidia Gtx 980
    32 GB DDR4
     
  17. nickyoso1

    nickyoso1

    Joined:
    May 2, 2011
    Posts:
    85
    77937 on 1920 x 1080 / Fantastic / 30 fps
    around 72k on 2560 x 1080 / Fantastic / 30 fps

    Intel i5 something
    Nvidia gtx 770 2GB
    8 GB DDR3
     
  18. Spoken_OS

    Spoken_OS

    Joined:
    Feb 15, 2015
    Posts:
    29
    Apparently I can only handle 300 cubes at 60 FPS...
     

    Attached Files:

  19. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Was that in the webplayer, as the browser could limit your FPS, if it is you can try another browser or download the standalone version.
     
  20. jpthek9

    jpthek9

    Joined:
    Nov 28, 2013
    Posts:
    944
    Just wondering, do these cubes have colliders on them or are they just for rendering?
     
  21. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    One test with colliders another without for both cubes and quads so 4 tests in total.
     
  22. davem250

    davem250

    Joined:
    May 28, 2013
    Posts:
    186
    i'm quite happy with the results :p 97,715 at 30fps and 73,016 on 60 fps, you can see in the screenshot the exact specs of my system :)

    Screenshot_collapsed.png

    i'm running at fantastic settings and res is 2560 x 1440!
     
  23. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    LOL odd background pattern going on there that doesn't seem to match the actual apps result screen! :rolleyes:;)
     
  24. davem250

    davem250

    Joined:
    May 28, 2013
    Posts:
    186
    yeah i had to cut Down the original screenshot as it took almost 4 mb :eek: for the 2560x1440 screenshot but yeah you can always say that i cheated by cutting from others :p though i don't do that then i guess it could be fun just to get the attention :rolleyes::p
     
  25. Spoken_OS

    Spoken_OS

    Joined:
    Feb 15, 2015
    Posts:
    29
    Oh I should have realized. Thanks.
     
  26. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Just tried 5.1 with a PC build.

    Running 3x Fantastic @ 2560 x 1600 full screen on my PC.

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0 / 5.1
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498 / 53,177
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899 / 51,693

    Odd a speed up for the 60 fps target but a drop for the 30 fps target?!

    Getting a slight speed boost in WebGL as well.
     
  27. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  28. appslabs

    appslabs

    Joined:
    Aug 5, 2013
    Posts:
    121
    Test results with my monster at 5760x1080 Resolution with all the benchmark settings set true and 3x times 'Run' sampling.




    I've installed Windows 10 Insider Preview on my PC. But it shows Windows 8.1 as installed OS.
     
  29. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Talk about wide screen, how many monitors?

    Well I'm guessing as Win 10 is not out yet Unity probably have not updated how they detect the OS or it could be that the preview version returns a value that is 'compatible' with older programs. I just ask Unity for the system details.
     
  30. appslabs

    appslabs

    Joined:
    Aug 5, 2013
    Posts:
    121
    @Arowx I'm using 3x 27" LG IPS monitor.

    Here's the benchmark result for WebGL
     
  31. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Quick update I have a PC build on http://arowx.itch.io/unity-cube-mark of the Latest Patch version 5.1.p3

    The reason being that it looks like Unity have a added a DX12 driver option to the Build, I don't have Windows 10 to test it on but thought that the developers with preview versions would like to compare DX11 and DX12.

    Note you should be able to specify the graphics driver on the command line http://docs.unity3d.com/Manual/CommandLineArguments.html

    I have the automatic API selection enabled, so fingers crossed DX12 works!
     
  32. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,919
    The DX12 implementation in Unity is a work in progress though, they still aren't using many threads yet, but I understand this is very much an ongoing task so your initial results will get improved on. The main bottleneck here is the CPU.
     
  33. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Well I'm hoping that Unity's batching will detect that the cubes use the same mesh and material and once that is passed to the GPU a new instance (especially the test without colliders) should just be a case of setting up a new instance.

    In effect I'm hoping to see the kind of speed boost that this directx 12 benchmark displays only with lots of cubes!



    So if you have DX12 please video and share your results!
     
  34. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158


    Edge Browser with ASM.js turned on.
     
  35. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Just tried 5.2 with a PC build with DirectX 12 (Experimental, only two render threads)

    Running 3x Fantastic @ 2560 x 1600 full screen on my PC.

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0 / 5.1 / 5.2
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498 / 53,177 / 55,370
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899 / 51,693 / 53,705

    Good work UT @Aras and team!
     
  36. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    CubeMark 5.2 WebGL version here

    My results by browser below.

     
  37. SpookyCat

    SpookyCat

    Joined:
    Jan 25, 2010
    Posts:
    3,081
    Tried the WebGL version but I get an exception error when I click the Run button in Firefox.
     
  38. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Hmm I got an exception in Chrome first time but when I tried again it worked?!

    Anyone else getting this problem and what browser does it happen in?

    [Update] I've updated the build to allow exceptions and I think I have fixed the bug you found!?
     
    Last edited: Sep 8, 2015
  39. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  40. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
  41. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Running 3 x Fantastic @ 2560 x 1600 full screen on my PC. (Latest Results in DX 11)

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0 / 5.1 / 5.2 / 5.2.2
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498 / 53,177 / 55,370 / 56,120
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899 / 51,693 / 53,705 / 53,976

    Note: latest results using DX 11 as DX 12 has gone spotty for me?!
     
  42. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    WebGL benchmark. 5.2.2

    Fastest > MS Edge / Firefox / Chrome
    30 fps > 23,665 / 10,402 / 17,718
    60 fps > 16,299 / 7,778 / 13,100

    Fantastic > MS Edge / Firefox / Chrome
    30 fps > 14,994 / 8,190 / 12,204
    60 fps > 14,335 / 7,944 / 11,724
     
  43. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Running 3 x Fantastic @ 2560 x 1600 full screen on my PC. (Latest Results in DX 11)

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0 / 5.1 / 5.2 / 5.2.2 / 5.3
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498 / 53,177 / 55,370 / 56,120 / 54,460
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899 / 51,693 / 53,705 / 53,976 / 44,427

    That's quite a drop in performance, I'll double check some of my older numbers just to see if it's changes to my PC/drivers. tomorrow.
     
  44. MikeUpchat

    MikeUpchat

    Joined:
    Sep 24, 2010
    Posts:
    859
    Wow that's a bit worrying, its supposed to have new graphics pipeline and improved multi threading, looks like another rushed out release, Ill stick with 5.2 me thinks until the patches come along.
     
  45. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    Did you see my comments on the performance in WebGL on the bug you reported? You should be able to get much better numbers out of it with some minor changes.
     
  46. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Good point, I'm just stuck in a loop automatically running the benchmark when a new release comes out.

    OK based on feedback from Jonas I have updated the WebGL version of the benchmark, there is a performance glitch in WebGL that kicks in when standard shaders are used and Unity needs to drop back to forward rendering. I believe the fix is due in 5.4.

    So I've tried to workaround it using a Legacy Shader and Forward rendering.

    So lets see:
    Code (csharp):
    1. WebGL Forward Standard
    2.  
    3.     Chrome
    4.             Fastest Fantastic
    5.     30 fps     21,621    15,887
    6.     60 fps     16,272    14,201
    7.  
    8.     Firefox
    9.             Fastest Fantastic
    10.     30 fps     17,826    12,792
    11.     60 fps     12,548    10,905
    12.  
    13.     Edge
    14.             Fastest Fantastic
    15.     30 fps     24,061    16,938
    16.     60 fps     17,116    14,633
    17.  
    18. WebGL Forward Legacy Spec Bump
    19.  
    20.     Chrome
    21.             Fastest Fantastic
    22.     30 fps     25,684    19,110
    23.     60 fps     19,256    16,615
    24.  
    25.     Firefox
    26.             Fastest Fantastic
    27.     30 fps     18,617    13,636
    28.     60 fps     13,739    12,022
    29.  
    30.     Edge
    31.             Fastest Fantastic
    32.     30 fps     25,684    18,303
    33.     60 fps     18,225    15,656
    This provides a bit of a boost, but maybe my workaround is suboptimal?

    Here is the 5.3 WebGL version with the issue https://dl.dropboxusercontent.com/u/19148487/CubeMark/Unity 5.3/webGL53/index.html

    And the version with the workaround https://dl.dropboxusercontent.com/u/19148487/CubeMark/Unity 5.3/webGL53f/index.html

    Do you get a larger difference in performance?
     
    Last edited: Dec 9, 2015
  47. ZJP

    ZJP

    Joined:
    Jan 22, 2010
    Posts:
    2,646
    Indeed... :(
     
  48. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158
    Running 10 x Fantastic @ 2560 x 1600 full screen on my PC. (Latest Results in DX 11)

    4.5 / 4.6 / 4.6.1 / 4.6.2 / 4.6.3 / 5.0.0 / 5.1 / 5.2 / 5.2.2 / 5.3
    30 fps > 51,716 / 48,853 / 48,254 / 48,681 / 49,835 / 59,498 / 53,177 / 55,370 / 56,120 / 57,718
    60 fps > 42,230 / 39,983 / 39,337 / 39,490 / 40,723 / 46,899 / 51,693 / 53,705 / 53,976 / 56,026

    OK I did not allow myself to play any videos while running this test, what it's boring watching spinning cubes for hours.

    Details from yesterdays run and todays runs.
    Code (CSharp):
    1.                 3x Yesterday        10x Today
    2.     Fastest    74,183    57,210        74,565    57,551
    3.     Fast       70,578    54,717        63,139    57,442
    4.     Simple     54,997    50,288        60,804    57,331
    5.     Good       51,166    48,373        59,639    57,171
    6.     Beautiful  49,022    47,154        58,771    56,850
    7.     Fantastic  47,652    46,161        57,718    56,026
    So this looks like a significant boost in performance. But I'll push out the build so you can try it for yourselves...

    http://arowx.itch.io/unity-cube-mark
     
    Last edited: Dec 9, 2015
  49. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    Yes, i got a much bigger difference in performance when I tries this (with numbers going up ~3x). But I did not pick a random legacy shader (not sure how Legacy Spec Bump should compare to Standard in performance). You can keep the standard shader if you switch to Forward rendering. Or even better yet, you can fix the deferred rendering performance issue in the Standard shader yourself (but this will break the shader's performance on low-end mobile GPUs, which is why we did not ship this yet):
    -Download the builtin shader source: builtin_shaders-5.3.0f4.zip
    -Copy Standard.shader
    -Rename the Shader internal name in the .shader file
    -delete all "#exclude_renderers" lines
    -Use this modified Standard shader for your material

    Now, you should be able to use the Standard shader in deferred in WebGL.
     
  50. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,158


    OK here are my WebGL results and the adapted standard shader does boost performance in deferred rendering mode.