Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Unite 2023 Registrations are now LIVE!

Question Running UI Toolkit with unity in batch mode for visual test in the CI

Discussion in 'UI Toolkit' started by Suduckgames, Aug 23, 2022.

  1. Suduckgames

    Suduckgames

    Joined:
    Nov 28, 2016
    Posts:
    218
    Hi there! We are using UI Toolkit for our development and we wanted to add some visual test to ensure that nothing is changed by accident.

    Basically, we have a baseline image that we compare pixel by pixel with the current result of the image that we render in the test

    We have a CI with CircleCI that will run the test with unity in batchmode.

    We are looking at how to implement the visual testing but we are getting some problem. It works well in normal mode but when we try with batchmode

    We are tested the following approaches:

    1. Try to render the UI Document directly as normal ( after the cameras). The problem is that Screencapture is not supported well in batchmode and the result of the image is not including the UI Document. However the normal canvas with an image (iGUID) is renderer correctly
    2. Since IGUID is rendered correctly we tried to render the UI Document in a renderer texture that then will be renderer in a canvas with a RawImage. But it seems that the renderer image is not renderering the UI Document either
    3. We added a camera and changed the canvas to be ScreenSpace camera to use the camera to take snapshots. The problem persists, no UI Document is rendered
    4. We changed the canvas to be WorldSpace and settled them in positions to be visible. The problem persists





      Is any way to accomplish this in batchmode?
     
    Suduckgames, Aug 23, 2022
    #1

  2. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    733
    Can you confirm that nothing is rendered when using batchmode even a local development machine, not just CircleCI ?
    Can you also share the Editor.log file as well as any relevant information about the machine (GPU, OS)

    Thanks
     
    antoine-unity, Aug 23, 2022
    #2

  3. Suduckgames

    Suduckgames

    Joined:
    Nov 28, 2016
    Posts:
    218
    Thanks for the reply! Yes, I am opening the project locally with batch mode from the console. I am not in the machine, I will give you the log tomorrow

    I have tried Unity 2021 and Unity 2019 without success

    I have a GTX 1080 TI with Windows 10
     
    Suduckgames, Aug 23, 2022
    #3

  4. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    733
    We believe this should work as we have similar tests internally who do something similar.

    Although we are using utilities from the Graphics Tests Framework (experimental) for Image Comparison.

    Can you share the command line as well ?

    It feels like a bug report would be the best way to proceed, to understand your setup a little bit more.
     
    antoine-unity, Aug 23, 2022
    #4

  5. Suduckgames

    Suduckgames

    Joined:
    Nov 28, 2016
    Posts:
    218
    Sure!

    This is the command line that we use to run them

    xvfb-run --auto-servernum --server-args='-screen 0 640x480x24' $UNITY_PATH/Editor/Unity \
    -batchmode \
    -projectPath "$PROJECT_PATH" \
    -logFile "$PROJECT_PATH/playmode-logs.txt" \
    -runTests \
    -testPlatform PlayMode \
    -testResults "$PROJECT_PATH/playmode-results.xml" \
    -enableCodeCoverage \
    -coverageResultsPath "$PROJECT_PATH/CodeCoverage" \
    -coverageOptions

    Since it may be related to a bug we decide to change the approach and delay the visual test for the future. Thanks for your help!
     
    Suduckgames, Aug 24, 2022
    #5