Trying to run a linux standalone on aws ec2 instance

I'm trying to run a linux standalone on an aws ec2 instance.

I created an ec2 instance on amazon of type small with ubuntu 14.04. I added ubuntu desktop and tightvncserver.
I opened the vnc port on the ec2 instance.
I also installed nvidia drivers on the ec2 instance although it has no real graphics card.
I also installed the mesa opengl software -- supposed to run opengl without a graphics card.

I created a unity linux project that simply writes a file to the streamingAssets folder. I set the build settings for headless. This seemed to work and the file was written and no errors occured in the log.
'/home/ubuntu/Desktop/unitytests/writefile/writefile.x86_64' -nographics -logfile log.txt

My actual unity project called "aws" runs fine on an ubuntu vmware vm running in vmware fusion on my mac. It also uses CoherentUI. On aws it gives many errors in the log. Could someone help me troubleshoot these errors? I'm confused as to which ones are related to vnc problems or unity problems, or possibly coherentui problems. If possible I'd like to run it so I could also see the graphics.


Here is the display info and below that is the log file

ubuntu@ip-172-31-9-169:~$ lshw -C video
WARNING: you should run this program as super-user.
*-display UNCLAIMED
description: VGA compatible controller
product: GD 5446
vendor: Cirrus Logic
physical id: 2
bus info: pci@0000:00:02.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: vga_controller bus_master
configuration: latency=0
resources: memory:f0000000-f1ffffff memory:f3000000-f3000fff
WARNING: output may be incomplete or inaccurate, you should run this program as super-user.
ubuntu@ip-172-31-9-169:~$ glxinfo
name of display: :1
Error: couldn't find RGB GLX visual or fbconfig
Error: couldn't find RGB GLX visual or fbconfig​

Xlib: extension "XInputExtension" missing on display ":1".
Xlib: extension "XInputExtension" missing on display ":1".
Xlib: extension "RANDR" missing on display ":1".
Xlib: extension "XInputExtension" missing on display ":1".
Xlib: extension "XInputExtension" missing on display ":1".
[2696:2696:INFO:UIContentClient.cpp(74)] GPU info collected in gpu-process process:
[2696:2696:INFO:UIContentClient.cpp(79)] Main GPU vendor 4115 device 184
[2696:2696:ERROR:gl_surface_glx.cc(354)] glxQueryVersion failed
[2696:2696:ERROR:gl_surface_x11.cc(57)] GLSurfaceGLX::InitializeOneOff failed.
[2696:2696:ERROR:gpu_child_thread.cc(185)] Exiting GPU process due to errors during initialization
op/robot8_Data/Managed/UnityEngine.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-CSharp-firstpass.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-CSharp-firstpass.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-CSharp.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-CSharp.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-UnityScript.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/Assembly-UnityScript.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/UnityEngine.UI.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/UnityEngine.UI.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/UnityEngine.Networking.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/UnityEngine.Networking.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/CoherentUINet.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/CoherentUINet.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/ICSharpCode.SharpZipLib.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/ICSharpCode.SharpZipLib.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/InputSimulator.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/InputSimulator.dll into Unity Child Domain
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/CoherentUICommon.dll (this message is harmless)
Loading /home/ubuntu/Desktop/robot8_Data/Managed/CoherentUICommon.dll into Unity Child Domain
- Completed reload, in 0.038 seconds
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/System.Core.dll (this message is harmless)
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/System.dll (this message is harmless)
The referenced script on this Behaviour (Game Object 'placewindow') is missing!
(Filename: Line: 1655)

UnloadTime: 0.756000 ms
[Coherent Browser] (Info) Initializing Coherent UI ver. 2.6.2.0
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

Coherent Browser system initialized..
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Error) Unsupported Unity3D renderer
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/Boo.Lang.dll (this message is harmless)
Platform assembly: /home/ubuntu/Desktop/robot8_Data/Managed/UnityScript.Lang.dll (this message is harmless)
unity start
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

IndexOutOfRangeException: Array index is out of range.
at cgi+$Start$30+$.MoveNext () [0x00000] in <filename unknown>:0
(Filename: Line: -1)

[Coherent Browser] (Info) PID: 2676 | 2676 17:42:22.088166 Staring Coherent UI Host ver. 2.6.2.0
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) PID: 2676 | 2676 17:42:22.088209 Detected OS: Linux 3.13.0-74-generic
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Warning) PID: 2676 | 2676 17:42:22.089225 [2676:2676:ERROR:browser_main_loop.cc(164)] Running without the SUID sandbox! See https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more information on developing with the sandbox on.
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) PID: 2676 | 2676 17:42:22.100840 [2676:2676:INFO:audio_manager_pulse.cc(258)] Failed to connect to the context. Error: Connection refused
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) PID: 2676 | 2676 17:42:22.121693 [2676:2676:INFO:UIContentClient.cpp(74)] GPU info collected in host process:
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) PID: 2676 | 2676 17:42:22.121704 [2676:2676:INFO:UIContentClient.cpp(79)] Main GPU vendor 4115 device 184
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) Coherent UI context initialized successfully
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Warning) PID: 2676 | 2676 17:42:22.178273 Requesting resource read for file:////home/ubuntu/.config/unity3d/DefaultCompany/Robot7/cookies.dat with internal id 0 reported FAIL
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Error) Trying to create Unity3D texture renderer with invalid type = -1. If you have changed the renderer type in Unity, please restart the Editor
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Error) Trying to create Unity3D texture renderer with invalid type = -1. If you have changed the renderer type in Unity, please restart the Editor
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Warning) PID: 2676 | 2694 17:42:22.228730 [2676:2694:WARNINGroxy_service.cc(899)] PAC support disabled because there is no system implementation
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)

[Coherent Browser] (Info) View creation failed because no valid surface was provided by the client
(Filename: /home/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)​

 

It looks like you're trying to run a normal player on a headless instance of Linux. Build a headless player instead (tick the Headless option in the build settings).

 

I tried two more things. I tried to take a screenshot in my test that writes to a file. The screenshot in the editor looks fine, on the linux build it looks black with random color pixels at the bottom. I also ran a new headless version of the aws project which gives the same output. Maybe the CoherentUI plugin I am using is overriding the headless mode. Is a screenshot possible in headless mode? Is there any possibility of running this without being headless?

 

Hmm, yes, if you are using a headless player, then it looks like the errors are all from the plugin you're using; it may not be meant to be used in a headless build. I cant tell for sure without the entire log file, but it looks like the plugin itself might be relying on Xlib and expecting a running X session.

 

However, the log looks like maybe you're trying to run the Editor, not the player? I can't tell. In any case, the Editor for sure won't work on a headless version of Linux.

 

I'm trying to run a standalone on the aws ec2 instance. Is it possible to run in non-headless mode without a graphics card somehow? That's what I was hoping for since the ec2 instances don't have graphics cards.

 

Update. I'm also trying an aws gpu instance on unity which includes a gpu. Unfortunately it gives the same exact errors which is odd.

 

For the standalone player:

• it is possible to run the headless player without X or a gpu, provided you're not using plugins (e.g. one that embeds CEF (!)) that require either of those
  
• it is possible to run the non-headless player without an X session, but the libraries are required to launch the executable

For the editor:

• currently a valid X session is required, even for nographics mode (this is a known issue which we plan to remedy, but we have no timeline at the moment)

You keep saying "standalone", but as natosha mentioned, the log you've posted is clearly from the editor. Which one are we dealing with here?

 

The log is from the linux standalone not the editor. There is no editor installation running.
from the command line I am running
'/home/ubuntu/Desktop/aws.x86_64' -logfile log.txt.

I have ubuntu desktop installed on the aws instance and can see the ubuntu desktop running when connected to vnc. Does this mean a valid X session is already running?

I would like to run it non-headless with an X-session.
I'm confused about what is missing from the aws installation because the project runs great on a normal linux machine, and a mac running a vmware ubuntu vm.

 

Yes. If you're not launching the player from that session (e.g. from a remote ssh connection, or as a child process from a network service), you'll need to ensure the following:

• The DISPLAY environment variable is correctly set in the environment where you're launching the player (open a terminal in your user session and echo $DISPLAY - usually :0 is correct/sufficient)
• The Unity player process is launched by the owner of the desktop session

 

Through ssh echo $DISPLAY is blank.
ssh:
$ lspci | grep VGA
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
$ glxinfo
Error: unable to open display

Right now it appears that unity thinks there is no graphics environment to draw anything. Is it possible to configure the ubuntu instance to use a virtual display or a software version of opengl?

The project would eventually launch multiple aws instances each one would run, take a screen shot, then close.

 

Yes - to use the display, you need to ensure that the DISPLAY variable is set in the session where you launch the Unity player. You can find the correct value to set by connecting to the graphical session (e.g. via vnc), opening a terminal, and running echo $DISPLAY (again, this usually resolves to :0 , but I don't want to make any assumptions about your setup).

You can set this in your session by running export DISPLAY=whatever just before launching the Unity player. Again, for any of this to work, the owner of the graphical session has to be the same user with which you launch the Unity player.

 

I created a simple unity project without CoherentUI that would take a screenshot. How would I call this from ssh and what would have to be setup ahead of time for this to work?

I tried to set the Display through vnc $ echo $DISPLAY is :1
on ssh $ echo $DISPLAY is blank
I tried export Display=:1 on ssh but it does not set
echo $DISPLAY is still blank

glxinfo on ssh is
Error: unable to open display
You mentioned before that if glxinfo won't work then nothing will work.

Would installing the unity editor on the aws instance help set up the aws machine for running the standalone?

 

Case matters here - if you set Display, it won't affect DISPLAY.
From the information you've provided, it does seem that export DISPLAY=:1 is the appropriate command for the ssh session in your situation.

 

I used the correct case in the actual command but did not type it correctly in the post.
Are there any resources or paid services you could recommend to help me troubleshoot and try and solve this problem since I seem to be stuck.

 

Is it possible to use Xdummy or xpra to run this without a graphics card on an aws ec2 instance?

 

Yes, potentially (I haven't tried either) - however, if you already have a running aws instance where there's an x session, you should already have everything you need.

 

I realize that this is an obscure request -- to get unity with coherent to run on an aws instance when you're not looking at it, but I'm totally confused on how to set this up. Although if this works anybody can run a unity standalone with graphics on an aws instance and fire up 100 of them to collect data and screenshots.

My understand so far:
Unity and Coherent together need a graphics surface which would be a display, a graphics card, and open-gl to run. The ec2 linux service has none of these. To run it I'd have to somehow setup a virtual display, a virtual graphics card, and also open-gl software rendering and launch unity in such a way that it uses them without me looking at it. I'm really confused about how all these things change when I remote into the instance vs when I'm not observing. Seems as tricky as quantum mechanics to me so far.