Native iOS Shader Integration

Hello everyone... I'm a huge fan of the Unity3D engine. I've been integrating some interestingly new hardware for iOS and using GLSL to speed up math calculations that would prohibitively eat up some fairly serious CPU cycles. Everything works great in my prototypes, but I've been recently trying to wire my framework into the awesome Unity3D engine. Any help would be greatly appreciated.

My native iOS (ES 2.0) glUseProgram shader works fine until I start rendering any vertices on the screen within Unity. At this stage, because of the inherent behind the scenes code handled in the Unity managed area I've kind of thrown in the towel. I get the same issue revealed by @pweeks in this thread where the glGetError returns back an INVALID_OPERATION at the start of my offscreen framebuffer operations. I'm assuming that there is some magic Unity is doing behind the scenes that is still in the GL pipeline or resident on the GPU (perhaps physics magic???) while my code is attempting to run. If I disable all the vertices and objects from rendering on the screen, simply having a Unity "Main Camera" with a background color set, my code runs like a champ.

My example setup is as follows...

Code (csharp):

1. - (void) Repaint //Unity Native iOS method.
2. {
3.    [myGlobalMutex lock];  //just for testing. this is an NSLock object for now.
4.    
5.    //standard Unity magic calls into managed/library handled methods.
6.  
7.    [myGlobalMutex unlock]; //just for testing
8. }
9.  
10. /* Elsewhere in another class far, far away... but within scope.  Running asynchronously... */
11.  
12. -(void) myGLMagic:
13. {
14.     [myGlobalMutex lock];
15.     /* At this point in my pipeline a glGetError will return an INVALID_OPERATION
16.        after a few iterations and vertices are rendered on screen */
17.  
18.     //Copy the old state of the Unity GL
19.     GLint oldFB[1] = {0};
20.     glGetIntegerv(GL_FRAMEBUFFER_BINDING, (GLint *)&oldFB);
21.     GLint oldRB[1] = {0};
22.     glGetIntegerv(GL_RENDERBUFFER_BINDING, (GLint *)&oldRB);
23.     GLint oldVP[4] = {0,0,0,0};
24.     glGetIntegerv(GL_VIEWPORT, (GLint *)&oldVP);
25.  
26.     //set my new GL state here. (offscreen frame buffer magic)
27.     [gl setPositionThresholdFramebuffer];
28.    
29.      glGenTextures(1, &myFrameTexture);
30.      glBindTexture(GL_TEXTURE_2D, myFrameTexture);
31.        
32.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
33.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
34.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
35.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
36.  
37.     //Snip place some data into the buffer
38.     glTexImage2D(...)
39.  
40.     glUseProgram(myFancyShaderProgram);
41.  
42.     //snip.. apply my uniforms to the shader...
43.     //snip... adjust the vertex attrib pointers...
44.     glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);  //do it!
45.  
46.     //snip... read the final texture information after the shader magic has been applied.
47.  
48.     glDeleteTextures(1, &myFrameTexture);
49.  
50.     //set the old Unity state back
51.     glBindFramebuffer(GL_FRAMEBUFFER, oldFB[0]);
52.     glBindRenderbuffer(GL_RENDERBUFFER, oldRB[0]);
53.     glViewport(oldVP[0], oldVP[1], oldVP[2], oldVP[3]);
54.    
55.    [myGlobalMutex unlock];
56. }

Regards,
Chris

 

Hello again. I have come back to revisit this endeavor after working the implementation into a Cocos2d framework. I have a feeling that I've been working "against the grain". My new goal is to integrate my shader with a Camera within the managed Unity tier. I'll then use the Update() method to call into my extern "C" method that returns back a buffer of data that I'll operate on with the shader.

Any further insight into such an operation would be greatly appreciated! I'm concerned about the initial cost of managed code calling into an unmanaged area, but I have faith that it will operate fairly quickly.

Kind Regards,
Chris

 

pass out the getinstanceid of a render texture (requires unity pro) or texture 2d, operate with it on the opengl side (but ensure to leave the state intact or it will backfire) and it should just work basically.
There is no need to return the buffer if you update the texture on the native side directly.

And the calling costs are rather low given you do anything reasonable outside. if you simply call a plugin cause you are not willing to write the glsl in compliance with the light stuff around it in shaderlab, then its definitely an overkill.

what you though can skip likely is the mutex as unity does not respect any kind of mutex, neither unmanaged nor managed, so the only thing you can lock out is yourself and as you execute the rendering upon call, that will never happen either as far as I see.

Keep in mind that you can't integrate unity just as 'rendering backend', its no SDK or open source thing like Cocos2D, it generates full selfcontained applications and its codebase is designed to work like this. For example you can't unload unity and load it up later again, its go up and stay there or be gone till restart.

 

Thanks for this dreamora. I'll give the "getinstanceid" trick a run for the money.

I will definitely remove the mutex implementation since it seems that the OpenGL execution is completely synchronous here. I had been working on the problem too long when I wrote that last post and was starting to make some strange assumptions.

The goal for this endeavor is to get my prototype toy hardware working with Unity for game play using the toy. I've been using an off screen GL buffer in order to aid the toy in performing responsively.

Thank you again for your input on this.
Kind Regards,
Chris