Passing audio buffer to Unity

Hi

We're a small team working to interface Puredata with Unity (http://puredata.info/).

We have a working bridge:

Code (csharp):

1.  
2. using UnityEngine;
3. using System;
4. using System.Runtime.InteropServices;
5.  
6. public class ZGUnity : MonoBehaviour
7. {  
8.     [DllImport ("ZGUnity")]
9.         private static extern IntPtr zg_new_context(int numInputChannels, int numOutputChannels,
10.         int blockSize, float sampleRate, long function, long userData);
11.    
12.     [DllImport ("ZGUnity")]
13.         private static extern IntPtr zg_new_graph(IntPtr context, string directory, string patch);
14.    
15.     [DllImport ("ZGUnity")]
16.     private static extern void zg_attach_graph(IntPtr context, IntPtr graph);
17.  
18.     [DllImport ("ZGUnity")]
19.         private static extern void zg_process(IntPtr context, float[] inputBuffer, float[] outputBuffer);
20.    
21.     [DllImport ("ZGUnity")]
22.        private static extern void zg_delete_context(IntPtr context);       
23.    
24.  
25.  
26.     const int numInputChannels = 2;
27.     const int numOutputChannels = 2;
28.     const int blockSize = 256;
29.     const float sampleRate = 44100.0f;
30.     float[] inputBuffer = new float[blockSize * numInputChannels];
31.     float[] outputBuffer = new float[blockSize * numOutputChannels];
32.    
33.     IntPtr zgContext;
34.     IntPtr zgGraph;
35.    
36.     void Awake() {
37.         zgContext = zg_new_context(numInputChannels, numOutputChannels, blockSize, sampleRate, 0, 0);
38.         zgGraph = zg_new_graph(zgContext, "/Users/dpc/Desktop/", "simple_osc.pd");
39.         zg_attach_graph(zgContext, zgGraph);
40.     }
41.  
42.     void Update() {
43.         zg_process(zgContext, inputBuffer, outputBuffer);
44.         Debug.Log(outputBuffer[0]);
45.     }
46.    
47.    
48. }  
49.  

But, now we need to pass the audio buffer (outputBuffer) to Unity's own audio session...I've read a few examples including Prime31's AudioRecorderBinding and some Objective-C code, but I'm stumped that there isn't a clear way to just pass audio samples into Unity's engine...am I wrong?

Does anyone know of a way we could do this simply?

David

 

Hi,
Audio buffer access is not available now (and in 3.0). We are aware that this is one of the most wanted audio features - and we are working on it. But again, we want to do it "the right way" to ensure low latency, while still interfacing Mono (think GC, threads etc.)

If you're running standalones you can use the FMOD plugin from http://www.squaretangle.com/FMODUnity.html.

 

Thankyouthankyouthankyou for replying and letting us know! Phew, such a relief to know you're working on access to audio buffers.

Just so you know, we're (today/tomorrow) interfacing Puredata to RemoteIO to be able to use zengarden (puredata runtime lib) in Unity through the c# layer. We can already pass data back and forth, but being able to make puredata audio point sources to, say, and AudioListener, would be fantastic (3d positioning, etc.).

I have to ensure we can target iOS, so squaretangle's work won't help here (or will it?).

Thanks again for listening, can't wait to see what you do. We're aware low latency and audio clock timing is not easy. Good luck!

David

 

Just for those still interested, we have a working bridge for Unity-Puredata (iOS and Android) on github: https://github.com/hagish/kalimba.

Cheers,

FBP

 

Thank you so much for kalimba, i was wondering if it works with unity webplayer or not? or is it only for smartphone?

Thanks again Fbp, and continue the great work

 

I'm interested in this too... has anyone used Pure Data in a Unity project, that we might be able to hear in action?

 

The guys working on Fract are making good use of pd in Unity.

 

Any example of a circular buffer of data flowing from pd into unity?