Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Help Wanted Training a Tensorflow model for DSP within Unity

Discussion in 'Barracuda' started by unity_KO-VCkmyhwJWTw, Nov 30, 2020.

  1. unity_KO-VCkmyhwJWTw


    Aug 20, 2020

    I've been charged with building a setup, where a pair of sound signals could be generated within Unity 3D environment and directly passed to Tensorflow for training general models, which learn to generate signal 2 based on signal 1 i.e modulate it in a desired way.

    I've tried to find suitable instructions, but it seems Barracuda is intended for loading already trained models and all the examples I've found about using ML-agents are concerned with reinforcement learning rather than signal processing. I'd like to ask
    1. Is using real time audio signals from Unity to train a Tensorflow model possible and feasible?
    2. How and from where do I begin to learn how to do it?
    One challenge is that while I've dabbled with Unity a little, I've done practically no AI or ML stuff and never used Tensorflow or Python. I'm more than willing to learn, though. I apologize for my lack of competence in these matters, but I hope the question is understandable enough for someone to provide me some pointers. Thanks in advance.


  2. alexandreribard_unity


    Unity Technologies

    Sep 18, 2019