Video Interactive Film/Music Video

Hey people,

I am currently working on a university project for which I want to create an interactive music video. Someone suggested using Unity for this, and now I'm looking for some advice.

Firstly, I have ZERO experience with code and creating games / apps, although I am going to hopefully change that by attempting this project. I do realise what I want to do is incredibly ambitious and that I might have to make a simpler version of my idea for now, considering time restraints and the difficulty of some of my ideas.

I want to work with only video and audio, and can't find much on this as most stuff is game based. I don't really know where to start, or what to learn first. Is there anything you guys could suggest I look at or try?

It's basically one song, where depending on the decisions, the song sections are different. I would need to be able to loop bits of sound when decisions are being made and the transitions between different audio files needs to be smooth.

(The audio files would all be the same song with the same A section, and then depending on the decision it would go into a different B section. The transition would have to be smooth)

Here are the things I would like to incorporate:

- Basic Story Branching Decisions/Questions (Think Bandersnatch)

- Story line changes triggered by Body Vital Signs (Heart Rate) etc. (I want to connect Smart Rings/Watches which the users wear to collect this data.)

- Gestures using gyroscopes and accelerometers inside phone/smartwatch/smart rings (So the listener/viewer can physically move his/her arm/finger to swipe and make decisions)

- Visual interactive elements (Again based on body vital signs, where the colours of the video might change depending on body temp.)

- Parts with 360 video (For Vr viewing too) (not for the full video, only when decisions are made.)

I want the music video to be available as an app and in-browser.

As I said, it's probably gonna be limited to just the basic decisions as the other stuff it too hard to do for me at this point..

 

I'm kind of heading in a similar direction and have come to terms it is going to take quite some time to get there. To date, I am learning Unity, Cinema Suite, Koreographer and Playmaker.
PlayMaker: https://assetstore.unity.com/publishers/311
Koreographer: https://assetstore.unity.com/publishers/15140
Cinema Suite: https://assetstore.unity.com/publishers/4892

Docs, tutorials, trial and error....then, one day, it will all fit together. Goal is to find as many tools as possible to get the ideas in motion before the grind saps enthusiasm. I've tried similar things in the past attempting to stick to all free tools and the minutia can kill your vision.

 

Hey,

I appreciate your comment!!

I will check those links out.

Cheers,


Folly