CoreML examples.

Hello All.

iOS introduces some nice new tech for mobile.

https://developer.apple.com/machine-learning/

I have made a suggestion on Unity Feedback to get simple CoreML integration like the recent ARKit. If you've used these you'll know how simple it is to get going.

These libraries allow many more people to access these technologies.

Thoughts?

 

I can't think of it's use in games off the top of my head. Got any in mind?

 

I agree. Integrate this as a scene into the ARKit unity example @jimmya

https://github.com/yulingtianxia/Core-ML-Sample

 

Core ML enables things like native language processing, image recognition, facial tracking, etc... These could all be amazing tools for games (and non game applications) built in Unity. For example, what if you could issue commands just by speaking in VR. Or what if your game/app could react dynamically to the facial expressions of your user. The possibilities abound.

 

The most impressive opportunity is in combining CoreML with ARKit itself- just imagine being able to place all important game elements based on actual physical settings of the user's environment. You can even create the game around mechanisms of finding something to progress in the game, i.e. find book, to read next clue -> find door -> find keys etc.

 

Hi I've voted for core-ml integration. How many votes needed for implementing this feature in unity? The vote count is 140 now.

 

This is one of the best reasons why this could be useful for games

You could make an AR shooter, sports, etc w/ this

 

any update bout this topic?

 

OUR PRAYERS HAVE BEEN ANSWERED

https://developer.apple.com/ibm/

Watson unity plugin

 

I managed to implement CoreML in Unity, the only problem I have right now is that you can't use it with Unity ARkit because both use an "AVCaptureSession" (this is an objective C / swift function). And I can't find a way to access the ARKit avcapturesession so I can add my CoreML output to it. I tried accessing it through the CaptureCameraController implementation from Unity but for some reason it won't link, Symbols not found for architecture (This is a very common error with a lot of different solutions, I tried them all, none seem to work)

 

For those of you who just want CoreML without ARKit, here is a tutorial I wrote: https://medium.com/@kevinhuyskens/implementing-coreml-in-unity-e91bcf80a3c5
It's just a quick draft and I will make it better once I have some free time, but for now it should help you get started.

UPDATE: The tutorial now includes a Unity package that allows you to use both Unity ARKit and CoreML. I suggest using a Mobilenet model for the smoothest experience (still not very smooth on my iPhone 6S tho, getting around 11fps when analyzing the pixelbuffer with CoreML). Resnet50 only gives 3fps on my iPhone 6S. So now I'm looking for a way to multithread the analyzing, never used multithreading before so it should be fun!

 

World needs someone like you. Thanks for the tutorial.

 

I suggest keeping an eye on it since I am still updating it whenever I get some spare time. I'm also planning on adding a github project. But I can't just upload my project I'm working on right now because it's for a company. Didn't sign any NDA since all I'm doing is research but I'm hoping to get a job there so it wouldn't be smart just sharing their projects I guess ... :')

Also I hope it's of any help to you and if you have any suggestions feel free to send them my way. I'm trying (still very far from it tho) to make a tutorial comprehensible for everyone. Always looking to improve it.

 

Hi! We released a plugin for using Core-ML in Unity: https://github.com/possiblecee/UnityVision-iOS

 

Hey,
I download this project But when I Build in Xcode it's giving an error.

 

Did you do this part:

For image classification, the plugin uses the InceptionV3 machine learning model. This model is provided in this repository inside the MLModel folder. Add this model to your Xcode project (generated by Unity after building), by dragging the model into the project navigator. Make sure that the model is added to the Unity-iPhone build target.

 

Hey,
Thanks for the reply. It's working now.
How do I create an Ml model or train model?