VR wizard fighting game - Gesture Recognition Tech Comparison

Hi there,

The other day I was thinking of making a wizard fighting game, like Harry Potter vs. Voldemort. Then I figured out I would need to detect controller movement while players are casting spells with VR controllers. So I searched on Unity Asset Store and found 3 VR gesture libraries, I tested their demo scenes. Here’s my two cents about these 3 VR gesture libraries.

First of all, these VR gesture libraries are :

• VR Infinite Gesture by Edwon Studio (Infinite hereafter)
• VR Magic Gestures AI by Raving Bots (Magic hereafter)
• VR Gesture and Signature (for HTC Vive) by AirSig (AirSig hereafter)

Testing result summary (5 is the best; 1 is the worst:
Bold and underline means the highest score in the item




To be clear, all the comparison and analysis are based on the testing of demo scenes of these gesture libraries. I didn’t have the chance to try the SDKs.

I tested 3 demo scenes on Vive. I compared them in different aspects like “feature”, “accuracy”, “price”, “developer-friendly” and “ratings”. Let me explain what those aspects mean.

• “Feature” : functions and capabilities the SDK provides. Since AirSig library has the most complete features, I was taking its features as the comparison basis later.
  
  
• “Accuracy” : Obviously it’s an important factor taking into account when you want to “recognize” player’s gestures. Bad accuracy causes lame user experiences and players will abandon it in no time.
  
  
• “Developer-friendly” : Is it easy for developers to develop? Easy to learn?
  
  
• “Price” : Well, that’s how much you gonna pay for it.
  
  
• “Ratings” : Ratings from Unity developers.
  
  
• "Compatibility” : VR platforms supporting capability
  

Feature:

AirSig has the broadest features. Infinite only has one feature. Features are explained as below.

1. Ready-to-use gesture : SDK built-in gestures for developer use. AirSig’s ready-to-use gestures include “c” letter,”heart” shape”,”^” shape. Magic’s ready-to-use gestures include “|” shape, “-” shape, “circle” shape, “m” letter. Infinite has no ready-to-use gestures.
   
   
2. Developer-created gesture: If the ready-to-use gestures are not enough for developers. Developers can also create their own gestures and build them into their VR content. AirSig is the only one provides this feature which is very useful for developers. Pity it’s not in the demo scene, so I haven’t try it.
   
   
3. Player-created gesture: Players of your VR content can create their own gestures. For example, a player creates a “star” gesture to cast his special spell in wizard game.
   
   
4. Player-created signature: This is a special feature for authenticating players in VR. For example, if a player wants to buy some virtual gears online during a shooting game, he can “sign” his name in the air with the VR controller to authenticate himself. Again, AirSig is the only one owns this practical feature. I recorded “Jack” during the test. And showed how I signed to my friend, then asked him to copy my signing to hack it.
   
   
5. Stroke-dependency : Gesture strokes matter or not. For example, if I record a circle shape in clockwise stroke order, will my counterclockwise circle gesture be recognized as a circle as well?
   

Accuracy:

The following table is testing result of accuracy.

AirSig got the highest score in accuracy and leaves its competitors far behind.

I drew each gesture 20 times to calculate my pass rate. For false match test, I intentionally drew wrong gesture(gestures other than ready-to-use, I created. "Triangle” in my case) to see it is falsely matched.

For player-created gesture, Magic and AirSig require players to record their gestures 5 times. Infinite does not limit the recording times. To be on the same basis of comparison, I recorded my gestures 5 times for all 3 libraries. However, more recorded gestures may improve the accuracy.

One weird thing is AirSig’s false match of read-to-use gesture seems way too high, judging from its other accuracy performance, I guess maybe the selection philosophy behind it is choosing the most similar one when not matched.

Stroke-dependency is actually not directly related to accuracy. But it does affect accuracy in some cases, some draw circle clockwise, others do it counterclockwise. From the testing result, Magic is the only one that does not care about the gesture stroke sequence. Although there is still zoom for its accuracy.





Ratings:

• Infinite : 4 stars under 27 reviews
• Magic : Not enough reviews
• AirSig : 5 stars under 24 reviews

Price:

• Infinite : $89.99
• Magic : $79.00
• AirSig : Free. However, I found its Daydream version prices at $9.99. So not sure for how long will AirSig keep it free.

Developer-friendly:

All the testing proceeded without reading any of their document first.

• Magic: Magic designs its demo scene’s UI simple, easy and fun. Nearly no learning curve for me. You can easily know which ready-to-use gestures it has. And the matching signal is very clear. Thumbs up! And its stroke-independent feature is also helpful for covering more players’ writing/drawing habits.
  
  
• AirSig: AirSig demo scene’s UI is kind-of plain and sometimes confusing. For example, the function title of ready-to-use gesture is “Developer-defined”, that confused me a while. And I didn’t know which ready-to-use gestures it has. The most confusing thing is the unclear matching signal. After drawing a gesture, it shows a score like 1.xx, which I didn’t know if I passed or not whatsoever.
  
  
• Infinite : Infinite designs the demo scene so complex and “geek”. It even uses “neural network” in its UI. During the test, I found I couldn’t delete my recorded gesture. Among all 3 demo scenes, it took me the most time to understand how to use. But Infinite has a function the other two don’t have, you can “edit” your gesture after you recorded it. This function gives you a chance to remove bad quality recorded strokesng recording.

Compatibility:

• Infinite : Supports HTC Vive + Vive Controllers + SteamVR
  Oculus Rift + Touch Controllers
• Magic : Supports HTC Vive
• AirSig : Supports HTC Vive(free), Google Daydream($9.99)

Leave your comment if you have used these gesture libraries or others.

 

I tried AirSig for HTC Vive, despite of a few rough edges in the demo scence, like @leeyingchao mentioned in the "VR wizard fighting game - Gesture Recognition Tech Comparison", its gesture recognition is really accurate.

Karen

 

I also tested all the three mentioned in this article. I echo that AirSig has more functions and better performance. Given that Infinite and Magic charges 89.99 and 79 respectively, Free AirSig seems to be an excellent bargain.

 

Hi guys, could someone make it work using developer defined gestures? When i save a sample of a gesture and then try to upload it, it trows me an error: "Upload failed. Invalid token or invalid gesture data"
Can someone help me please?

 

I haven't been able to create a gesture using airsig...

edit: I figured it out, it is working, just not the most intuitive UX thus far

 

Out of the box the demo scenes crash after finishing the 5/5 gestures, using 2017.3, anyone else?

 

I guess this has been abandoned.

 

Hi, also have a crash, looks like cant found db and at first demo write about it, but files copied from Assets\AirSig\StreamingAssets to Assets\StreamingAssets

 

Hi emaLorenzo,
Your problem may come from the language setting of Windows.
Some kind of language setting will change the internal format we have used.
You can set your windows to English, then the collector should be work correctly.

 

Hello beowulfkaine,
We could not figure out what happens in your environment. It will be helpful if you mail your screenshot to service@airsig.com

 

Hi,
Thanks for your report. We have fixed this bug and submitted a new version to unity asset store.

 

I have a crash after I enter demo scene called developerDefined, as soon as I perform one random move, the game crashes and it goes back to SteamVR home.

 

Hi,
I am not sure which platform you used.
But this issue usually happens when you miss the installation of .Net Framework or Microsoft Visual C++ 2017 Redistributable as the document ask
https://dev.airsig.com/docs/quickstart_vive

 

Hi, I was having problems with the demo scenes. It says couldn't find db files but I've already copy the streamingasset folder under the assets folder. I tried with both 1.28 oculus sdk and 1.32 oculus sdk but the same problem happens.

 

Does anyone know if AirSig can support 2 handed gestures?

 

I've tested a bunch of them and I find the best one is one you haven't mentioned, MiVRy:

https://assetstore.unity.com/packages/templates/systems/mivry-3d-gesture-recognition-143176

 

Free for non-commercial use but we have to contact the author to discuss the license cost for commercial use. That means we have no idea how reasonable the licensing fee is.

 

VRTK is free and has a basic library of controller movements for interactivity that may get people started.