[RELEASED] VR Interaction Framework

@BeardedNinjaGames

Thank you for your asset

I migrate my app from quest to pico.
Interactions work but UIs don't work

I use a player from Pico Demo scene.
I add a Canvas with a button and add Pvr_UICanvas component.



The input bridge is set to Pico :



But nothing works. An idea ?

 

The UI with Pico / VR Interaction framework no working...

 

Hi there. It looks like you don't have an EventSystem in your scene. If you are using Pico's UI system then you'll want to follow their setup. VRIF also has a UI system you can use called VRUISystem. You can copy the components over from the demo scene (EventSystem and the UIPointers to display a LineRender from the hand) to your scene as a quick way to get started. Just make sure you don't mix EventSystems / only have one in the scene.

 

What if I need an InputField? Is it supported in any way in this asset?

It could also be useful to add a rubber to the other end of 3D marker.

 

Hello, I am trying to understand how to use a model of my own with the FinalIK integration within the VR Interaction Framework.

1. After installing the FinalIK integration within VRIF, I opened up the VRIK scene and added my own model.
2. I followed this tutorial in order to set up the VRIK component, where I created a IKLeftTarget, IKRightTarget, and IKEyeTarget and assigned them to the requisite transforms in the Solver section of the VRIK component. I also changed the target rotations and positions so they match where the controller is supposed to be.
   
3. However, when I run the scene, my model's arms become stretched and distorted, and I'm not sure why this is happening. What makes this especially strange is that when I follow the same tutorial in the FinalIK Oculus demo scene (VRIK (Oculus)), it works perfectly fine without distorting my model's arms. Therefore, this leads me to believe that there may be an extra step that I need to do to make it work with VRIF's setup?

Here's what the model looks like before I run the scene.



Here's what the model looks like as soon as I run the scene.



Any idea what might be going on and how to fix it? I am happy to send you the model if it helps!

Thanks!

 

Thank you for your reply


I use Pico UI system but two problems :

- The orientation of the ray from Pico API is not the same as yours API.
- At a short distance, no event

I will try with your UIPointer

 

It works with your UI Pointer. Do you want to send you the player by email?

 

Yesterday fine, today this:

Assets\BNG Framework\Scripts\Helpers\HandTracking.cs(23,9): error CS0246: The type or namespace name 'OVRBone' could not be found (are you missing a using directive or an assembly reference?)

plus heaps of other errors...
I have changed nothing, made no code changes, no importing or deleting anything into the project.

Whether it is Unity, Oculus, OVR or this asset, if I can't rely on things continuing to work as they are supposed to, from day to day then I can't use these products for anything, they are useless.

 

Ive tried this with playmaker and get button actions but they don't seem to work. Is there a special function or script to use to get buttons to be recognized in playmaker?

 

I understand your frustration but that mindset won't solve the problem. It's impossible that you changed nothing and it stopped working. Think real hard about what did change, because something did. Worst case you will have to start a new project with VRIF only and get that working and go from there.

A lot of people don't use third-party assets because of issues like this. Something breaks and you have so much code you didn't write it becomes impossible to figure out the issue. I treat third-party assets as learning tools, not shortcuts. I use VRIF to see how the great BNG does things I also want to do. I don't just throw VRIF in my game and cross my fingers that it'll do exactly what I need it to do.

Sorry for your troubles, I hope you get it figured out!

 

Fresh project imported VRIF, exported physics hand scene by itself. Physics hands trail behind hand position extremely sluggishly.

Toddkc, unity does change things every time you restart it, for example script execution order, unless it is explicitly set.

We don't know what goes on under the hood, unless you have access to the source code perhaps?

 

Sorry...but I don't believe your project changed like that on it's own. Good luck though, of course I hope you get it figured out!

 

"I don't believe your project changed like that on it's own" thank you person on the other side of the world accusing people of lying. Good luck yourself.

 

To the actual BNG, the physics hand scene is tracking touch controllers very sluggishly. Using 2019.4.21.f1

The scene start launches everything on the guns table into the air, the car driven by the joystick also disappears maybe launching into the air as well can't tell, the Tool tip is the wrong size. Bows are right handed only as far as I can tell.

The only thing I actually am worried about is the sluggish hand controller tracking, the rest is just a heads up for something that may need checking for the next release.

You also don't mention supporting CV1, maybe this is the reason for the sluggish tracking?

edit -
Just tested an older version of your asset, no physics hands lag when tracking Oculus touch controllers for cv1 in a very heavy scene, lots of imported assets, no optimizations, works great.

 

I recommend a fresh import everytime BNG does an update. Seems to mess up your current project in a hurry.

Please don't. Once this kind of banter starts it doesn't end.

I have noticed this as well since the first time in the latest update. I chose not to use them, but that's me. I think some of the problem is unity itself. If I am not mistaken, BNG's original code started in version 2017.I cannot remember, so I found that version 2019.3.15f1 seems more stable with his code. I know on several occasions BNG has referred others to a specific version of unity based on the current code base.

If you want more help BNG has a discord channel where he checks stuff regularly, might be easier to ask questions there, if you want.
https://discord.gg/WRzJn8Ty

 

You say "Hand Jet" above. Where do I find more on this?

 

Has anyone managed to get to work this BNG VR Framework with the touch pad on Oculus Quest2?
I have added the touch pads input to the default thumb stick inputs for movement and rotation on the PlayerController, but the touch pad inputs does not take any effect for me.

 

There are couple of them on the table with markers, floating cube and small orange doll sitting on the edge. Another pair of jets are on the roof of highest scyscrapper with golden spike on top. Also HandJet.cs is responsible for this flying behavior

Why do you think that Quest2 really have touchpad? That's from their official documentation: "Oculus Quest does not have a touchpad on the HMD. Your app should not refer to an HMD touchpad when running on Quest." ( https://developer.oculus.com/distribute/publish-uploading-mobile/ )

 

@BeardedNinjaGames
would you please advice how to add impulse to Player when releasing Climbables? I mean that you for example push your hand holding ledge to the right, forcing moving your body to the left, release the grip and then the player falls down at parabolic trajectory. Now player just falling straight down ignoring it's impulse. I want to make something like jumps in "Climb" but dont know how to approach this. Player even don't have rigidbody (although I saw rigidbody reference in PlayerGravity.cs).. Any ideas?

 

I just mean these circular matt areas about 1.5 cm diameter on both hand controllers. What are they for?

 

They are just to detect if thumb is resting on controller when it doesn't touch any button. It allows to correctly animate thumb of virtual hand (is it pressed to the fist or not). I'm not sure but I believe that it isn't even accessible through SDK. And definitely you cannot receive thumb's coordinates, it is binary thing, like buttons: it is either pressed or not.

 

well actually it accessable through SDK, look for "thumbrest" here: https://docs.unity3d.com/Packages/com.unity.xr.oculus@1.0/api/Unity.XR.Oculus.OculusUsages.html

 

Sorry reading the doc and I'm bit lost missing something simple i think,..

but how can you change the movement from teleport movement to just walk forward using the left trigger ?

 

I don't believe using the left trigger to move forward is a default option. In the demo scene you can change from teleport to movement with the left joystick by clicking in the left joystick. You would then walk forward by pressing forward on the joystick.

 

Goodnight. I need a video tutorial to configure the demo project that includes this software, using Htc Vive, in Unity 2019.4.22f1. I can move my hands with the controls and the camera with the helmet, but the buttons on the controls do not work (the fingers do not move and the grip does not work).

 

Hi. Why the best way to mix Curved UI and your framework with Pico headset ?

Thank you for your help

 

I have followed the video of integration with emerald step by step, and i have a problem,

the ai takes damage and dies when touching with anything in the hands (or throwing it)

 

Hey there - for the HTC Vive you'll need to make sure you have the SteamVR SDK installed, with the actions mapped out. If you're using the original Vive with the wands you can use OpenVR Desktop as well. Otherwise you'll need the SteamVR SDK, or OpenXR (though OpenXR is still in preview as of now, but VRIF supports it). Check out the Vive wiki page, and definitely come check out the Discord if you get stuck. There are many Vive users (I have access to one as well) that can help you.

There is a CurvedUI integration package included. You can copy over the pointer prefab (or you can make your own / use the "custom" one from CurvedUI) to your rig. Just make sure you don't have VRUISystem or any Unity or VRIF UI stuff in the scene, as CurvedUI doesn't play nicely with those.

There is a new EmeraldAI integration that passes damage through on the emerald ai wiki page - EmeraldAI just added limb / locational damage (like last week I believe) and you can use this script for that as well. Also, on the EmeraldAIDamageable script make sure to disable "DestroyOnDeath" to make sure the ai plays it's damage animation, and not just disappear instantly. Lastly, if your ai is dying too quickly, lower the damage on the weapons (RaycastWeapon, DamageCollider) or increase the health on your AI. Hope that helps!


On another note, I'm currently wrapping some updates that I have mentioned on the Discord, but wanted to mention a couple of standout features here as well - specifically hand posing, auto posing, and posing using hand tracking. Lots of goodies coming out next update, but here are a couple of quick videos to check out until I can make some proper update videos :

Automatic Hand Posing :

Saving Hand Poses using Quest Hand Tracking :

After that I'll be doing another documentation revamp, as well as some more installation (2020LTS, woot!) and tutorial videos.

 

[QUOTE="

There is a new EmeraldAI integration that passes damage through on the emerald ai wiki page - EmeraldAI just added limb / locational damage (like last week I believe) and you can use this script for that as well. Also, on the EmeraldAIDamageable script make sure to disable "DestroyOnDeath" to make sure the ai plays it's damage animation, and not just disappear instantly. Lastly, if your ai is dying too quickly, lower the damage on the weapons (RaycastWeapon, DamageCollider) or increase the health on your AI. Hope that helps!

[/QUOTE]


I have used exactly that video from the wiki. everything works fine, except for the error I mentioned. I attach a video in which the failure is seen. It would also be necessary to explain how to damage the AI to the player.

the ai dies without shooting it.
Im not a coder sorry.
i have another finished project using (third person cover system) with a full configurated Ai, envoirement....
can you help me to integrate this too?

 

I have already managed to confirm the control of HTC vive. now I need to know how to use the keyboard and mouse, while still using the controls. I need input with all three devices (keyboard, mouse and HTC vive controllers). Thank you.

 

Hey,

We are instantiating the Rig at runtime, just after the level has finished loading. We get the following error:

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object
2. BNG.SmoothLocomotion.UpdateInputs () (at Assets/BNG Framework/Scripts/Core/SmoothLocomotion.cs:93)
3. BNG.VREmulator.CheckPlayerControls () (at Assets/BNG Framework/Scripts/Extras/VREmulator.cs:160)
4. BNG.VREmulator.Update () (at Assets/BNG Framework/Scripts/Extras/VREmulator.cs:92)

Any idea what might cause this?

Code (CSharp):

1.             // Start with VR Controller Input
2.             Vector2 primaryAxis = GetAxisInput();
3.             if (playerController.IsGrounded()  ) {
4.                 movementX = primaryAxis.x;
5.                 movementZ = primaryAxis.y;
6.             }

Seems to be in this part of the code, but it is not clear to me why it would give the null reference.

 

You can use the new Unity Input System - set up actions (or there are some already setup for smooth locomotion, snap turning, etc.). Then you can edit the action to support different types of devices. So for example, on an Oculus you can set it up to use thumbstick for snap turning, but on keyboard use Q, E. There is also the VREmulator component you can take a look at if you're looking to mimic a vr player by keeping a certain play hear height and things like that.

I would just remove that VREmulator component from your instantiated player, as you don't need that functionality. It's looking to access the playerController, but that component may not have been located / instantiated yet. I'll look into adding a null check for that component for cases like this.

 

Thanks. Can I also say that it is a wonderful asset you've built? I've been working with it for a couple of months now and the amount of work it has saved me is just stupendous.

 

Hello,

I am having an issue with an overlay UI (for mouse/keyboard). I can only click the first button. My mouse position isn't read, neither my keyboard inputs.

Test it : Just add a canvas into XRDemo scene with buttons and InputField in it.

What should I do to have both inputs from mouse (overlay canvas not visible from HMD) and VRInput (world canvas for controllers only) ?

PS : I'm using new InputSystem

 

Could someone help me out, this is the VRIF and Emerald AI scene file that I cant get to work. All you need id VRIF and Emerald to get this to work.

https://www.icloud.com/iclouddrive/0N5oIXl45EbtKsi7hOkjg5crw#Test_Scene

 

Join the discord, you'll find more active help there. I don't have emerald ai so I can't help, sorry.

 

We talked a bit on Discord, but just wanted to reply here for posterity. The current VRUISystem only supports world canvas, but I'll look into adding 2d support as well. In the meantime you can use CurvedUI (there's an included integration) or the new Unity Input System. For either system just make sure not to have VRUISystem or any other EventSystem in the scene, as they don't play nicely with each other.

There were some updates to EmeraldAI and how VRIF handles damage - you can grab the updated EmeraldAI integration script here from the wiki. EmeraldAI also added limb damage in their last update, so you may want to add that separately as well (there is also a limb damage script posted on that wiki page, and it's been pinned on Discord as well in #code-sharing fwiw). Those two should get you going.

 

Hi,

We are a VR development studio, and we were using Humanoid Control VR asset, but we had some issues with this package, and we are looking for other alternatives. One of our customer is looking for a VR multi-avatar networked app, with voice interaction. Can you tell me if you guys have tested this kind of setup, and how many avatars can be controlled in a scene with your asset ? We are thinking to develop certain kinds of optimization techniques like instancing, burst to speed up character skinning for the avatars. Do you have some recommendations to make about it ? And what kind of dedicated server setup should be used for Photon Pun ? Thanks for your help.


Eric

 

Hi there (dev here). I've used this in a couple of small business-related projects that were exclusively networked applications. We used Photon PUN with the Voice addon (there's a Photon Integration included with VRIF, but if you're proficient with networked applications it's not hard to adapt it to any other library that uses an ownership model). It is pretty quick to get up and running and the voice / sound quality was adequate. We only needed around 5 or so people in a room at once and haven't stress tested anything higher than that. I did notice that having voice chat going also consumes a fair amount of cpu (I believe this would be true with any networking library). There's also Mirror, UNet, and Normcore that are worth taking a look at (and I'm sure some others I'm not aware of).

 

Hi everyone Version 1.7 is ready for testing and will be submitted to the asset store next week. If you want to test this out before then there is a link posted in #Releases channel on the Discord.

Here is the changelog :

1. Added new Hand Poser tool with 40+ included poses. This hand poser tool can be used standalone outside of VRIF and works on any type of object (can be hands, a Final IK character, spider, etc). You can use this alongside an animator to get the best of both worlds. Makes creating custom hand poses so much easier.
2. Added Quest Hand Tracking functionality to Hand Poser tool - Make poses uses your Quest's Hand Tracking capabilities and save them for later use. Maps the Oculus Quest hand tracking model to a custom model for easy previewing in the editor.
3. Added Auto Poser feature to make hand posing easier - just click the button in the editor to auto pose or set to "Continuous Auto Pose" while you move the object around in real time.
4. Reworked Default Hand Positions - Previously the default hand pose was higher up in the air and aligned straight with the controller direction. This was causing some issues validating Oculus apps where the hands must match 1:1. I've updated the default pose to be exact with the controller positioning, but in the process I decided to move the grabber slightly down as well. That means your custom grab points may need to be updated to match this new alignment. More info on this coming soon.
5. Added new "Flick" type remote grab, similar to Half-Life:Alyx grab mechanic. It's fun, try it!
6. Added new bezier remote grab line - draws a nice rounded line as an alternative to the ring helpers.
7. Added an experimental "PunctureCollider" - stab things! Just an early concept but fun to play around with. The table in the back of the xr demo has a knife sticking into a ball. Will expand upon this in future updates.
8. Better handling of grab point positioning and live previewing - figuring out where your hand or object is going to snap should be much more intuitive
9. Added option to Grabbable to allow all child colliders to be Grabbable
   
10. Updated EmeraldAI script / integration
11. Updated Photon PUN example to include new "OnOwnershipTransferFailed" callback
    
12. Fixed issue where Robot Kyle was way up in the air on some devices
13. New option for raycast weapons to use internal ammo instead of bullet components
    
14. Fixed Issue with Steering Wheel return to center functionality
    
15. Added new menu item shortcut to create Grab Points
    
16. Added option to use Raycast or Spherecast for RemoteGrabber
17. UIPointers no longer send events if gameobject is inactive
    
18. Added Pistol preloaded clip example
    
19. Lots of misc bug fixes and code clean up

 

Hi, maybe a dumb question, but does VR Interaction Framework support XR Interaction Toolkit / openXR Plugin and the new input System?

 

Hey there. Not a dumb question at all. VRIF does indeed support OpenXR as well as the new Input System (there are default actions / bindings included, and many of the components (like snap turning or smooth locomotion) can take an Input Action as a parameter. As for XR Interaction Toolkit, you probably wouldn't want to use both interaction systems at the same time, but you may be able to use some features or concepts (like UI and Input) from it. Once XR Interaction Toolkit comes out of preview I'll look into potentially integrating it, if it makes sense to.

 

Thank you very much
that was my final decision on buying your Asset
As for XR Interaction Toolkit it makes the integration of the different VR Headsets a lot easier. my current target is Oculus PC VR (Rift S & Quest through link) but it would be great to integrate the other platforms in the future without having to redo all my VR Integration work.

 

Newbie question here -- can I use VRIF functions on meshes I create ? I have a scene in Unity with fbx meshes I've imported from Maya, and if I can hook up buttons and light switches, levers, etc with VRIF I will be light years ahead (compared to writing C# scripts for everything). Thanks in advance!

 

Hi BNG, and all, just dipped back into VRIF , the newest update and totlly loving it as before, but one wee problem I had at any rate just as headsup, the wee car (vehicle demo) seems really hard to stand on, and goes a bit crazy, dunno if this is the kegendarilly unstable unity wheel collider, the small RC versionis great, but it would be great to have access to a car or even flying vehicle (other than handjet) that works with VRIF as it has so much nice content, I have purchased other standalone vr car asset for oculus and it is really bad in my opinion, the only nice feeling ones (that I dont know how to integrate with VRIF) were from a xouple of really good VR-Youtubers on Patreon, another aspect, after getting on the car even for a short while (bucking bronco stylee), the player seems to be offset at a bad ange, its like being really drunk . could be a fun mini game if it was intentional!, anyway thought I would share, sorry I know a devs work is never done, but kudos for the asset !!

 

Problems after upgrade (clean install)
1- The integration with emerald AI is broken. (adding new and old script, selecting layers in the raicast pistol option ....) the Ai dont die.
2-Melee wapons dont hurt AI too.
3-And a request, how do I get the backpack back on my shoulder after releasing it, and not float.
Is there any way to only be able to extract clips from the belt that I have already collected before not infinte .. (if i drag 10 clips to the belt, i have 10 clips to use i mean).
Thank you.

 

First off, thank you for creating this asset. I was able to get it up and running on my Quest pretty quickly. I have been messing with all the features and they are great. The main feature that sold me was the "Flick" remote grab type. Now I have a question about that. It doesn't seem as responsive as I was hoping. I was hoping to see that the whole object would glow or something instead of just the circle. I was also hoping about some of indictor like the teleport move indictor showing where that object will travel if that object if farther away.

Is there any plans to add more things to the "Flick" remote grab?

 

Hello. Thank you for a terrific asset. It's saved me a ton of time setting up my game.

Newbie question here: I'm building a racing sim where the player is always seated. Can you recommend a way for me to set that up with VRIF?

 

And another: I'm using the joystick prefab, substituting my own graphical asset. It's attached to a spacecraft which rotates, and I've noticed that the joystick is behaving badly as the spacecraft rotates, suggesting that (maybe) the configurable joint or something is set up for world coordinates, even though that checkbox on the CJ is cleared. Any hints?

FYI: the spacecraft is moved with physics calls, so that’s not the issue.

edit: do I need to use another joint (fixed joint?) to connect the root joystick rigidbody to its containing spacecraft (also a rigidbody).

 

Hey there. You can definitely use your own meshes, though instead of applying the VRIF components to your meshes, I'd recommend starting with one of the VRIF prefabs that are already set up. Most prefabs have a "GFX" (short for 'graphics") transform you can just disable and put your mesh in. Then you can just resize the colliders to match your mesh. That way you are starting from something that is working, you can just tweak things to your liking, and then save as your own prefab variant.

The CharacterController gets a little whack on the vehicle, but there are ways around it. The easiest way to get started with a vehicle would be to use the "Barebones" player prefab and just parent it right to the vehicle. I'll include a sample scene for a vehicle in the next update, with the disclaimer that vehicles in vr can be sickness inducting in VR. I could also just post something on the wiki or Discord.

Hello. I'm not quite sure what's going on with the EmeraldAI integration - a couple of people have mentioned issues with it, but others have it working fine. EmeraldAI recently updated their damage system to include limb support as well, so I'll take another look at this integration and see what I can find out.

For the backpack - there is a component called "ReturnToSnapZone" that will return an item back to a snap zone once it's been dropped. Should work great for moving it to your back again. You may need to set the backpack to non-kinematic.

For your clips idea (I like that idea), you could use the OnSnapZoneEnter event to move the clip or just enable / disable clips that already exist around you.

Hey there and welcome If you take a look at the two balls in the back of the demo scene, they have a "bezier line" indicator on them which is what I believe you are describing :


I definitely plan on extending this functionality - this is just a first pass, but I think it's quite fun to play around with. Let me know if you have any particular feature requests for that.

Hello and welcome For a seated racing game I'd probably recommend starting with the Barebones player prefab and parenting that directly to your vehicle. There is a property on the BNGPlayerController called "Player Y Offset" - you can use that to offset the player's height. So let's say the player is 1.5 meters tall sitting or standing (super tall chair but your optimal height above the steering wheel is only 1 meter. You can then use that PlayerYOffset variable to make the player at the desired eye level (in this case setting it to -0.5 because you want the player to be 0.5 meters lower). A good way to figure out the player's height is to have an intro screen where the user needs to click a button to proceed (and maybe ask them to sit / stand straight). When they click the button you then store their height based on current hmd position from the ground.

Having joints on objects that move around at high speeds like a spaceship can get.. complicated. There are a few ways to make it work (have the object located outside of rigidbody and attach with a fixed joint, apply same forces to the stick, have the object follow along separately, etc), but you probably don't actually need the joystick to be physically activated, so I would just disable that. On the "JoystickControl" script set "Kinematic while Inactive" to true and you should no longer have to worry about your joystick freaking out while you're trying to fly You may want to tweek the Rigidbody settings and Smooth Look settings as well.