AVPro Live Camera

I believe we would have to write a camera-specific plugin to access those. I'm not sure, it's been a quite since we worked with machine-vision cameras..

 

Hi All, sorry about the cross-post, but I just wanted to announce a new RenderHeads plugin for Unity:
External Game View

It's a plugin for multi-monitor productivity and workflow improvement while using the Unity Editor and it's really changed the way we work. You can find out all about it here where you'll find documentation and a tutorial video.

Thanks,

 

I'm struggling with device activity and managing exceptions. I need to check if the camera is working properly to restart it or else. Basically:

Code (CSharp):

1. if (avplCamera.Device == null)
2. {
3.     avplCamera.SelectDeviceAndMode ();
4.     RestartDevice ();
5. }
6. else if (!AVProLiveCameraPlugin.IsDeviceConnected (avplCamera.Device.DeviceIndex))
7. {
8.     RestartDevice ();
9. }
10.  

But this is only two cases where the device is failing. I tried:
avplCamera.Device.isRunning <- can stay true if the camera has not been stopped correctly (even if camera is disconnected)
avplCamera.Device.isActive <- Same
avplCamera.Device.isConnected <- Same (which is clearly wrong)

Sometimes, when my camera stops, all these booleans remain unchanged.
Is this a bug or am I missing the way to manage this properly.
This is gonna be installed for a year in an exhibition, I need to be able to detect anomalies et failures.
Thanks.

 

Hi @boubouk

The way to monitor a device's connection in Live Camera is to use Hot Swapping. This can be enabled by checking the 'support Hot Swapping' box in the LiveCamera component.

Let us know if this helps?

 

It's already checked. But Update Hot Swap in Live Camera component was unchecked. By enabling it, camera is restarting after reconnection.
Will investigate further.
Thanks for the help.

 

@TerryRH I was using this asset for a while on my Windows-Only project, but since I had to port my project to MacOS, I was't able to use it and had to switch to NatDevice.
Unfortunately NatDevice is now locking existing customers from updating in favor of a subscription model and that's not something I am able to use right now.

So I apologize if it was asked a million times, but is there any chance of this plugin adding MacOS support?

Thanks!

 

Hi, thanks for your message. In short, there are plans for MacOS support, but they are very much long term plans, so we can't help you at the moment unfortunately.

 

Ah, okay, thank you for the clarification! I look forward to MacOS support whenever it does become available!

 

I just want to ask a follow up question:
I'm plan on using this for the Windows version of my app and will be using WebCamTexture for a "dumbed-down" implementation for MacOS.
To do this, I will use Scripting defines, but will there be any issue building for Mac with the plugin installed?
Is there a better way of writing the dual support than Scripting Defines?

Thanks!

Edit: I'm trying to access the outputTexture after calling Begin() on a Live Camera, but it's always being null for some reason. The LiveCamera inspector is working as expected.
Do I need to access the outputTexture in update? My intention is to grab it once, assign it to a material/UI element and change it when I change the camera. Is this not possible?

I got it working by commenting out part of this line in the CameraDevice class.
After doing some debugging,

 _formatTexture.ValidPicture

is always null despite the camera working fine.

Code (CSharp):

1.      
2. public Texture OutputTexture
3.         {
4.             get { if (_formatConverter != null/* && _formatConverter.ValidPicture*/) return _formatConverter.OutputTexture; return null; }
5.         }

Thanks again!

Edit 2:
To follow up about the Mac/PC issue: I figured I would add the manager and live camera components at runtime if the platform is windows, then move from there. This works in the editor, but not in builds -- it requires the components to be in the scene from the beginning.
But, if the components are in the scene, I get errors on mac.
My solution was to have them in the scene on an inactive gameObject, which gets enabled if the platform is Windows.

 

whats the api to set and update camera settings in script? Like an exposure setting maybe.

 

I'm sorry, we're really not sure about this one, so can't help you further. But we'd love to hear about your solution if you get it working.

 

Hi @mchangxe
We don't actually have an API reference list, but the scripting can easily be understood by examining the script in the CameraExplorerDemo scene.

 

I just used scripting defines and disabled the components if the platform is mac.

But the bigger problem is why _formatTexture.ValidPicture is always false. Any idea why this would be?
I'm using basically the same code as in the Camera Explorer Demo just adapted to use UnityUGUI and creating my own wrapper classes for making it simpler.
Commenting out part of that line makes it work, but obviously that's not a real solution.
I'm using a regular crappy USB webcam, then Canon EOS Utility with an M50mII.
Thanks!

 

No, idea why its not working, sorry.
LiveCamera uses Microsoft DirectShow API, is that what is blocking it?

 

Hi,

I'm wondering if this issue was ever resolved?

Thanks,
Dave

 

Hi Dave,

I'm very sorry that this hasn't been resolved for you. The dev who had been working on that has taken a break to pursue studies. I can try and contact him to see if he has any notes from when he was last working on it We're a very small team, so unfortunately I'm not sure when I can get someone to delve back into it, but I will try.

Kind regards,
Chris

 

Hi Chris,

Ok yes, please do try that. It was two years ago. I bought the asset back then thinking it would be fixed.

I'm getting closer to release now and am hoping to get this issue resolved.

Thank you,
Dave

 

Hi @ToxMox,

I'm trying to look into this issue - apologies if I repeat anything but would you be able to post your Unity console log from running the CameraExplorerDemo to clicking on the already highlighted (240) mode in 'Modes' please?

Cheers,

 

@RichRH Sure, here you go. Thanks

 

Hi @ToxMox,

OK, I think I've identified the issue - your capture card doesn't expose the 'IAMVideoControl' interface which Live Camera uses to enumerate framerates, so instead will just see the minimum and maximum supported (hence you seeing only 24hz and 240hz options).

As an intermediate solution I'm going to look at exposing a few sensible rates within the provided range, including the camera default (in your case 60hz). These changes will go into the next release which should be early September.

Cheers,

 

Excellent! Thank you so much

 

Hi All,

AVPro Live Camera version 2.9.2 has just been released!

You can find the updated version on the Asset Store and the free trial version on our website.

Changes include:

Improvements

• Added some default frame rates between provided min/max for devices that don’t support ‘IAMVideoControl’ interface

Changes

• Tested against Unity 2021/2022 (demos use obsolete, but still functioning, code)

Thanks to everyone that reported bugs that were fixed in this release

Please report any issues here on the forum, or preferably to our Github Issue tracker

Thanks,

 

@ToxMox A bit later than planned but hopefully the above release fixes your issue.

Cheers,

 

Hi, I'm attempting to run any of the demos that come with this Asset, and I get a crash every time. It seems to be crashing while initializing the plugin. I've tested this in both 2021.3.10f1 and 2020.3.40f1 with the same results. Any help would be appreciated.

Here is a piece of my stack trace if it helps

=================================================================
Native Crash Reporting
=================================================================
Got a UNKNOWN while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

=================================================================
Managed Stacktrace:
=================================================================
at <unknown> <0xffffffff>
at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraPlugin:Init <0x00098>
at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraManager:Init <0x00022>
at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraManager:Start <0x0005a>
at System.Object:runtime_invoke_void__this__ <0x00087>
=================================================================

 

Quick update on mine. I found the issue. I had NDI Tools installed which includes a virtual Webcam for NDI streams. I removed that and it works now! (Although, it would be awesome if this supported NDI Virtual Webcams!)

 

Hi @dfashbaugh
Yes, we are looking into resolving that issue with NDI Tools.
Kind regards,
Chris

 

Hi All,

AVPro Live Camera version 2.9.3 has just been released!

You can find the updated version on the Asset Store and the free trial version on our website.

Changes include:

Bug fixes

• Fixed crash when NDI devices are present

Thanks to everyone that reported bugs that were fixed in this release

Please report any issues here on the forum, or preferably to our Github Issue tracker

Thanks,

 

I've noticed in 2022.2b13, the device output texture seems to always be null with an Elgato Facecam. This was not the case before. The texture IS available if gone through the Plugin side with `AVProLiveCameraPlugin.GetFrameAsColor32` but this is cumbersome and the amount of time investment is not ideal. I haven't yet tested with other hardware, but I would suspect similar results.


Some notes, this is on the latest AVPro Live Camera, HDRP Sample scene, with the RayTracing quality preset chosen.
Printing the name and then the OutputTexture:

Code (CSharp):

1. Debug.Log(Camera.Device.Name);
2. Debug.Log(Camera.OutputTexture);

Even after getting everything to work when using the direct plugin method way the manager indicates that the framerate is at 0 but the texture IS updating at 60fps...

 

Hi @Soaryn,

Thanks for the heads up. I take it that you didn't have this issue in previous versions? We don't do support for beta versions, so hopefully the issue will get fixed before release. Its certainly something that we can look out for. Have you reported it to Unity?

Kind regards,
Chris

 

Haven't had this issue before, no. I don't have much choice over use of Beta however, since the Latest ECS is limited to 2022 and that takes priority over camera sources.

As for reporting it to Unity, I'm not sure how this would be a Unity related issue (so no I haven't), as this is specifically the Device.OutputTexture that is Null. If you Blit a null texture to a render texture, you get rather odd results, usually what ever is on the active scene camera. For now, I have a work around, but the results taking 8ms of cpu time to do which is non-ideal. Do note, that Unity's HDRP likely underwent rather large changes in 2022 if I recall correctly.

Also, related to the workaround:

Code (CSharp):

1. var result = AVProLiveCameraPlugin.GetFrameAsColor32(_device.DeviceIndex, (IntPtr)_frame.GetUnsafePtr(), _width, _height);

This takes roughly 6ms on its own to call.

 

Hi Team. For debug image logging I am working on I need to grab the image data cpu side of the lastFrameUpdated.
Currently I am using the main thread to update the GPU side images and use formatConverter -> lastFrameUpdated to get the frame number. I then have a rolling buffer in a second thread grabbing every new frame using GetFrameAsColor32 and store the correct frame by frame number. If i don't have the rolling buffer i end up grabbing the frame too late. This is fairly performance and memory wastey. Can you implement a function the same as GetFrameAsColor32 that returns the last updated frame? alternately it looks like you have a rolling buffer internally considering the command GetFrameFromBufferAtTime maybe a GetFrameFromBufferByNumber? Thanks

 

Hi,
we bought the plugin already and are wondering if the audio support is in your backlog for future releases. If not, is it possible to use the DeckLink plugin for capturing audio and video from other cards as well (elgato, ...)
Thanks.

 

Hi @Chri5topheru5,

Audio support is a possibility for a future release, but we don't have a timeline for that I'm afraid.
AVPro Decklink will only work with Decklink cards.
We have another product called AVPro Movie Capture, which can do video and audio capture. Free, fully featured, watermarked trial version here: https://github.com/RenderHeads/UnityPlugin-AVProMovieCapture/releases

Kind regards,
Chris

 

Where can I find example C# codes for switching the camera resoluton of a webcam when a key is pressed?
Do I have to use "hot swapping"?
My senario is like this :
1. In Start() function
1.1 Prepare RenderTexture rt_1280_720 for the resolutoin 1280 x 720
1.2 Prepare RenderTexture rt_1920_1280 for the resolutoin 1920 x 1080
1.3 Start webcam with 1280 x 720 resolution
2. In Update() funtion
2.1. If "R" key is pressed
2.1.2 If current resolution is 1280 x 720
2.1.2.1 Change the resolution to 1920 x 1080
2.1.3 else
2.1.3.1 Change the resolution to 1280 x 720
2.2. Copy OutputTexture of AVProLiveCamera to rt_1280_720 or rt_1920_1080 accordingly.
2.3. Display rt_1280_720 or rt_1920_1080 accordingly.

I mean for a single camera but multiple resolutions. Not between multiple cameras.

 

Hello. Having some problems with the Elgato Facecam Pro:

1. The 3840x2160 [MJPG] mode is stuck at 30hz even when I specify 60hz
2. The 3840x2160 [Unknown] mode just doesn't work. I believe these "unknown" modes are h264 based on the attached DirectShow Capture Capabilities output. The 1920x1080 [Unknown] mode works

 

Hi @kyuhyoung,

Hot Swapping is basically just for changing in and out different physical cameras, so it will repeatedly monitor the device connection.
You should find some information on scripting in the PDF documentation that comes with the asset. We provide components that do the job, but generally leave custom scripting as an exercise for the developer.

Kind regards,
Chris

 

Hi @ToxMox,

Out of interest, do you get the same result using Unity webcam texture?
Have you tested it as a build?
Have you made sure that you are in a brightly lit room?

Some easy things you can do that might help with FPS:

• You should check the settings of the camera to make sure it isn’t doing unnecessary processing which reduces the frame rate. Disable options like “low light compensation”, disable auto exposure and reduce exposure settings. You can access camera settings via the CameraExplorer demo (click “Configure”), or usually via the camera setup application on the Start Menu.
  
  
• Another possibility is that the data bus is overloaded. You may need to plug devices into different ports so they are on a different USB bus.
  
  
• You could also check that the checkboxes for Hot Swapping support and Settings Update are not checked in the AVProLiveCameraManager and AVProLiveCamera components. On some device drivers polling for device connection status and checking the settings can really slow things down and could be done manually instead.

This is a pretty new camera. We might see if we can get one for testing in-house. We will most likely have to start supporting Media Foundation with newer webcams like this.

Kind regards,
Chris

 

Hi @Chris-RH,

Thanks for getting back to me. Please see my responses in blue below (you'll need to expand the quoted text to see them all):

 

I am trying to use GetFrameAsColor32 to grab a frame from my webcam. My webcam seems to have 2 format types. One that identifies as YUV_422_YUY2, and one that identifies as RAW_BGRA32. Any resolution / frame rate I pick that is the YUV mode works perfectly ... Any resolution / frame rate I pick that is RAW mode does not work at all (pretty much comes through as all zero values). Just wondering if you have any advice to get it working as the resolution and frame rate I want to use only pops up in the RAW mode.

Thanks for any help!!!

 

Also .... I wonder what GetFrameFromBufferAtTime does and if it could help me? I am actually trying to use LiveCamera to do a sports training "instant replay" sort of project. Like when I hit space, go back about 5 seconds in time and show a slo-mo replay of what the camera saw starting 5 secs previous. What I was trying was just storing the last 5 secs in Color32 arrays and manually playing them back. Seems to work OK as long as mode is the YUV (again RAW mode will not grab as Color32 for me) ... But if there are already frames being stored at times maybe I could tap into that instread???

Sorry for all the questions but I cannot find too much documentation of these functions. Thanks!!

 

Hi @ImNotEvilsizer,
OK, that's not really a common use-case. Our plugin's goal is to get video from webcam-type devices into Unity textures as quickly as possible, so we use the texture format that is closest to the webcam device's native format.

Saying that, I'm surprised that the RAW format is not working, as we've certainly had a few users who have successfully used it. What camera are you using? Are you getting any error messages?
Have you tried to create a RenderTexture in the format that you require then you can call Graphics.Blit(sourceTexture, destTexture); to convert between formats.

 

Thanks for the quick reply. I have a TECURS 1080P 60FPS webcam I got off Amazon for $20. Hahaha ... It was the cheapest webcam I could find. Kind of embarrassing, but it actually provides a pretty clear, solid image and I wanted to prove my concept before I bought any expensive cameras.

I don't get any error messages. The Color32 array is just usually all 0's or sometimes a little garbage mixed in. But only in the RAW modes. Again, the YUV modes work as expected.

Using your idea of Graphics.Blit I decided to just save each frame using Graphics.CopyTexture() into an array of Texture2Ds that I can replay using a RawImage UI component. If I need access to the pixel data I will just use GetPixels32 or whatever from the Texture2Ds. Just tested it and it is running nicely ... WAY faster than the WebCamTexture approach I started with.

So I have a very good work around.

I plan to have 3 cameras total in the final package. While I have your attention, do you think getting a Quad Black Magic card for the 3 camera inputs would make things run even better? I see your plugin supports it. I have NO experience with those cards and would hate to buy one and have little to no performance boost from it.

Thanks for your help. Always nice when a developer supports his product.

 

Hi @ImNotEvilsizer

Heh, we don't tech-shame here I think mine cost about the same It would be very shortsighted to only support high-end hardware.

I would suggest that the next step is to add your extra camera(s) and see how it performs. The Blackmagic Quad is unlikely to bring much to Live Capture (also, it requires HDMI input). However, if it was something that you wanted to use, then you may get better performance in conjunction with our AVPro Decklink asset.

Kind regards,
Chris

 

@Chris-RH is this forum the proper place to get official support or should I be sending my issue elsewhere?

 

Hi ToxMox,

In the demo scenes, press spacebar to hide the debug mode selection UI as this eats up a fair amount of frame rate. And it can also help to not have the LiveCamera element selected in the inspector.

I've had a look at the specs and unknown is indeed H264

• H264 - Up to 2160p60
• MJPEG - Up to 2160p60
• Uncompressed (NV12) - Up to 1080p60

We were actually looking into trying to purchase one of these for troubleshooting, but they don't seem to be easy to get hold of.

Have you tried disabling vsync?

Is this the same device that you were previously having trouble with last year?

Kind regards,
Chris

 

Hi, I know this has already been asked last year but i would like to know if you are planning to support macOS soon or if it is still a long term plan?
Thanks!

 

Hi @Dan_G,

Its not going to happen this year I'm afraid. We're currently working on updating our main asset, AVPro Video. Once that is done, we can look at other assets. We have a few other features that we want to add to Live Camera as well as Mac support, so are keen to get it done. We know that there are a lot of customers wanting Mac support, so it is important to us.

Kind regards,
Chris

 

Hi @Chris-RH

Elgato just posted today on their socials that the Facecam Pro is back in stock.
Yup, tried disabling vsync.
No this is not the same device I previously had trouble with.

 

Hi,

I found a bug that is making Unity crash. Whenver i select the Polypop camera (it got installed when i installing the software https://www.polypoplive.com/) and i exit Playmode, Unity crashes. I can enter playmode and switch between my webcams or virtual cameras without it crashing but when i stop/exit Playmode, it crashes all the times.

Can you please have a look at this crash? I am on Windows 11, using Unity 2022.2.0b11 and i saw it also happening on 2022.2.1f1.

Thanks!

 

Hi @ToxMox
Could you open up this issue in our github please so we can keep better track of it as it is quite complex.
https://github.com/RenderHeads/UnityPlugin-AVProLiveCamera/issues/new/choose
We'll probably have to get hold of one to be able to fix this though
Kind regards,
Chris