Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

AVPro Live Camera

Discussion in 'Assets and Asset Store' started by AndrewRH, May 24, 2012.

  1. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I believe we would have to write a camera-specific plugin to access those. I'm not sure, it's been a quite since we worked with machine-vision cameras..
     
  2. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi All, sorry about the cross-post, but I just wanted to announce a new RenderHeads plugin for Unity:
    External Game View

    It's a plugin for multi-monitor productivity and workflow improvement while using the Unity Editor and it's really changed the way we work. You can find out all about it here where you'll find documentation and a tutorial video.

    Thanks,
     
  3. boubouk

    boubouk

    Joined:
    Oct 30, 2019
    Posts:
    3
    I'm struggling with device activity and managing exceptions. I need to check if the camera is working properly to restart it or else. Basically:
    Code (CSharp):
    1. if (avplCamera.Device == null)
    2. {
    3.     avplCamera.SelectDeviceAndMode ();
    4.     RestartDevice ();
    5. }
    6. else if (!AVProLiveCameraPlugin.IsDeviceConnected (avplCamera.Device.DeviceIndex))
    7. {
    8.     RestartDevice ();
    9. }
    10.  
    But this is only two cases where the device is failing. I tried:
    avplCamera.Device.isRunning <- can stay true if the camera has not been stopped correctly (even if camera is disconnected)
    avplCamera.Device.isActive <- Same
    avplCamera.Device.isConnected <- Same (which is clearly wrong)

    Sometimes, when my camera stops, all these booleans remain unchanged.
    Is this a bug or am I missing the way to manage this properly.
    This is gonna be installed for a year in an exhibition, I need to be able to detect anomalies et failures.
    Thanks.
     
    Last edited: May 10, 2022
  4. seabits

    seabits

    Joined:
    Dec 3, 2019
    Posts:
    199
    Hi @boubouk

    The way to monitor a device's connection in Live Camera is to use Hot Swapping. This can be enabled by checking the 'support Hot Swapping' box in the LiveCamera component.

    Let us know if this helps?
     
  5. boubouk

    boubouk

    Joined:
    Oct 30, 2019
    Posts:
    3
    It's already checked. But Update Hot Swap in Live Camera component was unchecked. By enabling it, camera is restarting after reconnection.
    Will investigate further.
    Thanks for the help.
     
    seabits likes this.
  6. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    477
    @TerryRH I was using this asset for a while on my Windows-Only project, but since I had to port my project to MacOS, I was't able to use it and had to switch to NatDevice.
    Unfortunately NatDevice is now locking existing customers from updating in favor of a subscription model and that's not something I am able to use right now.

    So I apologize if it was asked a million times, but is there any chance of this plugin adding MacOS support?

    Thanks!
     
  7. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi, thanks for your message. In short, there are plans for MacOS support, but they are very much long term plans, so we can't help you at the moment unfortunately.
     
  8. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    477
    Ah, okay, thank you for the clarification! I look forward to MacOS support whenever it does become available!
     
    Chris-RH likes this.
  9. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    477
    I just want to ask a follow up question:
    I'm plan on using this for the Windows version of my app and will be using WebCamTexture for a "dumbed-down" implementation for MacOS.
    To do this, I will use Scripting defines, but will there be any issue building for Mac with the plugin installed?
    Is there a better way of writing the dual support than Scripting Defines?

    Thanks!

    Edit: I'm trying to access the outputTexture after calling Begin() on a Live Camera, but it's always being null for some reason. The LiveCamera inspector is working as expected.
    Do I need to access the outputTexture in update? My intention is to grab it once, assign it to a material/UI element and change it when I change the camera. Is this not possible?

    I got it working by commenting out part of this line in the CameraDevice class.
    After doing some debugging,
     _formatTexture.ValidPicture
    is always null despite the camera working fine.
    Code (CSharp):
    1.      
    2. public Texture OutputTexture
    3.         {
    4.             get { if (_formatConverter != null/* && _formatConverter.ValidPicture*/) return _formatConverter.OutputTexture; return null; }
    5.         }
    Thanks again!

    Edit 2:
    To follow up about the Mac/PC issue: I figured I would add the manager and live camera components at runtime if the platform is windows, then move from there. This works in the editor, but not in builds -- it requires the components to be in the scene from the beginning.
    But, if the components are in the scene, I get errors on mac.
    My solution was to have them in the scene on an inactive gameObject, which gets enabled if the platform is Windows.
     
    Last edited: Jun 11, 2022
  10. mchangxe

    mchangxe

    Joined:
    Jun 16, 2019
    Posts:
    69
    whats the api to set and update camera settings in script? Like an exposure setting maybe.
     
  11. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    I'm sorry, we're really not sure about this one, so can't help you further. But we'd love to hear about your solution if you get it working.
     
  12. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @mchangxe
    We don't actually have an API reference list, but the scripting can easily be understood by examining the script in the CameraExplorerDemo scene.
     
  13. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    477
    I just used scripting defines and disabled the components if the platform is mac.

    But the bigger problem is why _formatTexture.ValidPicture is always false. Any idea why this would be?
    I'm using basically the same code as in the Camera Explorer Demo just adapted to use UnityUGUI and creating my own wrapper classes for making it simpler.
    Commenting out part of that line makes it work, but obviously that's not a real solution.
    I'm using a regular crappy USB webcam, then Canon EOS Utility with an M50mII.
    Thanks!
     
  14. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    No, idea why its not working, sorry.
    LiveCamera uses Microsoft DirectShow API, is that what is blocking it?
     
  15. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Hi,

    I'm wondering if this issue was ever resolved?

    Thanks,
    Dave
     
  16. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi Dave,

    I'm very sorry that this hasn't been resolved for you. The dev who had been working on that has taken a break to pursue studies. I can try and contact him to see if he has any notes from when he was last working on it We're a very small team, so unfortunately I'm not sure when I can get someone to delve back into it, but I will try.

    Kind regards,
    Chris
     
  17. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Hi Chris,

    Ok yes, please do try that. It was two years ago. I bought the asset back then thinking it would be fixed.
    I'm getting closer to release now and am hoping to get this issue resolved.

    Thank you,
    Dave
     
  18. RichRH

    RichRH

    Joined:
    Jan 11, 2019
    Posts:
    13
    Hi @ToxMox,

    I'm trying to look into this issue - apologies if I repeat anything but would you be able to post your Unity console log from running the CameraExplorerDemo to clicking on the already highlighted (240) mode in 'Modes' please?

    Cheers,
     
  19. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    @RichRH Sure, here you go. Thanks :)
     

    Attached Files:

    RichRH likes this.
  20. RichRH

    RichRH

    Joined:
    Jan 11, 2019
    Posts:
    13
    Hi @ToxMox,

    OK, I think I've identified the issue - your capture card doesn't expose the 'IAMVideoControl' interface which Live Camera uses to enumerate framerates, so instead will just see the minimum and maximum supported (hence you seeing only 24hz and 240hz options).

    As an intermediate solution I'm going to look at exposing a few sensible rates within the provided range, including the camera default (in your case 60hz). These changes will go into the next release which should be early September.

    Cheers,
     
  21. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Excellent! Thank you so much :D
     
  22. RichRH

    RichRH

    Joined:
    Jan 11, 2019
    Posts:
    13
    Hi All,

    AVPro Live Camera version 2.9.2 has just been released!

    You can find the updated version on the Asset Store and the free trial version on our website.

    Changes include:

    Improvements
    • Added some default frame rates between provided min/max for devices that don’t support ‘IAMVideoControl’ interface
    Changes
    • Tested against Unity 2021/2022 (demos use obsolete, but still functioning, code)

    Thanks to everyone that reported bugs that were fixed in this release :)

    Please report any issues here on the forum, or preferably to our Github Issue tracker

    Thanks,
     
  23. RichRH

    RichRH

    Joined:
    Jan 11, 2019
    Posts:
    13
    @ToxMox A bit later than planned but hopefully the above release fixes your issue.

    Cheers,
     
  24. dfashbaugh

    dfashbaugh

    Joined:
    Feb 4, 2017
    Posts:
    2
    Hi, I'm attempting to run any of the demos that come with this Asset, and I get a crash every time. It seems to be crashing while initializing the plugin. I've tested this in both 2021.3.10f1 and 2020.3.40f1 with the same results. Any help would be appreciated.

    Here is a piece of my stack trace if it helps

    =================================================================
    Native Crash Reporting
    =================================================================
    Got a UNKNOWN while executing native code. This usually indicates
    a fatal error in the mono runtime or one of the native libraries
    used by your application.
    =================================================================

    =================================================================
    Managed Stacktrace:
    =================================================================
    at <unknown> <0xffffffff>
    at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraPlugin:Init <0x00098>
    at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraManager:Init <0x00022>
    at RenderHeads.Media.AVProLiveCamera.AVProLiveCameraManager:Start <0x0005a>
    at System.Object:runtime_invoke_void__this__ <0x00087>
    =================================================================
     
  25. dfashbaugh

    dfashbaugh

    Joined:
    Feb 4, 2017
    Posts:
    2
    Quick update on mine. I found the issue. I had NDI Tools installed which includes a virtual Webcam for NDI streams. I removed that and it works now! (Although, it would be awesome if this supported NDI Virtual Webcams!)
     
    satoshi_jp likes this.
  26. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @dfashbaugh
    Yes, we are looking into resolving that issue with NDI Tools.
    Kind regards,
    Chris
     
  27. RichRH

    RichRH

    Joined:
    Jan 11, 2019
    Posts:
    13
    Hi All,

    AVPro Live Camera version 2.9.3 has just been released!

    You can find the updated version on the Asset Store and the free trial version on our website.

    Changes include:

    Bug fixes
    • Fixed crash when NDI devices are present
    Thanks to everyone that reported bugs that were fixed in this release :)

    Please report any issues here on the forum, or preferably to our Github Issue tracker

    Thanks,
     
  28. Soaryn

    Soaryn

    Joined:
    Apr 17, 2015
    Posts:
    328
    I've noticed in 2022.2b13, the device output texture seems to always be null with an Elgato Facecam. This was not the case before. The texture IS available if gone through the Plugin side with `AVProLiveCameraPlugin.GetFrameAsColor32` but this is cumbersome and the amount of time investment is not ideal. I haven't yet tested with other hardware, but I would suspect similar results.


    Some notes, this is on the latest AVPro Live Camera, HDRP Sample scene, with the RayTracing quality preset chosen.
    Printing the name and then the OutputTexture:
    upload_2022-11-1_23-23-51.png

    Code (CSharp):
    1. Debug.Log(Camera.Device.Name);
    2. Debug.Log(Camera.OutputTexture);
    Even after getting everything to work when using the direct plugin method way the manager indicates that the framerate is at 0 but the texture IS updating at 60fps...
    upload_2022-11-1_23-46-56.png
     
    Last edited: Nov 2, 2022
  29. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @Soaryn,

    Thanks for the heads up. I take it that you didn't have this issue in previous versions? We don't do support for beta versions, so hopefully the issue will get fixed before release. Its certainly something that we can look out for. Have you reported it to Unity?

    Kind regards,
    Chris
     
  30. Soaryn

    Soaryn

    Joined:
    Apr 17, 2015
    Posts:
    328
    Haven't had this issue before, no. I don't have much choice over use of Beta however, since the Latest ECS is limited to 2022 and that takes priority over camera sources.

    As for reporting it to Unity, I'm not sure how this would be a Unity related issue (so no I haven't), as this is specifically the Device.OutputTexture that is Null. If you Blit a null texture to a render texture, you get rather odd results, usually what ever is on the active scene camera. For now, I have a work around, but the results taking 8ms of cpu time to do which is non-ideal. Do note, that Unity's HDRP likely underwent rather large changes in 2022 if I recall correctly.

    Also, related to the workaround:
    Code (CSharp):
    1. var result = AVProLiveCameraPlugin.GetFrameAsColor32(_device.DeviceIndex, (IntPtr)_frame.GetUnsafePtr(), _width, _height);
    This takes roughly 6ms on its own to call.
     
    Last edited: Nov 3, 2022
  31. Dustybike

    Dustybike

    Joined:
    Sep 14, 2019
    Posts:
    1
    Hi Team. For debug image logging I am working on I need to grab the image data cpu side of the lastFrameUpdated.
    Currently I am using the main thread to update the GPU side images and use formatConverter -> lastFrameUpdated to get the frame number. I then have a rolling buffer in a second thread grabbing every new frame using GetFrameAsColor32 and store the correct frame by frame number. If i don't have the rolling buffer i end up grabbing the frame too late. This is fairly performance and memory wastey. Can you implement a function the same as GetFrameAsColor32 that returns the last updated frame? alternately it looks like you have a rolling buffer internally considering the command GetFrameFromBufferAtTime maybe a GetFrameFromBufferByNumber? Thanks
     
  32. Chri5topheru5

    Chri5topheru5

    Joined:
    Dec 21, 2017
    Posts:
    5
    Hi,
    we bought the plugin already and are wondering if the audio support is in your backlog for future releases. If not, is it possible to use the DeckLink plugin for capturing audio and video from other cards as well (elgato, ...)
    Thanks.
     
  33. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
  34. kyuhyoung

    kyuhyoung

    Joined:
    Jul 16, 2012
    Posts:
    3
    Where can I find example C# codes for switching the camera resoluton of a webcam when a key is pressed?
    Do I have to use "hot swapping"?
    My senario is like this :
    1. In Start() function
    1.1 Prepare RenderTexture rt_1280_720 for the resolutoin 1280 x 720
    1.2 Prepare RenderTexture rt_1920_1280 for the resolutoin 1920 x 1080
    1.3 Start webcam with 1280 x 720 resolution
    2. In Update() funtion
    2.1. If "R" key is pressed
    2.1.2 If current resolution is 1280 x 720
    2.1.2.1 Change the resolution to 1920 x 1080
    2.1.3 else
    2.1.3.1 Change the resolution to 1280 x 720
    2.2. Copy OutputTexture of AVProLiveCamera to rt_1280_720 or rt_1920_1080 accordingly.
    2.3. Display rt_1280_720 or rt_1920_1080 accordingly.

    I mean for a single camera but multiple resolutions. Not between multiple cameras.
     
    Last edited: Mar 8, 2023
  35. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Hello. Having some problems with the Elgato Facecam Pro:
    1. The 3840x2160 [MJPG] mode is stuck at 30hz even when I specify 60hz
    2. The 3840x2160 [Unknown] mode just doesn't work. I believe these "unknown" modes are h264 based on the attached DirectShow Capture Capabilities output. The 1920x1080 [Unknown] mode works
     

    Attached Files:

  36. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @kyuhyoung,

    Hot Swapping is basically just for changing in and out different physical cameras, so it will repeatedly monitor the device connection.
    You should find some information on scripting in the PDF documentation that comes with the asset. We provide components that do the job, but generally leave custom scripting as an exercise for the developer.

    Kind regards,
    Chris
     
  37. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @ToxMox,

    Out of interest, do you get the same result using Unity webcam texture?
    Have you tested it as a build?
    Have you made sure that you are in a brightly lit room?

    Some easy things you can do that might help with FPS:
    • You should check the settings of the camera to make sure it isn’t doing unnecessary processing which reduces the frame rate. Disable options like “low light compensation”, disable auto exposure and reduce exposure settings. You can access camera settings via the CameraExplorer demo (click “Configure”), or usually via the camera setup application on the Start Menu.

    • Another possibility is that the data bus is overloaded. You may need to plug devices into different ports so they are on a different USB bus.

    • You could also check that the checkboxes for Hot Swapping support and Settings Update are not checked in the AVProLiveCameraManager and AVProLiveCamera components. On some device drivers polling for device connection status and checking the settings can really slow things down and could be done manually instead.

    This is a pretty new camera. We might see if we can get one for testing in-house. We will most likely have to start supporting Media Foundation with newer webcams like this.

    Kind regards,
    Chris
     
  38. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Hi @Chris-RH,

    Thanks for getting back to me. Please see my responses in blue below (you'll need to expand the quoted text to see them all):
     
  39. ImNotEvilsizer

    ImNotEvilsizer

    Joined:
    May 25, 2017
    Posts:
    20
    I am trying to use GetFrameAsColor32 to grab a frame from my webcam. My webcam seems to have 2 format types. One that identifies as YUV_422_YUY2, and one that identifies as RAW_BGRA32. Any resolution / frame rate I pick that is the YUV mode works perfectly ... Any resolution / frame rate I pick that is RAW mode does not work at all (pretty much comes through as all zero values). Just wondering if you have any advice to get it working as the resolution and frame rate I want to use only pops up in the RAW mode.

    Thanks for any help!!!
     
  40. ImNotEvilsizer

    ImNotEvilsizer

    Joined:
    May 25, 2017
    Posts:
    20
    Also .... I wonder what GetFrameFromBufferAtTime does and if it could help me? I am actually trying to use LiveCamera to do a sports training "instant replay" sort of project. Like when I hit space, go back about 5 seconds in time and show a slo-mo replay of what the camera saw starting 5 secs previous. What I was trying was just storing the last 5 secs in Color32 arrays and manually playing them back. Seems to work OK as long as mode is the YUV (again RAW mode will not grab as Color32 for me) ... But if there are already frames being stored at times maybe I could tap into that instread???

    Sorry for all the questions but I cannot find too much documentation of these functions. Thanks!!
     
  41. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @ImNotEvilsizer,
    OK, that's not really a common use-case. Our plugin's goal is to get video from webcam-type devices into Unity textures as quickly as possible, so we use the texture format that is closest to the webcam device's native format.

    Saying that, I'm surprised that the RAW format is not working, as we've certainly had a few users who have successfully used it. What camera are you using? Are you getting any error messages?
    Have you tried to create a RenderTexture in the format that you require then you can call Graphics.Blit(sourceTexture, destTexture); to convert between formats.
     
  42. ImNotEvilsizer

    ImNotEvilsizer

    Joined:
    May 25, 2017
    Posts:
    20
    Thanks for the quick reply. I have a TECURS 1080P 60FPS webcam I got off Amazon for $20. Hahaha ... It was the cheapest webcam I could find. Kind of embarrassing, but it actually provides a pretty clear, solid image and I wanted to prove my concept before I bought any expensive cameras.

    I don't get any error messages. The Color32 array is just usually all 0's or sometimes a little garbage mixed in. But only in the RAW modes. Again, the YUV modes work as expected.

    Using your idea of Graphics.Blit I decided to just save each frame using Graphics.CopyTexture() into an array of Texture2Ds that I can replay using a RawImage UI component. If I need access to the pixel data I will just use GetPixels32 or whatever from the Texture2Ds. Just tested it and it is running nicely ... WAY faster than the WebCamTexture approach I started with.

    So I have a very good work around.

    I plan to have 3 cameras total in the final package. While I have your attention, do you think getting a Quad Black Magic card for the 3 camera inputs would make things run even better? I see your plugin supports it. I have NO experience with those cards and would hate to buy one and have little to no performance boost from it.

    Thanks for your help. Always nice when a developer supports his product.
     
  43. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @ImNotEvilsizer

    Heh, we don't tech-shame here :) I think mine cost about the same ;) It would be very shortsighted to only support high-end hardware.

    I would suggest that the next step is to add your extra camera(s) and see how it performs. The Blackmagic Quad is unlikely to bring much to Live Capture (also, it requires HDMI input). However, if it was something that you wanted to use, then you may get better performance in conjunction with our AVPro Decklink asset.

    Kind regards,
    Chris
     
  44. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    @Chris-RH is this forum the proper place to get official support or should I be sending my issue elsewhere?
     
  45. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi ToxMox,

    In the demo scenes, press spacebar to hide the debug mode selection UI as this eats up a fair amount of frame rate. And it can also help to not have the LiveCamera element selected in the inspector.

    I've had a look at the specs and unknown is indeed H264
    • H264 - Up to 2160p60
    • MJPEG - Up to 2160p60
    • Uncompressed (NV12) - Up to 1080p60
    We were actually looking into trying to purchase one of these for troubleshooting, but they don't seem to be easy to get hold of.

    Have you tried disabling vsync?

    Is this the same device that you were previously having trouble with last year?

    Kind regards,
    Chris
     
  46. Dan_G

    Dan_G

    Joined:
    Dec 4, 2017
    Posts:
    31
    Hi, I know this has already been asked last year but i would like to know if you are planning to support macOS soon or if it is still a long term plan?
    Thanks!
     
  47. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224
    Hi @Dan_G,

    Its not going to happen this year I'm afraid. We're currently working on updating our main asset, AVPro Video. Once that is done, we can look at other assets. We have a few other features that we want to add to Live Camera as well as Mac support, so are keen to get it done. We know that there are a lot of customers wanting Mac support, so it is important to us.

    Kind regards,
    Chris
     
  48. ToxMox

    ToxMox

    Joined:
    Jul 23, 2020
    Posts:
    25
    Hi @Chris-RH

    Elgato just posted today on their socials that the Facecam Pro is back in stock.
    Yup, tried disabling vsync.
    No this is not the same device I previously had trouble with.
     
  49. Dan_G

    Dan_G

    Joined:
    Dec 4, 2017
    Posts:
    31
    Hi,

    I found a bug that is making Unity crash. Whenver i select the Polypop camera (it got installed when i installing the software https://www.polypoplive.com/) and i exit Playmode, Unity crashes. I can enter playmode and switch between my webcams or virtual cameras without it crashing but when i stop/exit Playmode, it crashes all the times.

    Can you please have a look at this crash? I am on Windows 11, using Unity 2022.2.0b11 and i saw it also happening on 2022.2.1f1.

    Thanks!
     
  50. Chris-RH

    Chris-RH

    Joined:
    Apr 21, 2022
    Posts:
    224