Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

[RELEASED] Dlib FaceLandmark Detector

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Jun 4, 2016.

  1. moyado

    moyado

    Joined:
    Jan 2, 2013
    Posts:
    7
    Is there an example for getting Natcam and OpenCV for Unity working with the Dlib AR sample? .

    That would be great. I wonder if that could improve the webcam resolution and speed....
     
    Last edited: Nov 5, 2018
  2. thanhle

    thanhle

    Joined:
    May 2, 2013
    Posts:
    162
    When i replace video dance.avi by my video ... The system not work !
    How i can fix this ?
     
  3. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  4. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Please replace the WebCamTextureToMatHelper of ARHeadWebCamTextureExample with NatCamPreviewToMatHelper.
     
    ina likes this.
  5. neomarian

    neomarian

    Joined:
    May 11, 2012
    Posts:
    43
  6. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  7. neomarian

    neomarian

    Joined:
    May 11, 2012
    Posts:
    43
    It will be implemented in a new update? It will be very useful to me. Or how much do you want to implement this new feature?
     
    Last edited: Nov 30, 2018
  8. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    For now, I do not plan to implement that method in DlibFaceLandmarkDetector.
     
  9. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  10. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  11. Gamify3D

    Gamify3D

    Joined:
    Dec 14, 2016
    Posts:
    1
    Can anyone verify that the frame rate optimizer example for the Dlib FaceLandmark Detector works on IOS? I would like to know before I buy it. I am skeptical because the openCV examples don't work on my android phone (crashes). Thank you!
     
  12. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    I apologize for the inconvenience.
    OpenCV for Unity v2.3.3 is in pending review status.
    I asked Unity Support about the review. I got a reply that the Asset Review Team is now in the holiday season so it will take some more time to finish the review.

    Until OpenCVForUnity 2.3.3 is published in AssetStore, you can download examples of older versions from the following DL link.


    DlibFaceLandmarkDetector_v1.2.5_to_v1.2.4_Patch.unitypackage
    https://www.dropbox.com/s/s4ggi4ps4...ctor_v1.2.5_to_v1.2.4_Patch.unitypackage?dl=0
    By importing this unitypackage you can downgrade DlibFaceLandmarkDetector v1.2.5 to DlibFaceLandmarkDetector v1.2.4. Please import DlibFaceLandmarkDetector v1.2.5 from AssetStore and then import this patch. FaceSwapperExample 1.0.7, FaceMaskExample 1.0.6 and CVVTuberExample 1.0.1 require DlibFaceLandmarkDetector 1.2.4.


    CVVTuberExample1.0.1
    https://www.dropbox.com/s/tccnr216bwq1xiq/CVVTuberExample1.0.1.unitypackage?dl=0

    DlibFaceLandmarkDetectorPlayMakerActions1.0.2
    https://www.dropbox.com/s/jkwsk6c2p...tectorPlayMakerActions1.0.2.unitypackage?dl=0

    OpenCVForUnityPlayMakerActions1.0.5
    https://www.dropbox.com/s/x4xo907pnda7chh/OpenCVForUnityPlayMakerActions1.0.5.unitypackage?dl=0

    FaceMaskExample1.0.6
    https://www.dropbox.com/s/aly2gtf5vmji6x6/FaceMaskExample1.0.6.unitypackage?dl=0

    FaceSwapperExample1.0.7
    https://www.dropbox.com/s/6gwa0v8eunqjnv5/FaceSwapperExample1.0.7.unitypackage?dl=0

    FaceTrackerExample1.1.8
    https://www.dropbox.com/s/u2cx1nlkddz8lnn/FaceTrackerExample1.1.8.unitypackage?dl=0

    MarkerBasedARExample1.2.1
    https://www.dropbox.com/s/u2rfto39ohu18ja/MarkerBasedARExample1.2.1.unitypackage?dl=0

    MarkerLessARExample1.0.4
    https://www.dropbox.com/s/rpovz53y3hhhlgj/MarkerLessARExample1.0.4.unitypackage?dl=0

    RealTimeFaceRecognitionExample1.0.4
    https://www.dropbox.com/s/rjot3qaih580xyp/RealTimeFaceRecognitionExample1.0.4.unitypackage?dl=0
     
  13. heaversm

    heaversm

    Joined:
    Jul 12, 2011
    Posts:
    11
    I'm having an issue with DlibFaceLandMark Detector running the ARHead example in the editor, but not when compiled (with both scenes selected in build settings). It loads the scene, the camera starts, but then shuts off, and I never do see a video feed. The ARHead example compiles fine when it is the only scene in the build.

     
  14. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Could you tell me the environment you tried?
    DlibFaceLandmarkDetector version :
    OpenCVForUnity version :
    Unity version :
     
  15. heaversm

    heaversm

    Joined:
    Jul 12, 2011
    Posts:
    11
    I've noticed that the WebcamTextureExample only gets about 5 fps on my machine (a newer Macbook Pro), whether I run it in the editor or not. Is there any way to speed this up (other than to run it on a more powerful machine). 5fps is almost unusable.

    I've noticed if I stop rendering the quad (that renders the webcam image), it jumps up to 80fps.
     

    Attached Files:

  16. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    The resolution of the image that can be acquired by Unity's WebCamTexture seems to differ depending on the WebCam you use. Perhaps the image resolution that can be acquired with webcams of macbook pro is the maximum resolution only.

    Also,
    Using ImageOptimizationHelper.cs it is possible to scale down images easily. ImageOptimizationHelper.cs is used in FrameOptimizationExample.
    https://github.com/EnoxSoftware/Dli...timizationExample/FrameOptimizationExample.cs
     
  17. fcglobalmedia

    fcglobalmedia

    Joined:
    May 22, 2017
    Posts:
    9
    Hello! I bought OpenCV for Unity and DlibFaceLandmarkDetector and everything is working fine.

    However, I'd like your help regarding a problem I'm facing: using "WebCamTextureFaceMaskAdditionalExample" scene, how can I increase the size of the FaceMask to cover more of a face? I mean, is it possible to make it cover the ears and also more of the chin and forehead (with the forehead I was able to by slightly changing "AddForeheadPoints()" methods and increasing its points values)?
    Let me know if i wasn't clear enough, thanks in advance!
     
  18. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    If you use your own extended face landmark points, you need to create corresponding mesh data.
    For details on how to create new mesh data using scripts, please refer to the "ExtendForehead()" method.
     
  19. DataArtDev

    DataArtDev

    Joined:
    Mar 5, 2019
    Posts:
    1
    Hi, @EnoxSoftware,

    I have bought your OpenCVForUnity and DlibFaceLandmarkDetector packages. Everything works fine, but in my project I need some more detalisation in lipsync. I have tried to train my own shape predictors on publically available datasets (like Helen), but your sp_human_face_68.dat looks much better in most cases.

    Can you please tell which dataset you used to train your sp_human_face_68 shape predictor?
    Which parameters did you use?
     
  20. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Since the training dataset consists of flickr CC0 licensed images, the Shape Predictor model files are available for commercial use. For details of the training data set, please refer to "DlibFaceLandmarkDetector/DlibFaceLandmarkDetectorTrainingDataset.txt".
     
  21. ptolemy_unity

    ptolemy_unity

    Joined:
    Oct 22, 2018
    Posts:
    2
    @EnoxSoftware
    I have just bought OpenCV and Dlib Facelandmark detector.

    'ARHeadWebCamTextureExample' is not included in the package, despite that scene being used in the asset store thumbnail. Also, the test scene has a button to take me there that is simply broken. Where is the scene?

    I only need simple face tracking, and to be able to overlay a 2D image onto that face in real time.
     
  22. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    In the example combining OpenCVForUnity and DlibFaceLandmarkDetector, you need to import additional packages. Please see ReadMe.pdf for details.
    • Import and Setup the “OpenCV for Unity”.
    • Import the "Assets/DlibFaceLandmarkDetector/ DlibFaceLandmarkDetectorWithOpenCVExample.unitypackage".
    • Add all of the “***.unity” in the “DlibFaceLandmarkDetectorWithOpenCVExample” folder to [Build Settings] – [Scene In Build].
    WithOpenCVSampleSetup.PNG
     

    Attached Files:

  23. Stevekratos

    Stevekratos

    Joined:
    Mar 14, 2019
    Posts:
    1
    Purchased the assets(OpenCV for Unity,Dlib face landmark detector) to create facemask and final build size is 170+ Mb and performence in devices like nokia 6.1+ is 10-15 fps.
    is there a way to reduce the size and improve the performence(Removed unwanted examples also).
     
    ina likes this.
  24. reinabada

    reinabada

    Joined:
    Apr 13, 2017
    Posts:
    7
    Hi! Thanks for creating this! I am having a lot of fun with the Mask example. My question is: can I get rid of the small MaskFace picture that shows next to my face? Trying to make a simple Makeup game and I would love to get rid of that image while trying different makeup products. I realized the FaceMaks image thumbnail is rendered in the Canvas with the phone camera image.

    Thank you!
    Screensho with my coments in red so I can make myself understood better.
     

    Attached Files:

    Last edited: Apr 21, 2019
    kisstech likes this.
  25. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    The example does not provide the option to hide the mask preview.
    Could you comment out lines 530 to 561 of the code (WebCamTextureFaceMaskExample.cs)?
    https://github.com/EnoxSoftware/Fac...ple/WebCamTextureFaceMaskExample.cs#L530-L561
     
  26. KingKong320

    KingKong320

    Joined:
    Mar 2, 2018
    Posts:
    21
    hi
    is there any way to draw landmarks for foot detection?
     
  27. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    In order to detect the foot, it is necessary to load a model for foot detection.
     
  28. VideoTutorial

    VideoTutorial

    Joined:
    Jun 27, 2017
    Posts:
    29
    @EnoxSoftware Can this detect and track a patch on the forehead? (and how?)
     
  29. KingKong320

    KingKong320

    Joined:
    Mar 2, 2018
    Posts:
    21
    i guess its not worthy for mobile apps becuause the size of APK is minimum 170 MB which is not worthy for me at all.if it still works for you than use it. And as much i have read this plugin there is nothing to detect just forehead.
     
  30. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Sorry for my late reply.
    It is possible to reduce the app size by using model file for mobile apps ( sp_human_face_68_for_mobile.dat or sp_human_face_17_for_mobile.dat).
    Unfortunately, the model file included with this asset does not contain the forehead points.
    landmark_points_68.png
     
  31. kisstech

    kisstech

    Joined:
    Apr 7, 2017
    Posts:
    2
    How to Cover All Faces with Masks?
     

    Attached Files:

  32. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Could you try "Assets/FaceMaskExample/WebCamTextureFaceMaskAdditionalExample" ?
    WebCamTextureFaceMaskAdditionalExample is an example of adding forehead points.
     
  33. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Hey, amazing asset!
    Cannot get it to run on webgl though.
    I'm getting a missing function: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6resizeEjc exception on the build.
    Also, I had to modify the example code for the facemask detection since this line:
    DlibFaceLandmarkDetector.UnityUtils.Utils.getFilePath ("sp_human_face_68.dat");
    was returning an empty string.

    I have both OpenCV and DLib installed, the latest releases on the store,
    I have done no configuration, I'm expecting it to work out of the box.

    Here is another report of the same problem:
    https://forum.unity.com/threads/webgl-development-build-always-fails-2019-1-8f1.700754/
     
  34. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Could you tell me the environment you tried?
    Unity version :
     
  35. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    It was Unity 2019.2.0f1
     
  36. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    If properly "ImportSettings" of "opencvforunity.bc" is not set, an emcc error will occur.
    By selecting MenuItem[Set Plugin Import Settings], ”opencvforunity.bc” corresponding to version of Unity editor is automatically set.
    importsettings1.png
    importsettings2.png
     
  37. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    I have tried that without success:

    missing function: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6resizeEjc
    printErr @ UnityLoader.js:4
     
  38. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    I somehow managed to solve the issue with the build. Now the camera doesn't activate. The permission is never requested.

    http://ideas.farm/TestWebGLDLib/
     
  39. Yeung

    Yeung

    Joined:
    May 31, 2013
    Posts:
    2
    Is there any planning to support gpu acceleration for dlib ?
     
  40. douglima

    douglima

    Joined:
    Apr 26, 2017
    Posts:
    11
    Hi, I'm having an issue where the webgl build takes too long to start the camera. Any ideia what could be causing it? It seems like it's downloading something before the initialization.
     
  41. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  42. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  43. douglima

    douglima

    Joined:
    Apr 26, 2017
    Posts:
    11
  44. pekarnik1

    pekarnik1

    Joined:
    Sep 11, 2019
    Posts:
    5
    Hello! Do you have an example of ArHead WebCam with NatCam Texture?
     
  45. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
  46. pekarnik1

    pekarnik1

    Joined:
    Sep 11, 2019
    Posts:
    5
  47. pekarnik1

    pekarnik1

    Joined:
    Sep 11, 2019
    Posts:
    5
    Can I create a transparent facemash behind which I will hide 3D objects(that is, if I open my mouth, the object will be visible inside it, when closed-not) Tried to make it with ARHeadWebCamTextureExample, apply a transparent shader to it, but how can I do same thing with face mask?
     
  48. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    Last edited: Sep 16, 2019
  49. HeladioMoralesDorantes

    HeladioMoralesDorantes

    Joined:
    Mar 2, 2019
    Posts:
    3
  50. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,574
    I don't have an example of Face Alignment with OpenCVForUnity and DlibFaceLandmarkDetector. However, since getRotationMatrix2D method and warpAffine () method is implemented, I think it is possible to implement Face Alignment using OpenCVForUnity.

    getRotationMatrix2D
    https://enoxsoftware.github.io/Open...mgproc.html#ab9b8b89051891d37da62abf9bd0dfb5a

    warpAffine()
    https://enoxsoftware.github.io/Open...mgproc.html#a74017885add09e19513333b797acc29b