Search Unity

[RELEASED] Dlib FaceLandmark Detector

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Jun 4, 2016.

  1. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Please disable all check boxes for plugins not in use.
    importSettings_same_name_error.PNG
     
  2. james14123

    james14123

    Joined:
    Jan 22, 2017
    Posts:
    20
    Hello
    Is possible to detect face in different direction?
    It seems like face tracking only work in landscape orientation when i use Iphone.
    I have no idea about tracking in portrait orientation.

    Can you give some tips ? Thanks !
     
  3. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
  4. jamestrue

    jamestrue

    Joined:
    Jan 25, 2017
    Posts:
    3
    Hi folks, I have bought OpenCV and Dlib FaceLandmark Detector and the advanced samples are NOT in either package. Does anyone know where I can find "WebCamTextureARExample" It is not included along with any other "Advanced Examples" as it says in the ReadMe.pdf?
     
  5. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    The procedure is described in ReadMe.pdf.

    Quick setup procedure to run the Advanced examples using “OpenCV for Unity”scene
    1.Import and Setup “OpenCVfor Unity”.
    2.Unzip “DlibFaceLandmarkDetectorWithOpenCVExample.zip”, Import DlibFaceLandmarkDetectorWithOpenCVExample.unitypackage.
    3.Add all of the “***.unity” in the “DlibFaceLandmarkDetectorWithOpenCVExample” folder to [Build Settings] –[Scene In Build].
     
  6. Nodrap

    Nodrap

    Joined:
    Nov 4, 2011
    Posts:
    50
    I am using OpenCV and dlib together for a project but since putting them together with a modified version of the webcam demo and adding it to our project, it now takes 2 hours to build in the cloud for Android and the iOS version doesn't work at all. It is a very large project now too so any advice on how to shrink it down would be great too.
     
  7. james14123

    james14123

    Joined:
    Jan 22, 2017
    Posts:
    20
    Hello i have tried both dlib and facemark. These are very useful tools. Is it possible to only make the face shape change but not the facial features ? Because in some situation, the target face will be too similar to another face. I want to change face and keep the original facial features simultaneously.

    Can you give some advices ? Thanks
     
  8. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    What error is displayed in the error log of UnityCloudBuild?
    Also, If you do not use opencv_contrib module, build size will be reduced by using native plugin file excluding opencv_contrib module.
    1. Replace OpenCVForUnity/Plugins/Android/libs folder to OpenCVForUnity/Extra/exclude_contrib/Android/libs folder.
    2. Delete OpenCVForUnity/Assets/OpenCVForUnity/org/opencv_contrib folder and OpenCVForUnity/Examples/Contrib folder.
     
  9. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    I think that this page will be helpful.
    https://www.auduno.com/2014/04/29/twisting-faces/
     
  10. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    841
    Why isn't this used as the main WebCamTextureToMatHelper?

     
  11. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    841
    Or, ways to match camera unit to real world scale
     
  12. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Please use OptimizationWebCamTextureToMatHelper.cs.
     
  13. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Unfortunately, I do not have such an example code.
     
  14. Sphoenix44

    Sphoenix44

    Joined:
    Sep 20, 2015
    Posts:
    5
    may I know the exact meaning of rectDetection.detection_confidence and rectDetection.rectDetection.weight_index ?
    especially what does "weight" stand for? Thanks a lot!
     
  15. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    http://dlib.net/dlib/image_processing/object_detector_abstract.h.html

    http://blog.dlib.net/2014/02/dlib-1...howComment=1452557469108#c4097701660774221609
     
  16. Gustavo-Quiroz

    Gustavo-Quiroz

    Joined:
    Jul 26, 2013
    Posts:
    36
    Hello Enox,

    I'm trying to load VideoPlayer.texture.GetNativeTexturePtr() into DlibFaceLandmarkDetector.FaceLandmarkDetector.SetImage ( IntPtr intPtr,
    int width,
    int height,
    int bytesPerPixel
    ), but it always crash.

    I'm trying this to see if I can optimize the flow by avoiding a move like converting VideoPlayer.texture to Texture2D and then to Mat or at leats set the frame image into the detector faster.

    Code (CSharp):
    1. faceLandmarkDetector.SetImage(videoPlayer.texture.GetNativeTexturePtr(), videoPlayer.texture.width, videoPlayer.texture.height, bytesPerPixel);
    *I have tested bytesPerPixel being 1, 3 or 4.
     
  17. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    intPtr argument of FaceLandmarkDetector.SetImage() corresponds to a pointer to the pixel data array.
    The return value of Texture.GetNativeTexturePtr () is a pointer to the native texture.
     
    Gustavo-Quiroz likes this.
  18. Gustavo-Quiroz

    Gustavo-Quiroz

    Joined:
    Jul 26, 2013
    Posts:
    36
    Thanks!

    I have another question, ¿Is there any async sample for Dlib?
     
  19. Morseliot

    Morseliot

    Joined:
    Jan 10, 2015
    Posts:
    46
    Hello everyone!
    Can't find any other place to report about bugs. In Unity Editor on windows based machine everything works just fine, but currently working on project for Linux and experiencing issues with OpenCV plugin for Unity. After making a build for Linux x86_64, opencv can't run with this error in debug log. Pic.1

    ***********
    Couldn't open home/username/Beta/Linux/project_Data/Plugins/x86_64/libopencvforunity.so, error libopencv_bgsegm.so.3.1:cannot open shared object file. No such file or derictory.
    *********** and next one*******
    DllNotFoundException: opencvforunity
    ****************end**************
    The libopencvforunity.so is inside plugin folder in project's data folder, but it stil gives this weird error.
    See that in Ubuntu 16.04 , build x86_64, opencv 3.1, integrated webcam.

    And next picture appears in same build type and os, but opencv 3.2...


    and it tries to get it form this path directory, that looks like windows path of project's origin.

    Could you please suggest any walk around or anything I could check/do, to solve this issue?
     
    Last edited: Jul 15, 2017
  20. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Unfortunately I don't have examples using asynchronous threads.
     
    Gustavo-Quiroz likes this.
  21. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Have you done the Linux setup procedure of ReadMe.pdf already?
     
  22. Morseliot

    Morseliot

    Joined:
    Jan 10, 2015
    Posts:
    46
    Thank you Enox, it helps, everything working now. Just one fix... in libraries list there is "libavcodecdev" , should be "libavcodec-dev". Make sure it's updated in readme file next time. Thanks a lot again!
     
    EnoxSoftware likes this.
  23. slava_pankratov

    slava_pankratov

    Joined:
    Jan 17, 2015
    Posts:
    42
    Hi! Could you provide low pass filter for face mask example? The mask shakes a lot. Thank you!
     
  24. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    There is no example of a low pass filter., but it is probably useful for stabilizing facial points.
     
  25. david-arcus

    david-arcus

    Joined:
    Jul 17, 2017
    Posts:
    9
    Hello, thank you for your blog post illustrating how to extract the shape predictor for the dog faces. In your example you're manually specifying the bounding box around the dogs faces. Can you advise on how to get the bounding box for the object detector, so it works automatically?
     
  26. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Object detectors for dogs are not distributed by the author of dlib, so you need to train object detectors for dogs.
    For details, please refer to Q & A of ReadMe.pdf.
     
  27. david-arcus

    david-arcus

    Joined:
    Jul 17, 2017
    Posts:
    9
    Ok cool, thanks. I note that the readme points to the svm HOG based trainer provided by the dlib author - can I use the dnn_mmod trainer instead (found here)? Will I be able to use this? Cheers
     
    Last edited: Oct 17, 2017
  28. hukamabid

    hukamabid

    Joined:
    Oct 16, 2017
    Posts:
    1
    Hi, I see Dlib FaceLandmark Detector is deprecated, do you have any explanation?
     
  29. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    DlibFaceLandmarkDetector supports HOG based ObjectDetector only.
     
  30. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    I realized that commercial use of dataset used to train "shape_predictor_68 _ face_landmarks.dat" is prohibited. Therefore, I temporarily stop publishing assets.
    "shape_predictor_68 _ face_landmarks.dat" is the trained model file published by the author of dlib.
    http://dlib.net/face_landmark_detection.py.html
    # You can get the trained model file from:
    # http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2.
    # Note that the license for the iBUG 300-W dataset excludes commercial use.
    # So you should contact Imperial College London to find out if it's OK for
    # you to use this model file in a commercial product.

    In the future, I plan to resume publishing assets by changing to a trained model created using another dataset or by excluding a trained model.
     
    Gustavo-Quiroz likes this.
  31. Gustavo-Quiroz

    Gustavo-Quiroz

    Joined:
    Jul 26, 2013
    Posts:
    36
    Thanks for the clarification.
    Is there any way (Maybe a repo) that we could help you create a new dataset that can work with all kind of angles?
     
  32. SengitalLtd

    SengitalLtd

    Joined:
    Jan 20, 2015
    Posts:
    2
    I am a noob to unity and I have no idea how to select SSE4 and AVX through the plugin. Is it automatically done in examples? How can I select the compiler optimizations?
     
  33. ZahCraft

    ZahCraft

    Joined:
    Jul 18, 2017
    Posts:
    1
    Hey, i've tried the WebCamTextureFaceMaskExample and also tried adding masks of my own but the faceLandmarkDetector returns result = 0.
    What am i missing?
     
    Last edited: Oct 31, 2017
  34. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    If you want to use the native plugin file enabled SSE4 or AVX compiler option, please replace the native plugin file in the “DlibFaceLandmarkDetector/Plugin/” folder with the “DlibFaceLandmarkDetector/Extra/” folder. And, select MenuItem[Tools/Dlib FaceLandmark Detector/Set Plugin Import Settings].
    extra.PNG
     
  35. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    How did you add the mask? Could you tell me the steps to reproduce it?
     
  36. Manuj

    Manuj

    Joined:
    Aug 2, 2016
    Posts:
    2
    Hi,

    I am using Unity 2017.2 with latest versions of the facemask example from github along with Dlib FaceLandmark Detector and OpenCV for Unity (versions as of the writing of this post.)

    I see in the WebCamTextureFaceMaskExample and also WebCamTextureFaceMaskAdditionalExample

    and also the facemask example android app on the google play store

    the mask is very jittery and vibrating

    Its not as stable as
    (I randomly found this video on youtube)

    Any tips on how to make the mask less jittery? Thans. (Let me know if you need a video of the jitter)

    Thanks in advance.
     
  37. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Could you replace with these new files?
    (Deleted)https://www.dropbox.com/s/lfhnluyypgo2ley/sp_human_face_68.dat?dl=0
    (Deleted)https://www.dropbox.com/s/n41soy75rr3wk92/sp_human_face_68_for_mobile.dat?dl=0

    These files will be added to Dlib FaceLandmark Detector v1.1.7.
     
    Last edited: Nov 11, 2017
  38. dovetorabbit

    dovetorabbit

    Joined:
    Nov 8, 2017
    Posts:
    1
    Hello .
    ExampleDataSet.cs -> landmarkPoints

    How do you know the coordinates panda Vector2

    thank~
     
  39. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    If you add a face that can not be automatically detected with a face detector like a panda or an animated face, you need to manually add landmark point data.
    dlibfacepoint.png
     
  40. ArrowT

    ArrowT

    Joined:
    May 18, 2013
    Posts:
    27
    Hi!
    When using Android, I get a very low FPS, how can I optimize it?
     
  41. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
  42. barcode802

    barcode802

    Joined:
    Apr 6, 2014
    Posts:
    26
    @EnoxSoftware - how easy would it be to map the landmarks to a custom 3d model - and real-time update the models shape - either to bones or blendshapes (eyebrow positions, eyelids open/close and mouth shape)? I looked through information on the various assets you have and couldn't see an example that.
     
  43. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Unfortunately, I do not have such an example.
     
  44. ArrowT

    ArrowT

    Joined:
    May 18, 2013
    Posts:
    27
  45. ArrowT

    ArrowT

    Joined:
    May 18, 2013
    Posts:
    27
    I found the problem in the "objdetect_CascadeClassifier_detectMultiScale_10" void in CascadeClassifier class. This most heavily loads the system, but I can not see the details, since this is part of the dll. I really hope for your help with optimization.
     
    Last edited: Nov 27, 2017
  46. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    I think that it is possible to optimize the balance between speed and accuracy by changing the parameters of detectMultiScale().
    https://stackoverflow.com/questions...values-for-opencv-detectmultiscale-parameters
     
  47. diegoe8

    diegoe8

    Joined:
    Aug 9, 2017
    Posts:
    2
    Hi,

    Is it possible to scale all the objects in ARHeadExample? I must see the cam in 768x1366 resolution, and when I scale objects the web cam picture dont match with the objects.

    Thanks
     
  48. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    Is it possible to take a screen shot of that state?
     
  49. ArrowT

    ArrowT

    Joined:
    May 18, 2013
    Posts:
    27
    How I can detect the face with a significant rotation of the device or turning the face?
     
  50. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,161
    DlibObjectDetector can detect face only from one direction.
    You need to rotate the pixels of the image using OpenCV.