Search Unity

[RELEASED] Dlib FaceLandmark Detector

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Jun 4, 2016.

  1. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564
    Unfortunately, there is no tool for placing points.
     
  2. Harko_yo

    Harko_yo

    Joined:
    Feb 25, 2017
    Posts:
    6
    Ok, we will continue moving them by hand then :p

    Another thing. Is the Quad resizing to the screen size? It seems to match the screen height but not the width and on mobile screens the sides are empty.
     
  3. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564
  4. Harko_yo

    Harko_yo

    Joined:
    Feb 25, 2017
    Posts:
    6
    I'm having some new funny problems.
    First, sometimes the ARObjects are not showed because they are flipped backwards and moved behind the camera. Going back to main menu and back to the mask scene may fix it, some times it takes a lot of tries others only one.

    And the second, the project will run on phones/tablets but after the builds the fps dropped to 10 fps.
    The scenes have masks and ARObjects for every face. I checked the code and find it calculates two times the faces (one for mask another for objects). Fixed that and now it go up to 20 fps.

    Before tried to add skipframe and downscale ratio. ARobjects give some problems with downscale when i try it on the tablet. Skipframe need a little more of working so the mask don´t blink when a frame is skipped.

    I need more ideas to make the fps go up a little more and a way to stop the ARobjects to flip and move backwards (this one is frustrating)
     
  5. CaptainPyFace

    CaptainPyFace

    Joined:
    Apr 12, 2013
    Posts:
    9
    Is it possible to improve dlib performance by scaling down the resolution of the Mat used for face detection without having the scale down video resolution output? Are there other ways to speed up dlib performance (e.g. skipping frames, etc.) without sacrificing accuracy. Thank you
     
    Last edited: Nov 18, 2023
  6. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564
    FrameOptimizationExample is an example of downScale and frameSkip to improve fps.
    https://github.com/EnoxSoftware/Dli...timizationExample/FrameOptimizationExample.cs
     
  7. hannagur

    hannagur

    Joined:
    Jan 22, 2020
    Posts:
    5
    Hello,
    I was wondering if you have a sample that shows how to place a mask just over the mouth. I am using a WebCamTexture.
    Thanks.
     
  8. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564
    Unfortunately, I do not have such an example. I do, however, have an example of a mask that covers the entire face.
    https://github.com/EnoxSoftware/FaceMaskExample
     
  9. hannagur

    hannagur

    Joined:
    Jan 22, 2020
    Posts:
    5
    Thank you so much!
    I have been looking at the ARHeadExample- I see it has some mouth-specific parts.
    I had a few questions:
    1. I see the 'mouth' GO is set to active or inactive in the Update function in ARHeadWebCamTextureExample.cs, but where is its position set?
    2. Also, when calling faceLandmarkDetector.DetectLandmark(detectionResult[0]), how can I specify how many points I want returned (68, 17, 6, 5)?
    3. Is there documentation somewhere about which parts of the face the points match up to?
    Thanks!
     
  10. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564
    1. You can change the position of the children of the ARObject in the inspector.
    change_dlib_object_1.png

    2. You can specify the dlibShapePredictorFile you wish to use when initializing the FaceLandmarkDetector class.
    https://github.com/EnoxSoftware/Dli...eadExample/ARHeadWebCamTextureExample.cs#L354

    3.
    https://github.com/EnoxSoftware/Dli...er/Assets/DlibFaceLandmarkDetector/ReadMe.pdf
    landmark_points_6.png landmark_points_17.png landmark_points_68.png
     
  11. hannagur

    hannagur

    Joined:
    Jan 22, 2020
    Posts:
    5
    Thank you.
    What is the difference between the WebCamTextureFaceMaskExample and the WebCamTextureFaceMaskAdditionalExample?
    Also, could you give me some guidance as to how to modify the code in this example to put a mask just over the mouth, and not over the whole face?
    Thanks!
     
    Last edited: Dec 26, 2023
  12. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,564

    WebCamTextureFaceMaskAdditionalExample is an example of a simple extension of the polygons in the forehead area of a face mask based on the estimated facial organ points. This example was created in response to a request to cover the forehead with a mask.


    One example of how to display a mask only around the mouth is to create a mask image created with a transparent PNG.
    FaceMaskExample_mouth.PNG test_face_mask1.png
    However, the drawback of this method is that it causes distortion of the image.