Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Question how to build AR application for necklace try-on using face detector AR foundation

Discussion in 'AR' started by rureeri24, Mar 3, 2024.

  1. rureeri24

    rureeri24

    Joined:
    Feb 7, 2024
    Posts:
    4
    Need help with placing the necklace object at the appropriate location but unable to do so.
    I've been following the AR Unity learning course that builds an application for glasses.
    Any suggestion as to how should I proceed?
     
  2. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,178
    This is probably going to be a somewhat challenging app to design with respect to the capabilities of today's mobile AR platforms.

    You would probably proceed in one of these two ways (or try both and see which works better for you):

    1. Use face tracking in the user-facing camera to get a face position, and place the necklace on the user by calculating an offset from the face position.

    2. Use body tracking in the rear-facing camera to get a body position, and place the necklace on the user by calculating an offset from a known body joint.

    We have a samples app that you can use as a getting started point that includes demos of both of these features: https://github.com/Unity-Technologies/arfoundation-samples
     
  3. rureeri24

    rureeri24

    Joined:
    Feb 7, 2024
    Posts:
    4
    Well I am trying the first approach you've mentioned, using the AR face manager, if I place my necklace prefab in the face prefab field, it appears on the face of the user instead of the neck.
    i did write this script below

    using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    using UnityEngine.XR.ARFoundation;
    using UnityEngine.XR.ARSubsystems;

    public class NecklacePlacement : MonoBehaviour
    {
    [SerializeField]
    private ARFaceManager arFaceManager;

    [SerializeField]
    private ARAnchorManager arAnchorManager;

    [SerializeField]
    private GameObject necklacePrefab; // Reference to your necklace prefab

    private void Start()
    {
    if (arFaceManager == null || arAnchorManager == null || necklacePrefab == null)
    {
    Debug.LogError("Missing required components or references!");
    return;
    }

    // Subscribe to the AR Face Manager events
    arFaceManager.facesChanged += OnFacesChanged;
    }

    private void OnFacesChanged(ARFacesChangedEventArgs args)
    {
    foreach (var face in args.added)
    {
    // Create an anchor at the detected face's position
    CreateNecklaceAnchor(face.transform.position, face.transform.rotation);
    }
    }

    private void CreateNecklaceAnchor(Vector3 position, Quaternion rotation)
    {
    // Adjust the offset to move the necklace down from the face to the neck
    Vector3 offset = new Vector3(0f, -1.5f, 4f);
    position += offset;
    // Create an anchor at the detected neck position using AddComponent<ARAnchor>()
    // ARAnchor anchor = arAnchorManager.AddAnchor(new Pose(position, rotation));
    ARAnchor anchor = arAnchorManager.gameObject.AddComponent<ARAnchor>();
    anchor.transform.position = position;
    anchor.transform.rotation = rotation;
    Debug.Log("Creating necklace anchor at position: " + position + ", rotation: " + rotation);

    // Instantiate and attach the necklace prefab to the anchor
    GameObject necklaceInstance = Instantiate(necklacePrefab, anchor.transform.position, anchor.transform.rotation);
    necklaceInstance.transform.SetParent(anchor.transform);
    }
    }

    But the necklace prefab doesn't appear on the face.
    Is there an approach where using the AR face manager, i try to add some ar default face in face prefab field and then position these necklaces?
    what should i do?
    Kindly help.
     
  4. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,178
    You would have to use either the face position or the vertices of the face mesh to see if there is way to calculate an offset that works for you.
     
  5. rureeri24

    rureeri24

    Joined:
    Feb 7, 2024
    Posts:
    4
    I'm done with placing the necklaces by using an ar default face and then just facing the necklace model.
    In my UI canvas, I have buttons that represent each necklace, so I want to render that particular necklace only when it clicked on, how can I achieve that?
     
  6. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,178
    Unity employees may not be able to help you with general app development questions in the forums. UI and buttons work the same in AR as they do in non-AR projects, so I recommend that you seek more information in places such as Unity Learn or YouTube to understand how to build UI in Unity.
     
  7. rureeri24

    rureeri24

    Joined:
    Feb 7, 2024
    Posts:
    4
    Okay thanks
     
    andyb-unity likes this.
  8. yashmangrola00

    yashmangrola00

    Joined:
    Jan 23, 2024
    Posts:
    1
    My English is not well enough to explain perfectly but I try my best
    Unity version 2022.3.19f1
    using 3D(URP) Core

    @andyb-unity When I am using a face mesh to a put 3D necklace on a neck in unity I test on a necklace in my Android phone so the necklace will not be perfectly stable on the neck. Necklace automatically move means high and width will automatically change.
    How to fix these issues

    one more question for you
    without a mesh I can do it 3D necklace on face perfectly
     
    Last edited: Mar 27, 2024
  9. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,178
    Unity employees cannot give you development advice outside the domain of Unity API's. I'm happy to help if you have questions about AR Foundation. It is your responsibility to design and build solutions for your use case.