Getting started with Vuforia Model Target - problems and questions

I'm just beginning here, so if I'm missing anything please point it out:

1) I have an .obj of a tool I want to use as a target. I open up the Vuforia Model Target Generator and import it; I've set it's position, it's size (note: even though I select 'inches' it's still displaying as meters in the view: ie Size: 0.790m x 0.065m x 0.140m -- would be better to display as inches, no?)

2. I set it as landcape and Set Detection Position

3. I then 'Generate Target' (note I've used the same model 3x to figure out this units display issue, and for some reason it has seen them as 3 distinct models, so I've only got 7 out of 10 left. Is there a way to delete some to free up during testing?


I switch to Unity. I add an ARCamera, I doubleclick the vuforia Test.unitypackage and import the model target. I set the database to my Test, my model target to my Test, it displays the size as it does above so I know it's loaded.

What is Guide View Mode? I've looked within the api / docs, couldn't find it but I'll look again.


If I click 'Add Model Target' it flips to my dev.vuforia page - I see two dbs I've got up there but both are empty. I've got one that's a cloud db but it's only allows Target Images. The device db allows a 3d object but only ones uploaded by the Vuforia Object Scanner. So I'm assuming I don't need either of these databases. How can I delete them if not needed? If I wanted a database of multiple objects and have an app recognize them as in the video from Vuforia here

how is that done? Docs, tutorial?

Lastly, once I run the above with the embedded model target I get no outline of the Tool target whether I'm using no guides, 2d guides, or 3d guides.

Also, much like the Unity Remote: Do I actually have to build to device to test this within an ios device or could I somehow select the ios device as the source camera rather than the Facetime Camera on my macbook? I realize this last part isn't Vuforia specific so if I've got to build to test, so be it. But I'm seeing NOTHING currently while running on my macbook and doubt I've even got this setup correctly.

Thanks for any help; appreciate it.

 

Guide View Mode allows you to generate the outline of the Model to guide users to the correct detection position.

IIRC, Databases are stored in StreamingAssets/Vuforia - you should be able to remove databases there.

The video you made a reference to is still in early access and not available to the general public yet. Once it's public, there will be more info available for it.

We currently don't have a Unity Remote type of option - what you can do instead is to attach a standard web camera to simulate a device to make development go a bit faster - if you want to see what it will look like on device, then you do need to deploy it.

HTH

 

Hey @vad710 - I'm trying a simple model target at this point using the Vuforia Model Generator on Windows. I've got my object / model in unity and trying to follow specifically to this tutorial: https://library.vuforia.com/content...olution/introduction-model-targets-unity.html

The example of the 'Database' section in Section 8 is not the same for me; where it shows checkboxes I've only got this:


Now I also see the 'databases will be automatically loaded if it's TrackingBehavior is enabled on scene load' -- perhaps that's the issue I'm having but I'm assuming the behavior IS on:



But once I run anything I see no target on my screen; in fact in order to get an ARCamera view it's almost like I have to hit pause for the Vuforia view to be triggered (will show video in a moment).


And yet I also see this:

 

Here's a dropbox link to video:

https://www.dropbox.com/s/wvj32w9v6u1gh0f/vuforia_model_target.mp4?dl=0

 

Hmm.. seems to be a failure to load PPM_Test dataset and that's why the pause on error; unsure why it won't load it though...

 

@grimmwerks What kind of license are you using?

 

Currently I think mine is still the free license -- my biz partner and I had spoken to John Iaia (jiaia@ptc.com) a few months ago about using vuforia for a medical device app; he had talked about all the elements of 7.3 and how that would work for us. He said he'd set me up for a dev license to test it all out. I've emailed him a few times and got no response so finally signed up with a free license just to test if this could work for us -- but so far it's not; I can't even find consistent docs for what I'm seeing in Unity.

I've signed up for the "Model Targets with Advanced Recognition Early Access Program" only yesterday. I *just* want to see this working / in action for a demo and then I can guarantee that we'd be on board with a few medical device clients.

 

Also JUST got a reply via facebook now -- was getting very frustrated as I was trying to email support, John, etc.

 

By free license do you mean leaving the license field blank, or a development license? When did you generate the development key if that's what you are using? Can you share the dataset you are using that is failing?

Is it correct that you are using Unity 2018.1 and Vuforia 7.1.34?

Also, you've linked some MTar information, this is a different feature from the commercial Model Targets and requires EAP. EAP comes with a VIP subscription. If you have questions about that and you are signed up you can use the VIP support system to get assistance.

 

Thanks for responding; I'll try to explain everything out.

I'm not leaving the vuforia license clear, I'm putting in the license I have, which is the free development key. The license key was generated March 6th, 2018.

I'm trying this with one Model Target that I used the Vuforia Model Target Generator tool on windows (using a cad object rather than scanning via android). I generated the dataset and imported it via custom import.

I'm using Vuforia 7.134 and Unity 2018.10f2 Personal but yet my gui doesn't match (as you see above) the example guis for utilzing a Model Target; I keep getting that it failed to load my dataset. Which is interesting as I've also attempted the VuforiaMars_ModelTarget example and it TOO fails to load it's dataset, so I'm unsure what I'm doing wrong.

Appreciate your response; really want this to work and happy to fork over licensing if it does; but it's been a lot of radio silence thus far -- John finally responded today from an April 4th email.

 

Are you testing this in a blank project or the Model Target sample? Can I see a screenshot of your scene hierarchy? Can you send me (via DM if preferred) a copy of the Unity Package for your dataset?

 

hello all
i've got the exact same problem !!!
Using vuforia 7.2.20 and Unity 2018.1.6f1
Using a new free developper License created yesterday
Working on Mac (iMac)

i've done exactly the same steps :
1) create target in Target Model Generator (TMG)
2) get the unity package and import it in Unity
3) add an ARCamera and a ModelTarget from Vuforia prefabs

after 3 tests and 3 models generated from TMG ...i've got always the same error :
Could not associate DataSetTrackableBehaviour 'mouse' - no matching Trackable found in DataSet!

you can find an image (capture screen) : hierarchy, inspector and console log


could you tell me what's wrong ? I'm working on it since two days, can't firgure out what's happen !!

 

Would you be able to share your dataset with me so I can attempt to reproduce it? Feel free to private message me the data so you don't have to share it publicly.

 

Yes i can share it, even publicy....since it's a test, the model is a free one
Thanks for you help

 

I'm seeing the same issue as well. Which version of the MTG did you use? Could you share the original model as well so I can generate it and test the full workflow?

 

Hello
again big thanks for helping and sorry for my poor english
i'm using the last MTG version, which is 7.2.20 launched from windows PC of course (is MAC version is on roadmap for MTG ?)
you can find the ZIP with the model i'm testing : it's a mac mouse model .
For the test i've loaded the FBX model into MTG...do you think this model could be the problem ?

Since i've got a dev licence with the limitation of 10 models generated from MTG, is there a solution to erase the 3 first tests i've done ?

waiting for you help, thanks
regards
alto971

 

It looks like there might be an issue with extremely small size values with a Model Target. The dataset you sent me had the width at 0.05meters. I generated a dataset with this set to 0.5meters and 52meters and did no encounter the issue here. I've uploaded one of the ones I generated that did not encounter this error. Could you test it and see if you get the error?

I'd also like to mention that the model you are using will likely encounter difficulties detecting once this error is resolved. This is due to limited geometry. Certain guideviews may still work however, but you'd need to experiment. This article has more information: https://library.vuforia.com/articles/Solution/model-targets-user-guide.html

Outside of upgrading to a VIP account, there is no way to get more target generations with the MTG.

 

ok thank you very much...but it sounds like Vuforia Model Target can't track little objects ?? As i can't use small size like 5 cm ... isn't ?
I will test with biggest size thuesday, i'll tell you if it's works

I know the geometry of the mouse's fbx is not really perfect, but thank you for the warning.

About MTG, so i can't delete the ones i've already done ?
Finally about the VIP subscription , which is the price, we can't see any information about it on the website...

Have a good day
regards

 

Hello,

The Model Targets feature itself has no limitation on object size. The biggest challenge we've experienced with smaller targets is with the device's ability do macro (close up) focus without the OEM's continuous autofocus feature "hunting" or "pumping". Those camera features are implemented by the device OEM, ARKit or ARCore - Vuforia abstracts those calls via our APIs and calls them natively.

Thanks,
Vuforia Engine Support

 

Hi, is anyone encountered an offset between the model target coordinates and the Hololens coordinate (my holograms having a big shift from where they supposed to be (after target acquire)

Thanks, Ariel

 

Hi Ariel,

Which version of Unity and Vuforia are you using? Do you see this with the sample project as well?

 

Hello,

I am having the same issue as Ariel.
When the model is recognised in the deployed app (HoloLens), it appears shifted to the right.

I am using Unity 2017.4.8f1 with Vuforia 7.0.57.

Thanks!
Inigo

 

@arielfel @InigoGregorio

Besides the offset are you by chance notice instability issues with the hololens? I am also experiencing the shift to the right but i am also noticing that the generated model position is always quite jumpy. Either of you notice that with yours as well?

Using: Unity 2018.2.5f1
Vurforia: 7.2.24

 

@arielfel @Jarediavionic

I might have solved the offset issue regarding my models (not sure this will work for everybody).

I have been working with my CAD models in Vuforia Model Target Generator.
It seems they are more "reliable" now because of the detection position.
If that position aligns with your real scenario, the tracking somehow becomes more accurate.
(I just use Guide View Mode 3D, the rest of check boxes in Model Target Behaviour are deactivated).

I followed this idea: "My hypothesis is that Model Targets get placed in the scene based on the rotation and position/distance you had to fill in for the 'detection position' in the Model Target Generator tool (MTG)."
Which can be find in this post: https://developer.vuforia.com/forum/model-targets/hololens-model-targets

I hope it will help you too.

 

My target package shows this warning：Json Node is null why？

 

Hi, could someone from Vufria team can explain how model target triggered?
It could be helpful if we had some guided lines instead of theories. Like if the 2D snapshot effects on target acquisition or it's just the properties of the CAD file with the materials and lightning.
Thanks, Ariel

 

Hello,

Here are some practical guidelines that you might find helpful.

Detection:

There are a handful of actions you can take to improve the detection of your 3D model when using the Model Targets Feature:

1. Correct scale of the model is important for detection and tracking. Confirm that the scale of your CAD model is an accurate representation of your physical model. We also recommend using meters as your unit of measurement.
2. A more detailed model should provide a better tracking experience. While higher poly counts can work, we recommend keeping the count below 100k for best performance.
3. Which elements to remove from your 3d model may depend upon the experience you're trying to create. For example, if you want to detect a car from an exterior vantage point, then the interior elements are not going to provide benefit to detection and tracking and as such you can consider removing them.
4. Connected or fused meshes do not necessarily improve performance.
5. Remove transparent geometry from you CAD data. For example, if you are trying to detect a car, it is recommended that you do not include any of the windows or lights in the CAD model that will be used for the Model Targets dataset.
6. Include texture differences in your CAD model. Using the car example again, if the car has a decal painted on it or has a segment, such as the hood, that is a different color from the rest of the model, make sure that is included in your CAD model. This difference in texture will be used in the detection of the target, although it is difficult to quantify how much this can improve performance.
   1. Additional note here: The current model targets is not designed to work with realistic-looking textures with many different colors. They work best with single-colored surfaces that contain logos, etc.
7. Use different colors on the model when there are different materials or colors on the real object. While Model Targets is designed to detect objects by shape, giving additional information on the materials will often greatly improve detection robustness.
   1. There is no need for model colors to match the colors of the real object - the specific color is not relevant, just the differences in colors.
   2. Colors can be used to indicate different parts with small gaps in-between that are strongly visible but are difficult to extract from a rendered view. One example is the area between the body and the door of a car.
   3. If possible, avoid single-color models and file formats that can’t transport color information such as STL.
8. Simplification tools, such as Simplygon, are able to process logos and transform them into geometry (edges). However, if you used Blender for simplification, it may destroy this info.
   
9. Model targets is designed to be used with CAD-like models that use only a few colors to indicate different materials. Please avoid photorealistic textures from game-like models, or photogrammetry scans. If you need to use photogrammetry scans, please remove the texture before importing into the MTG.
10. Try a different guideview for your target. If the guideview you are using doesn't contain enough of the geometry of your target, you may encounter issues detecting it. Often times a slightly angled view of the target will provide better results than a direct front view of the target.
    1. Recommended to avoid fronto-parallel viewpoints and choose diagonal views for detection, where possible.
11. There are a few of different approaches to a different guideview you could try that may result in better performance:
    1. Try zooming in on the section of the object with the most geometry for your guideview, while keeping the rest of the model out of the view of the guideview. This will allow detection to be based off a smaller portion of the Model Target that is rich in geometry, but also will keep the rest of the model present. While moving back to see the rest of the model you will begin mapping the surroundings and likely end up with a better tracking experience.
       
    2. Another potential guideview would be closer, but at an angle from the front-left or front-right of the object. This will allow you to get the full object in view, but also start closer and allow more tracker mapping as you move around the object.
    3. Avoid highly cluttered areas on your object as detection views, such as areas with heavy wiring or highly repeated structures. Choose a view that has sufficient distinctive geometry without too many small details. It is not necessary to see the whole object for detection, but you can zoom in onto a suitable area if that is feasible for your application.

Note: Material differences are used for Model Target detection, but they do not appear in the guide view. Thus, the guide view is not the final indicator for object geometry

Tracking:

There are a handful of actions you can take to improve the tracking of your 3D model when using the Model Targets Feature:

1. Ensure that you are using an officially supported device from this list: https://library.vuforia.com/content.../Solution/model-target-supported-devices.html. In order for a 3D object to be tracked when the camera moves away from the Model Target (e.g. Extended Tracking), the device needs to have an embedded IMU (gyro+accelerometer), and have been calibrated by Vuforia.
2. Try moving the device in closer to the Model Target after detection, and then moving around a bit more to allow Vuforia to do additional environment mapping. This should help with stabilizing the tracking.
3. Model Targets tracking extracts surface features, taken from the object being detected, to improve robustness. For example, if your object is all the same color, feature extraction will be suboptimal and tracking robustness may be degraded.
4. The scale defined in the MTG should match your physical model as closely as possible. This data is used in the extended tracking of Model Targets and incorrect values here will likely causing drift and poor tracking.

Extended Tracking will not work well for smaller objects (<= 2cm) as this may trigger autofocus 'pumping' or 'hunting' due to device camera working in macro mode.

Model Target tracking performance can differ from other Vuforia features, specifically with the robustness of the tracking experience when moving around the target in hemispherical motions. This is in part because it uses a combination of visual features and IMU data to continue tracking, so moving the camera to a different side of the model (with different visual features) can result in the tracking being lost.

Once tracking is lost, it can only be restored by moving the camera view back to the guide view position for detection.

We continue to work to improve the Model Targets feature, including tracking robustness improvements.

Thanks,
Vuforia Engine Support

 

[QUOTE =“grimmwerks，post：3503792，member：13661”]嘿[USER = 510724] @ vad710 [/ USER] - 我正在尝试使用Windows上的Vuforia模型生成器进行简单的模型目标。我已经将我的对象/模型统一起来并试图专门遵循本教程：https： //library.vuforia.com/content/vuforia-library/en/articles/Solution/introduction-model-targets-unity.html

第8节中“数据库”部分的示例对我来说并不相同; 它显示复选框我只有这个：
[ATTACH =满] 279445 [/ ATTACH]

现在我也看到'如果在场景加载时启用了TrackingBehavior，'将自动加载数据库' - 也许这就是我遇到的问题，但我假设行为是：

[ATTACH =满] 279446 [/ ATTACH]

但是一旦我运行任何东西，我的屏幕上看不到任何目标; 事实上，为了获得一个ARCamera视图，我几乎必须暂停才能触发Vuforia视图（稍后将显示视频）。


然而我也看到了这个：
[ATTACH =满] 279447 [/ ATTACH] [/ QUOTE]
你好，你是怎么解决的？

 

My function as a model target for vuforia also encountered this problem. How did you solve it in the end? I have encountered such a problem.

 

QQ：1760004035

 

@meedabit ,@Vuforia-Strasza hello, can you help me please?
I added an object to MTG and then I imported it into the database, DL scanned it, then I added the base as a unity, created a target model and see all the same errors

my object is egg)
Unity version 2018.1.7f1 and Vuforia 7.2.24
Using a free developer License
Working on Windows

 

@meedabit ,@Vuforia-Strasza new problem (
help pls

 

Hello @Seleznyov,

Which version of the MTG are you using? Unfortunately, we had to break backwards compatibility between MTG device databases and the Engine SDK somewhere between the 7.2 and 7.5 versions.

Suggestion is that you download the most recent version of the Unity Editor (2018.3.3 with Vuforia Engine 8.0.10), redownload the Core Samples from the Asset Store, and use the latest 8.0 version of the MTG. This will guarantee compatibility of the device database and SDK.

Thanks,
Vuforia Engine Support

 

@meedabit thank you , works

 

@meedabit with your suggestion I resolve the problem of loading a dataset
...but I have a new problem!
I want to debug the program on my Hololens, and I can do it.
But when opening the app on the Hololens it doesn't work: I only see a black scene
and Unity seems not to start...
are there any incompatibility between the Unity 2018.3.3 and Hololens??
Thank you for the help!

 

Hello @bibuzzola,

We are not aware of any compatibility issues with Unity 2018.3 and HoloLens.

Thanks,
Vuforia Engine Support

 

hello... if i add multiple guide views in the model target generator. can it detect those views simultaneously or one at a time????? i mean.....if the user is not looking in an exact direction in one view can it detect one of the other view in list
@meedabit

 

Hello,

Need to clarify that there are two (2) different types of Model Targets now: "trained" and "untrained".

Trained Model Targets are ones where the Model Target Generator project is uploaded to Vuforia web services for machine learning. When creating this type of Model Target, you still define guide views. However, the machine learning aspects of the device database can recognize when the object's side matches a guide view. Then, the guide view is auto-displayed so that the user can align and initiate tracking.

Untrained Model Targets only have the static guide views as defined during the authoring process. These have no machine learning, so app logic must be created to determine which guide view to show (and when). Note that Vuforia Core samples (on the Unity Asset Store) will scroll through the guide views each time you tap on the Mars Lander icon. This is just an example implementation but the code is there for you to freely review.

Finally, and to answer your question, the guide view (trained or untrained) must be aligned with the object to initiate tracking. Thus, if you present the right-side guide view but are facing the left-side of the object, you will not detect it.

Thanks,
Vuforia Engine Support

 

Thanks for your info @meedabit ... i would also like to know that do we still need to come into the initial guide view position to detect again after a tracking lost event happened

 

Hello,

The current version of Vuforia Model Targets (8.1 SDK and below) requires that the object view be repositioned to align with the guide view after tracking is lost.

We continue to work to improve the feature, adding features that will simplify the user experience.

Thanks,
Vuforia Engine Support

 

I'm also facing the same problem with the error "Failed to load data set" and I tried to resolve it by increasing the size of my Model by 10 times as per your above explanation and i still get the same error
I would be thankful to you if you kindly respond regarding this..

Thanks,

Kalpana Shankhwar

 

Hello @kalpana1,

Normally, this type of failure is license related. Be sure that your Unity Project has a Vuforia Engine Developer license (free, with watermark), or Pro license (paid, no watermark):

Unity Editor->Window->Vuforia Configuration->Global->App License Key

Thanks,
Vuforia Engine Support

 

Thankyou very much for responding!! I added the License Key from the Vuforia engine developer portal. but I still get the error "Failed to load dataset". I'm extremely upset with this and trying to find all possible ways to sort this out. Please take a look in the attached file if I'm doing anything wrong.

Thanks a lot!

 

Hi there,
I just "puchased" a new license and did all the Steps from the Tutorial.

I get the same Error Message on pushing "play".
"Failed to load Dataset"

Here is the related call stack:

Failed to load data set Toyota Corolla.
UnityEngine.Debug:LogError(Object)
Vuforia.DatabaseLoadARController:LoadDatasets()
Vuforia.VuforiaARController:StartAfterInitializationCompleted()
Vuforia.VuforiaARController:UpdateState(Boolean, Boolean)
Vuforia.DigitalEyewearARController:Update()
System.DelegateynamicInvoke(Object[])
Vuforia.DelegateHelper:InvokeDelegate(Delegate, Object[])
Vuforia.DelegateHelper:InvokeWithExceptionHandling(Action)
Vuforia.VuforiaBehaviour:Update()

Can you point out what else we can try please?

 

Hello,

Vuforia Engine licenses enable features. If you're trying to load a device database for a feature that is not permitted for a license then it will fail to load.

The most evident case I can think of is trying to use a Classic license with a Model Target device database. The Classic license does not have permissions to load a Model Target device database, so it will fail to load.

You can easily test this by swapping your paid license for a free Development license. If the device database loads with the Development license then it is most likely you're using the wrong license for the feature.

https://developer.vuforia.com/vui/pricing

Thanks,
Vuforia Engine Support

 

Hi,

I want to implement an ar application having component identification, inspection and navigation for the user to move accordingly please share how to implement this as i am beginner in ar.

For reference -

Is it possible through vuforia sdk if possible how can i use a
1.3d model as target with guidance on its parts like KTM Bike service instruction provided by vuforia http://site-654059.bcvp0rtal.com/de.../ktm-bike-model-target-:-service-instructions, is there any sample code available for it?
2.track camera centre position and activate target when centre position maps to it

please assist as i am new to this environment just starting in ar development. I already searched a lot but not getting any positive results.Any help will be appreciated thanks.

 

@meedabit please tell me if it possible to make a universal person model, to work with all people, to make something like virtual fitting room based on device and mirror?

 

Hi @Marek_Bakalarczuk
Model Target has been designed to use and track object.
"A wide variety of objects can be used as Model Targets, from home appliances and toys, to vehicles, to large-scale industrial equipment and even architectural landmarks."
I don't believe your use-case can be achieved by using Model Targets.

More info on MT here: https://library.vuforia.com/features/objects/model-targets.html

Thank you.
Vuforia Engine Support

 

@Sika_MC my daughter is recognised in app, and some sort of pictures. I'm digging further to it