Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Arcolib - Augmented Reality Components

Discussion in 'Assets and Asset Store' started by yezaldivar, Jan 20, 2017.

  1. Zendist

    Zendist

    Joined:
    Oct 29, 2012
    Posts:
    5
    Hi @yezaldivar

    We're currently working on a project and would like to prototype a minified AR version of it. I have two questions before we consider buying your package:
    1. We load models at runtime rather than at build time (the users inject their own models into our application), would this also be possible with your asset?
    2. Can we get your asset to load models by looking at QR codes generated outside of the application?
    Thank you for your response :) - your asset looks great!

    Best Regards,
    Andreas
     
  2. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Agree with you! Actually I am redesigning the trackers inspector to allow more custom behaviors like this. Take a look to the new inspector for trackers (not finished yet).
     

    Attached Files:

    marcipw and theolagendijk like this.
  3. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Thank you for being interested in the library. We provide generic components to handle the detection (with static prefab loading), but with a few simple modifications you can load models when markers or images are detected. To load dynamic models or scenes you must modify the way to load your prefabs, by using AssetBundle instead of GameObject.Instantiate.

    https://docs.unity3d.com/ScriptReference/AssetBundle.LoadFromFileAsync.html

    Each detected symbols has a unique ID, you can match the detected codes with your models. Qr detection is almost ready (not available in the current public version) QR codes can be generated by the Unity Editor or from any other external tool but it must be surrounded by a bounding rectangle to be uniform with other barcode types. Qr code should look like the image attached to be detected, the good news is that you can keep them tracked together with other marker types and you are also able to track multiple Qr at the same time with a good performance. This will be available in the next release.
     

    Attached Files:

    • qr.png
      qr.png
      File size:
      12 KB
      Views:
      1,261
    Last edited: Apr 6, 2017
    marcipw likes this.
  4. Zendist

    Zendist

    Joined:
    Oct 29, 2012
    Posts:
    5
    Hi again, thank you for your swift response! I have another question to clarify :).

    What if our objects never become prefabs, but are just meshes loaded at runtime? Do you do some sort of in-editor processing that is required before objects can be viewed in AR? Cause that would be an issue for us :).
     
  5. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi!
    When trackers report a symbol as tracked or detected it comes with the estimated 3D pose in world coordinates respectively to the camera, the transform is represented in a Matrix4x4 but you can easily get it as Unity Transform using
    Code (CSharp):
    1.  
    2. Symbol.ApplyTransformMatrixTo(Transform transform, bool applyOrientation = true)
    3.  
    Any object in the Unity scene that support transformations thru Matrix4x4 or Transform can be an augmented layer, regardless the method you have used to load it. We just apply the proper transformation to each object. The integration source code is available inside the package so you can modify it as you needed.
     
  6. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    65
    Hi yezaldivar, really liking arcolib so far. Thanks! Had a question regarding iOS builds. While building i saw a bunch of warnings that libarco was compiled for iOS 10.1 (I was targeting 9+ at the time). Is that the minimum version I can target with arcolib?

    EDIT: Additionally, I had to write a postprocess script to disable bitcode on iOS builds for it to compile properly. Is that intended?
     
    Last edited: Apr 7, 2017
  7. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi!
    I can try to compile Arcolib for lower versions on iOS. Normally the iOS apps will compile just running the Xcode project generated by Unity. Some warnings can arise depending the version of Unity you are using, but no needs to create postscripts, maybe you are unchecking bitcode on your project but the library was compiled with that. I will fix this if that is the reason. The next version will be more optimized. In iOS, the libarco is heavy because includes all platform binaries in it (fat static lib).
    For mobile devices don’t forget to use NativeCamera when export your projects. It will provide faster camera previews.
     
    Last edited: Apr 8, 2017
  8. TobiasW

    TobiasW

    Joined:
    Jun 18, 2011
    Posts:
    91
    Hi yezaldivar! I have a question regarding the feature tracker. Let's say I want to track a DinA4 image. Can the tracking already start once the camera sees a part of the image, and will it continue to track as the user moves the camera over the image, always only seeing a part of the whole? If this would work, are there any restrictions for this kind of tracking?
     
  9. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi!
    Yes, it is possible to detect and image from just a portion of it and later continue the tracking if the image is not too wide or the camera movement is not too fast. With the new tracking algorithm (will be available soon) it applies a continuous detection and correction so this variant of tracking works better. The other approach is to split the image as multiple targets to be found separately and use all of them as a single group (knowing their relative space distribution). You just have to average the quaternions of all symbols detected (the parts of the image) and average the position with their offset. That's a better way to do this but is not implemented in the library by default. I will implement a sort of SymbolGroup to achieve something like this in the future.
     
    Flavelius likes this.
  10. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    65
    I was actually using Unity's cloud build system (just for convenience at the moment) and it threw those errors.
     
  11. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    65
    Had one more question. On our iOS builds, when someone first loads the application and gets the camera permission prompt, the camera does not work once they accept. However, if they completely close the application and then launch it again, it works. It seems like the initial lack of permission is never re-checked and it just stays in the failed state. How should that be handled? Is there an event or something I can subscribe to that would give me an opportunity to re-initialize the library?
     
  12. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi!
    Did you set the description for camera when you export the ios project? This will create the proper permission on the Xcode plist file. If you are using the native camera capture the module loads the camera (using Objective C API) when permission requested is accepted. There is no need to listen for callback because it blocks until you accept the dialog. See the picture attached for configuration on Unity. I am not having that issue on my devices but I will try to figure out whats going on. If you continue with the problem after applying the description please just let me know.
     

    Attached Files:

  13. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    65
    My settings match yours (with different strings in the description and bundle ID sections). Being built with Unity 5.6.0f3 and unity's cloud build system
     
  14. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    65
    Sorry to be a pain and ask so many questions, but i had one more regarding auto rotate. If i set the project to "landscape left" instead of auto rotate, the application itself stays locked in landscape. However, the camera view still rotates on android (iOS doesn't appear to do this). If i set the application to auto rotate, android works fine, but on iOS the camera output sometimes flips upside down when the orientation of the app is still right side up.

    EDIT: Additionally, I did another test with the accept dialog today and it worked just fine. Not sure what happened before.
     
  15. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Don’t worry its ok! All your questions are interesting for me, so shoot anytime you have doubts. The issue you are reporting is probably a valid problem because the NativeCamera gets the device orientation from it native APIs. But for Android I use
    https://developer.android.com/reference/android/view/Display.html
    getRotation()
    That value should be synchronized with the display orientation. I tested the app in all my devices and it works fine (with auto or fixed rotation modes). Please can you tell me what Android version you are using and the devices you use? Can you try it in other device to see if that happens?

    Edit:
    Can you export your app to Android project to see if Unity is updating the manifest.xml file?
     
    Last edited: Apr 13, 2017
  16. detomato

    detomato

    Joined:
    Aug 16, 2013
    Posts:
    54
    @yezaldivar any plan of releasing of a free/developer version so everyone can test it out.
    hope someone can make a comparison with other mainstream AR kit out there.
     
  17. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    I think the price of Arcolib is really, really fair compared to most of the AR kits out there. I don't think @yezaldivar should release a free version.

    I will post some videos soon and will have a few apps live on IOS and Android so you can see a few of my experiments in action and test them out.
     
  18. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    Hello,

    is it possible to load target images at runtime?
     
  19. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Yes, Actually there is a demo scene in the package called RuntimeModelGeneration that is exactly what you are asking for. This feature allows you to create runtime targets (models for Arcolib) from camera feed, static images, or any source convertible to a Unity Texture.
     
    theolagendijk likes this.
  20. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Arcolib is pretty young nowadays. I am not sure if it will be free or open-source someday, but I can guarantee that it will always be under the "only one fair fee" model forever. Some of the tools included in it has taken us weeks of research and coding so the current price I think is fair for that hard work. I don’t really want to create free versions with watermarks or restrictions. Arcolib is beautiful because it doesn't have any restrictions for use, just the unique lifetime price...
     
    Last edited: Apr 20, 2017
    theolagendijk likes this.
  21. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Thanks Marc!
    We will love that demo apps. I am eagerly waiting to see what other developers and designers can do with the library, I am sure good things can be done.
     
    theolagendijk likes this.
  22. attilavarga

    attilavarga

    Joined:
    Feb 3, 2017
    Posts:
    1
    Hi there! I am looking for Windows PC, iOS, Android and Hololens support and you mentioned Hololens before in one of your answers. Can you confirm that we can use Arcolib with Hololens in the current version? If not, when will it be supported?

    best regards,
    Attila
     
  23. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi, Arcolib is structurally compatible with any x86, x64, ARM architecture (currently only for Windows, MacOS, iOS, Android). But for Hololens OS (running Windows 10 arch x86) some modifications should be made to be completely usable from Unity. I will publish a post when the library is ready to be used with Hololens without any additional changes.
     
    theolagendijk likes this.
  24. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    Excellent! What would be the best way to get Arcolib working with Google cardboard and GearVR?
     
    ethanwgross likes this.
  25. ethanwgross

    ethanwgross

    Joined:
    Jun 25, 2014
    Posts:
    83
    Just purchased, could not resist getting on board with this. I am really excited to get working with this game changing asset. Thanks for the amazing price!
     
    yezaldivar and marcipw like this.
  26. eyeofhorus

    eyeofhorus

    Joined:
    Oct 10, 2013
    Posts:
    10
  27. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    I replaced Vuforia with Arcolib in a project today and I am so pleased with the results. This asset really is such amazing value for money! If you want to work on AR projects, this is one of the best purchases you can make.
     
    Gustavo-Quiroz and yezaldivar like this.
  28. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Arcolib will have an special kind of ArcoCameraRig (StereoCameraRig just drag it to the scene and thats all!) to properly deal with stereo vision apps. That will be available in coming versions. But modifying the Cardboard SDK examples you can also get it working. I am making the list of all features requested by the community and sorting them by priority (I dont want to miss anything). The next release has a lot of improvements (new Model Editor, behaviors sugested by @theolagendijk ,QR codes detection and tracking sugested by @JoeStrout, automatic dealing with wide images @mrmdesign and other surprises) I am sure you will love it.
     
  29. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    That sounds amazing. I can not wait to get my hands on the new release.
    Thank you for listening to the community and implementing some of our suggestions.

    The drag and drop cardboard camera rig sounds like a great idea. You are going to make it too easy for us. ;-)
     
    theolagendijk likes this.
  30. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    ... on another note, I wonder if anyone from Unity is watching this thread. The Vision Summit is next week and it would be great to see this asset showcased to the industry. @yezaldivar certainly deserves a ton of new sales. :)
     
    ina, yezaldivar, Malbers and 3 others like this.
  31. ethanwgross

    ethanwgross

    Joined:
    Jun 25, 2014
    Posts:
    83
    agreed.
     
    yezaldivar, Malbers and theolagendijk like this.
  32. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    The frame rates appear to be very bad in your demo - do you have any benchmarks for iOS - compared to Vuforia?
    How many image targets do you support, do you support switching tracker databases, etc?
    I would be eager to switch / try - but difficult due to projects with deep integrations with Vuforia.
     
  33. neuston

    neuston

    Joined:
    Feb 13, 2017
    Posts:
    5
    Thank you for the hard work. I have just brought and try to run your FeaturesTrackerDemo, but I cannot start working, since I encountered the following two problems:

    (1) The camera (ArcoCameraRig) is rendered as black screen. I am using laptop with windows 8.1, just choose the default laptop webcam. My unity version is 5.6.

    (2) Then I try to build and run in my Android phone, to see whether the camera will work in another platform, I got the following error: "Unable to convert classes to dex format" Detail of error message is pasted as follow.

    CommandInvokationFailure: Unable to convert classes into dex format.
    C:\Program Files\Java\jdk1.7.0_80\bin\java.exe -Xmx2048M -Dcom.android.sdkmanager.toolsdir="C:/Android/sdk\tools" -Dfile.encoding=UTF8 -jar "C:\Program Files\Unity\Editor\Data\PlaybackEngines\AndroidPlayer/Tools\sdktools.jar" -

    stderr[
    Exception in thread "main" java.lang.UnsupportedClassVersionError: com/android/dx/command/Main : Unsupported major.minor version 52.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:278)
    at SDKMain.main(SDKMain.java:127)
    ]
    stdout[

    ]
    exit code: 1
    UnityEditor.Android.Command.Run (System.Diagnostics.ProcessStartInfo psi, UnityEditor.Android.WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg)
    UnityEditor.Android.AndroidSDKTools.RunCommandInternal (System.String javaExe, System.String sdkToolsDir, System.String[] sdkToolCommand, Int32 memoryMB, System.String workingdir, UnityEditor.Android.WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg)
    UnityEditor.Android.AndroidSDKTools.RunCommandSafe (System.String javaExe, System.String sdkToolsDir, System.String[] sdkToolCommand, Int32 memoryMB, System.String workingdir, UnityEditor.Android.WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg)
    UnityEditor.HostView:OnGUI()
     
  34. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Please verify that your current export settings on Editor is different from WebGL, because sometimes it is the default configuration and it will request permission to start the camera.(check the console to see if some error arises). Also on ArcoCameraRig you can choose NativeCamera is faster and it avoids the web sandbox issue. Also just verify if your camera is not being used by other app.
    Please install jdk1.8.0_45 or higher. The Android native plugin (jar) was built with that version. Also there is a thread where you will find instruction for this Unity configuration issue.
    https://forum.unity3d.com/threads/unable-to-convert-classes-to-dex-format-android-issue.197061/

    Hope you can solve the problem, if not just write me.
     
  35. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi
    The frame rate is around the 24fps in the most of situations, it will be improved when I add some GPU implementations. I had not performed any assessments in comparison to Vuforia mainly because we are still working in the tracking algorithms and other modules (besides we don't want to be compared with Vuforia please). In theory you can add as many models (targets) as you want, but of course it will reduce the performance. Each tracker can analyze up to 10 Symbols simultaneously, if you need more just add two or more trackers with other models to detect and the same input, both can add symbols to the same augmentation layer this is a unique feature and only possible with Arcolib. Models can be added or excluded from analysis in real time, you can add a large database of models to the tracker, and only a few of them enabled, later you can turn on /turn off the models as you want in real-time using the enabled property on models. Arcolib uses different concepts but is easy to understand or integrate in your projects, it’s designed for simplicity and it provides separated tools to make it more flexible.
    Best regards
     
  36. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    I like the general philosophy of ArcoLib, but am trying to evaluate which project types would work.

    I think my question is more about recognizing and not about tracking at the same time... Usually, you are not tracking 10 symbols simultaneously... but it's common to have 10 different "markers" that load 10 different types of content. In such an example where the user is tracking on the camera at most 1 or 2 "markers", how many markers can it recognize?

    Are you taking into account camera intrinsics of popular cameras for your optimizations? I believe Vuforia is cheating a lot with that, as their iOS x64 performance is extremely good - but falls short for other platforms and random Android phones
     
  37. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    If you are using BarcodeModel (barcode markers) you can recognize unlimited amount of markers because the information is encoded in the markers no matching is required. In that case it depends on the kind of dictionary used by markers (Aruco : up 1023 markers, QR: all possible combination of 1024 characters which is a huge number of markers). Using real image recognition you can also use as many models (FeaturesModels) as you want but due it need to perform matching the response time in detection will be affected when you use more than 20. Internally it use a kdtree to speed up the search but features extraction and matching is computationally expensive in mobile devices. This can be improved using a GPU approach but currently all implementations are executed in the CPU using parallelized strategies. For camera projection I am using an intrinsic general calibration that fits to the most camera sensors, but not really using intrinsic instructions yet.
    Have a nice day!
     
    Malbers, ina and marcipw like this.
  38. LifeHackStudios

    LifeHackStudios

    Joined:
    Dec 7, 2016
    Posts:
    6
    Hey your asset looks awesome. I am looking to buy it. Just one quick question. Can it be use for Mixed reality? AR/VR? Thanks and keep up the good work!!!!
     
  39. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    @Mmfox A drag and drop Stereo camera is planned.
     
    LifeHackStudios likes this.
  40. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    If I were just scanning QR codes, I would just use the free zxing - so that problem is too simple for your package ;)

    I have a few extensive card games and painting galleries using image targets in Vuforia - currently these are at hundreds of different cards that can be tracked one at a time, but can be recognized randomly - relatively quickly. Vuforia rates these device image targets at up to 1000, and if you need more you can switch databases. I realize you are not trying to compare to Vuforia, but just to make sure I understand, it seems that in this case your system can only do up to 20 or so?

    I think performance on iOS will greatly improve if you use camera intrinsics for at least iPhone 5S+ (arm64) - the tracking reminds me of QCAR / Vuforia in 2011/2012 iOS before camera intrinsics. I hope you do look into this - because it will make your package much more powerful for the case of tracking very few targets.
     
  41. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    What if you could mix and match Vuforia with Arcolib? There are some existing prefab's already well-defined in Vuforia that could be shared?
     
    LifeHackStudios likes this.
  42. Gustavo-Quiroz

    Gustavo-Quiroz

    Joined:
    Jul 26, 2013
    Posts:
    38
    I don't think you could do much with that, Vuforia has a very closed system, they don't allow to get another Input sources (like a video for example), what you can do is to get the Image per frame already processed and send that to Arcolib but that doesn't make much sense since is already processed by Vuforia (By processed I mean that the system already tried to look for features or image targets on the texture), something that it could be useful mixing Vuforia with another plugin is for example add post effects to the image (See OpenCVforUnity, it already has a sample with Vuforia) or add VR with Google Cardboard.
     
    yezaldivar likes this.
  43. blacksun666

    blacksun666

    Joined:
    Dec 17, 2015
    Posts:
    214
    Can you add to you list of plans, the ability to create a 3D terrain or mesh by taking a number of pictures of a surface like a table complete with any objects sitting on the table?
     
  44. neuston

    neuston

    Joined:
    Feb 13, 2017
    Posts:
    5
    Everything work fine on my Android phone, after I installed the newly version of java sdk (1.8.0_131). However, the camera is still shown as black screen in my pc. My export setting on Editor is set to be pc, not webgl. It doesn't bother me atm because I am developing mobile app, not pc app.

    Besides, it seemly only work when the camera put closing to the mark. Is it possible to also track successfully in the distance?
     
  45. LifeHackStudios

    LifeHackStudios

    Joined:
    Dec 7, 2016
    Posts:
    6
    I finally bought Arcolib! AWESOME asset... so far.. I am sure with other updates this will be a very robust AR asset. In addition to all the common AR examples.. I was able to do depth masks and videos successfully! The second image below it's actually a video with transparent background and audio... and it WORKS! :)

    Untitled-1.gif Capture2.JPG
     
    Last edited: May 2, 2017
  46. deddygnwn14

    deddygnwn14

    Joined:
    May 2, 2017
    Posts:
    9
    Hi
    Look awsome!!
    But one question, does the plugin need internet connection to initialize like Vuforia & easyAR does?
    Because I need to create mobile ar apps without permission to internet at all...
    My client refuse any internet connection on the apps no matter what, for security reason.
     
  47. theolagendijk

    theolagendijk

    Joined:
    Nov 12, 2014
    Posts:
    117
    No internet connection required. It all works offline. Everything you need is in the app. No need for an internet connection to get the AR going.
     
    marcipw, Malbers and yezaldivar like this.
  48. yezaldivar

    yezaldivar

    Joined:
    Aug 24, 2016
    Posts:
    147
    Hi
    I will evaluate your request for future updates. For 3D reconstruction with non-depth cameras you need to use stereo disparity techniques, and probably two cameras are required. Thank you!
     
    blacksun666 likes this.
  49. deddygnwn14

    deddygnwn14

    Joined:
    May 2, 2017
    Posts:
    9
    Wow nice! thanks for the answer. Will buy this plugin soon!

    I'v been stressing this past month looking for a good & stable AR sdk that require no internet connection at all.

    For yezaldivar, please consider to make this plugin stay working offline in the future, because there are some company out there which having this kind of policy just like my client.

    Thanks!
     
    theolagendijk and marcipw like this.
  50. marcipw

    marcipw

    Joined:
    Apr 18, 2013
    Posts:
    239
    @deddygnwn14 You have definitely found the right AR sdk with Arcolib. I don't think there is anything else out there which comes with such great features and such a reasonable price tag. @yezaldivar has loads of great plans for future updates and his support is amazing.