Mobile AR: Qualcomm vs String

Have you put libstring.a in a folder called plugins and do you have an image called "Marker 1" in a folder called StreamingAssets ?

 

The only libString.a files I have are labeled "libStringOGLDemo-1.0.a" and "libStringUnityDemo-1.0.a".

Putting either, both, and renaming either to libString.a in the Plugins directory still gives me that same error log.

I'm going to try installing a demo of the latest version of Unity on another box along with the String demo and see if that fixes it.

Appreciate the help, still no dice so far.

/edit - and my Marker 1 is in place. Using the default String image labeled Marker 1 that went out with the demo DL.

 

Sorry, I'm being daft. I meant String.bundle, not libString.a

You also have to be using Unity iPhone, which a new trial on a new machine will give you. Also, go to build settings for IOS and make sure it's ARM6+7

 

Also, finally, just to eliminate the obvious, this is on a Mac, with Unity Pro + IOS (Trial as I said should work.) This does actually work with IOS basic too, but only when you deploy to the device- no in-editor testing - a restriction of Unity.

 

Well i didn´t set the AR scene up. That was somebody else with more knowlegde.

I only had to update it to iPhone4s native resolution and integrate AVFoundation (the project was made before, and the developer gave his best to even get it to work - somewhat awkward - without it).
But i already failed at getting AVFoundations output buffer to ARTToolkit. You are completly lost if you don´t understand c++ and Buffers and native imageconversions or formats.

By loosing the track i rather meant that, that the marker couldnt get recognized, at fast movements and hard angles. There are ways to adjust that, but trying to tweak the values didn´t seam to have to much of an effect.
Moreover we where limited to the old markers - black border and black squares within. I really like that string changed that!

 

I'm running Unity 3.1.0f3 or something like that on a Macbook Pro 2011 with OSX latest greatest, Xcode 4.x something and iOS 4.3 latest SDK's are installed. I've output apps to my iPad lots of times on this machine.

Here's the latest info though - I installed demo unity on my desktop MacPro, so it's running Unity 3.3.0f4, and I am not getting the compiler error anymore. The problem here is that I don't have a camera on this machine, and if I'm correct the Unity remote doesn't work very well with String?

So.... I'm downloading the 4GB Xcode and iOS SDK and will install it, publish the test app, and hope it works shortly. Will keep you updated.

But initial impression is that for whatever reason String will not work on Unity 3.1. (which maybe is to be expected..)

/edit - BTW, I appreciate the help runonthespot.

 

Shouldn't be- I had it working On 3.0 with Xcode 3, iOS 4.0 (should work back to iOS 3.2). Very strange. Don't have problems with unity remote and string but unity remote can't use the device camera if that's what you mean? In my mixamo string test, I controlled the horse via unity remote with string running using the MacBooks webcam.

Let me know either way- obviously I'm happy to keep suggesting ideas or failing that you could always dropbox your project to me and i'll take a look.

Regs,
Mike
@runonthespot

 

@Marjan - interesting! Yeah, re fast string, I find it handles quite rapid movement of marker and really the scene reappears as fast as unity can draw it if lost and refound. Fast enough to flicker! To be fair, I havent used ARTToolkit but then I'm slightly allergic to other peoples c++ code

 

Here's something-- I tried to send a build of an AR app to my parents to have them test it and at the String screen in the beginning, a popup error said "Invalid String License —*This app seems to be built for App Store release with the demo or development version of the String SDK."

So apparently, you can't even send your String AR app around to people you provision with your Apple Developer account for testing?!!

 

For that kind of thing I suggest emailing String. I'm pretty sure that's not the intended behaviour. Do that and if you don't have any joy, pm me and I'll get on to them too. I suspect it's extremey difficult to protect any plugin from being deployed to AppStore and likely there's an unintended side effect. They're pretty responsive in my experience.

 

Sure you used dev provision not AdHoc? AdHoc would be a deployment build too (not just the appstore) but nothing prevents you sending around stuff with the dev provision and build

 

Apparently this isn't the intended behaviour for adhoc distributions - string will be looking at it.

 

So, to say something good about string:
I downloaded the demo yesterday and were able to get that run within 30 minutes. Could have been quicker if i read the Docs for setting up the xcode project more closely.

It also looses the Marker on quick movements. That might be due to light conditions and the poor iPad2 Camera. Hmm. My idea is actually to combine the marker recognition with gyroskope. So, whenever the marker is lost, use gyroskope Data instead. This way you can move the camera to follow a moving object even when that moves so far, that the marker is not in the cameras view any more.
(Actually string could incorporate a similar behavior nativly)

 

I actually suggested exactly that to them but their answer was that gyroscope is so inaccurate compared to vision based stuff, that the switching would be jarring and not a good user experience. Worth a try maybe but probably not worth it. Re tracking I reiterate- attach marker to card or something rigid- makes a huge difference

 

String has very fast marker tracking, which is helped by the black/white rectangle. But wondering, does anyone know is it possible to track a marker without the black/white rectangle?

 

Not with String, but Qualcomm does this (with some limitations- e.g. image must have sufficient detail, not be symmetrical etc)

One trick though, is to use a black image with some white around it, as String did on the "Sneaker" demonstration.

 

Thanks runonthespot. I really appreciate your insights on the things about String. My choice so far is that String is the best. I wish they had a developer forum...don't understand what's taking them so long to put up a forum.

 

A pleasure.

The easiest question of all to answer They just launched, and have been flooded with work and partnerships- and they're a small company! I understand that it will happen. Meantime, feel free to PM me any support type questions, or ask on these threads.

I think the forum is part of a larger dev portal they're working on. I'm also not sure what's taking so long, but it fits with their general quality control ethos, as far as I've seen. Even early betas of String that I played with were already more mature than a lot of release software I've worked with, but was iterated not only for bugs, but also improving user experience (release notes including things like "Decreased String's overall CPU consumption on iOS by ~10%" and "Improved artifact dampening in tracker." as well as the usual bug fixes).

I personally hope they get strike a balance and get the support infrastructure sorted out soon though, as it's probably more of a hole right now.

 

Currently the String demo allows for snapping photos, is it possible for it to capture a video file and save it?

 

Not currently, but I'll ask they guys if it's a planned feature. To be honest, though, I doubt it.

 

Hi all,

A couple of little questions for who might know :
I'm running Unity basic + IOS basic, so there is already a "made with Unity" splashscreen.
How does that add up with the "Powered by String"?

Or do I need to move to Unity Pro to run String ?

Thanks

Yann

 

It's a few seconds splash screen + a few seconds splash screen ;-) To be honest, I think even if you move to pro, you still get the unity splash screen- it's just you can customize it. Re String, it's the price you pay for a cheaper version of the product- think of it as a compulsory ad. Not ideal I know :-/

Regarding IOS basic, it all works fine, except you can't test on the machine- when I did some dev in Unity Basic + String, I basically just turned the camera code off and set the camera to a position more or less where I'd expect to be viewing the marker from. Occasionally I'd deploy to the device to make sure everything worked as expected. I didn't actually find this too difficult, but depends really on how complex your project is. Unity IOS Pro is definitely a better user experience - being able to test with your webcam etc.

 

Qualcomm iOS SDK is out.

https://ar.qualcomm.at/qdevnet/sdk/ios

 

Thanks a lot.
I'll probably go for Strings dev license then... and save some money to get Unity+iOS pro later!

I downloaded Qualcomm iOS SDK yesterday evening to give it a try, but couldn't make it run on my iPad 2.
"Failed initialisation" straight out of the box, for the target marker demo. Too bad.

 

String Plugin seems to have a Problem with Unity 3.4, maybe someone is more familiar with xcode's Errormessages then I'am, and finds the Problem.

I tryied to Build my project with unity3.4 and get 2 errors 2times one for armv6 and 7. I've started a new Xcode Project, added the needed files and library's for the string plugin and the precompilermarko. So i hope I made everything right, did the same as with unity3.3.
The errors I get are:

Code (csharp):

1. Ld build/Unity-iPhone.build/Release-iphoneos/Unity-iPhone.build/Objects-normal/armv7/sprintdemo normal armv7
2. cd "/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02"
3. setenv IPHONEOS_DEPLOYMENT_TARGET 3.2
4. setenv PATH "/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin"
5. /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/g++-4.2 -arch armv7 -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.2.sdk "-L/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/build" "-L/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02" "-L/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/Libraries" "-L/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/../../Libraries" "-F/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/build" -filelist "/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/build/Unity-iPhone.build/Release-iphoneos/Unity-iPhone.build/Objects-normal/armv7/sprintdemo.LinkFileList" -dead_strip -all_load -weak_framework CoreMotion -Wl,-S,-x -miphoneos-version-min=3.2 -framework Foundation -framework UIKit -framework OpenGLES -framework QuartzCore -framework OpenAL -liconv.2 -liPhone-lib -framework AudioToolbox -framework CFNetwork -framework MediaPlayer -framework CoreLocation -framework SystemConfiguration -framework CoreGraphics -framework AVFoundation -framework CoreGraphics -framework CoreMedia -framework CoreVideo -lStringUnityDemo-1.0 -o "/Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/build/Unity-iPhone.build/Release-iphoneos/Unity-iPhone.build/Objects-normal/armv7/sprintdemo"
6.  
7. ld: duplicate symbol PresentContext_UnityCallback(UnityFrameStats const*) in /Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Libraries/libStringUnityDemo-1.0.a(StringAppController.o) and /Users/patricschmid/Documents/Unity Projekte/String iOS SDK Demo 1-0/Unity Tutorial/String_Demo_xCodeProj_02/build/Unity-iPhone.build/Release-iphoneos/Unity-iPhone.build/Objects-normal/armv7/AppController.o
8. collect2: ld returned 1 exit status
9. Command /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/g++-4.2 failed with exit code 1
10.  
11.  

I would be happy if someone who's better into the string plugin and Xcode stuff could give it a try if it still works with unity3.4

Thx

 

Just to contribute, Qualcomm released today the Beta version of their Augmented Reality library for iOS, there is also the Unity package. And, it's FREE!

GC

 

Did you mean to target iOS 3.2? ("setenv IPHONEOS_DEPLOYMENT_TARGET 3.2") -- off the top of my head, I'd guess that's your problem.

Yes, String works just fine with Unity 3.4

 

I just left the deployment target settings that was already set up in the string demo Project and that was iOS version 3.2. These settings worked well yesterday with unity3.3.

And the function PresentContext_UnityCallback in the AppController that is mentioned in the error report changed quite a bit, between Unity 3.3 and 3.4 but I have no clue, what this error message means ^^

Anyways changing the target iOS does not change anything

My Xcode Version is still Version 3.2.5 with iOS version 4.2, I didn't upgrade until now, because of the problem that unity 3.3 had with iOS version 4.3. If that changes anything.

 

String have announced on twitter that there is an issue with Unity 3.4 + String, and to revert to 3.3 for now pending a fix. I suspect it's something to do with the reticle orientation with the new native rotation functionality.

 

Hello Aron,

The video looks great, and shows that the IOS QCAR library actually works contrary to my results...
I can get the demo project compiled to IOS just fine. But i can't get it to actually work.. It's shows what the camera is seeing, but doesn't seem to recognize the image targets. I'm using the default image targets, and the default config.xml.

Is there some other trick to getting QCAR working on IOS?

 

What device is it?

 

It's an iPad2, certainly high specs enough as it's the fastest IOS device.

 

The question was actually not due to the part of performance as thats not of primary interest but if the hardware is of new generation or 3GS and related to currently present limits. But the ipad2, as laughable as its cam is, fullfills the requirements on that end.

 

How can i touch object in String ! I buy developer licence i dont have examples fro appstore like Sniker....Where is documentation !!!

 

I'm trying to decide between Qualcomm and String, but while QCAR seems "free" the license agreement stipulates otherwise:

So, while String's fees are hefty, at least we know what they are. Does anyone from Qualcomm care to comment? Because I'd hate to get something working and then discover there's a huge licensing fee.

 

Another question — I love the effect of having the car come out of an elevator buried in the surface. Is there a simple way of doing this trick, or are you effectively rendering a portal until the car breaches the surface, then moving the car into the "real world"? (In essence, is there a way of making a material block rendering?)

 

I think the not being commercial is just the terms of the beta. Here is a q&a off the Qualcomm AR site:

Default Can we put apps with Qualcomm AR on the store?
Hi there,

Sorry if this is a stupid question, but I was wondering if it's okay to put apps on the store that use this?
I was reading through the terms and conditions and it seems like this can only be used for testing purposes, is that right?

Thanks
Pete
***************************************************
Moderator

Just as we did with the Android version of Qualcomm AR, we are seeking feedback to help us get the iOS version from beta to final release as soon as practical.

We look forward to your input to this process, and seeing the technology being used in some exciting applications.

 

I have to agree that although String AR is really impressive (fast, stable tracking, not much jitter in rotation (unlike QCAR)), the pricing scheme is absolutely ridiculous.
I wish them good luck with that.

 

Yes it is possible. You must use DepthMask material. Create a box and assign depthMask material on it. Then you can hide stuff behind the box.


I think the String is more stable because it requires you to an entire image in camera field. (or atleast 85% of it). In QCAR i have managed to get away with only 20-25% of the image and it was still able to track 3d objects.

EDIT: And you really dont have to worry which software you use for AR tracking. As long as you use Unity you can easily port your entire project from QCAR to String and vice versa. I suggest you to develop your app and then try both version and see which suits your needs better.

 

@milk: Thanks for your reply. I figured the DepthMask shader thing out for myself (and you actually need to change the queue tags to make it work, I made a note in shader wiki).

I'm a bit shocked that String is _more_ stable than QCAR because I find it pretty jittery. String is very restrictive in terms of what your marker has to look like, which is a bit of a concern. I suspect it allows them to have a much cheaper (CPU-wise) algorithm.

 

@elecman It's pretty hard to find any one of these frameworks (if you can call them such) with a straightforward pricing model. At least String's pricing model is simple and straightforward. And you can develop and test your app without paying a cent. Metaio (another library in this family; I don't think it does Unity support though) charges for developer seats AND per product. (And they won't say how much in public.)

It's pretty sad, because they're all just ripping off algorithms published in an academic paper (but implemented in Java). (I think the academic is also trying to sell a framework...)

 

Does anybody know how one can make a ScreenShot of a String AR Scene in app? I tryed with prime31 etcetera plugin, but so far i only see the videoCapture but without the Augmented 3D images.

Damned.

Edit:
Funny enough it works with the not so good old Application.CaptureScreenshot("unityScreenshot.png");

But since we are stuck with Unity 3.3 in combinition with string this doesn´t work with MSAA.

 

@podperson to improve stability of String (you mention Jittery), pay close attention to the paper you print on - as matte as possible, and lighting conditions. Also, make sure that the paper marker is attached to something rigid (cardboard typically if you want to move it around).

Regarding "They're all just ripping off algorithms published in an academic paper" I'd check your facts before asserting blanket allegations like that. In the case of String, I know that to be false.

 

String...........Money....

 

And another question:
Did anybody have success with String and enabling MSAA under unity 3.3? i am trying on ipad 2, doesn´t work.
Either the quality settings are ignored, setting those up results in No MSAA.
if i set appctrollers.mm #define MSAA_DEFAULT_SAMPLE_COUNT 4 then i see the video but no 3D objects any more.

 

Update from String:-
(Incidentally, access to the live video texture now available: Cue strange reflections?)

String™ iOS SDK 1.1 is now ready for download. It has a host of new features and fixes -- more details on that below. Most importantly for Unity users, it introduces full compatibility with Unity 3.4. Due to the nature of Unity 3.4's changes, String™ iOS SDK 1.1 is not backwards compatible with earlier versions of Unity.

Please note that the preprocessor macro "PresentContext_UnityCallback(x)=PresentContext_UnityCallback_Old(x)" previously required by String's Unity library is no longer required as of 1.1.

Unity's "Use Animated Autorotation" option is not supported by String™. Please uncheck that option when using auto rotation.

Download String™ iOS SDK Demo 1.1 here. The terms and conditions of your existing Demo license agreement apply.

Future updates will be delivered through the String™ website.

Changes since 1.0:
Full support for Unity 3.4.
Access to the live video texture.
Improved GLES performance.
Improved GLES state handling and cleanup upon StringOGL deallocation.
Support for Unity's screenshot function.
Easier to integrate with other Unity plugins.
Lots of minor fixes and improvements.

 

Does Access to live video texture means what i think it means? Can we play videos on a 3D object in augmented reality?

 

I haven't tested it yet, but I assume so. In previous versions, you can see that the background plane that is the video happens via a texture/material, and Johan mentioned they were looking at making that texture accessible.

 

I purchased the String AR SDK and had a go with it. I'll compare the QCAR Unity plugin vs the String AR Unity plugin.

Setup:
Both with QCAR and String quite easy. Both contain sample projects for Unity. The latest update from String doesn't require the Xcode modifications as well.

Camera settings:
String doesn't need to download camera settings from the web while QCAR does. This shouldn't be too much of an issue normally but this QCAR "feature" created many headaches for me due to a lack of internet connection at times, or the OSX beta version not willing to download the settings at all.

Preview on PC/Mac:
String has a camera preview feature which enables you to see the augmented reality graphics on your mac instead of the mobile device. Very neat. It allows you to develop apps much faster. Especially debugging is much easier. I have yet to get it to work on my Hackintosh though.

Marker management:
With String, you don't need to upload your markers to an online server to have it processed. You just copy a png to the Unity project folder and the local software takes care of the marker recognition.

Speed:
With String, the speed of a marker being recognized after it has been lost is very, very fast. In fact, it seems to be instantaneous while there is a visible delay with QCAR.

Latency:
String has not much latency between moving the physical camera and the entire scene (camera picture plus augmented 3d content) moving with it. It is quite snappy. QCAR has a visible delay. Low latency is very important for a convincing AR scene. And the problem isn't with the android phone because the build-in camera-snapshot mode has no visible delay between moving the camera and the screen moving with it.

Rapid motion:
With String, the marker is still being tracked during rapid camera movement. QCAR looses the marker easily with rapid camera movements. This might have to do with the camera quality though, as camera movement creates blur if the exposure time is too long. Not sure what is causing the difference... And no documentation from both QCAR or String on how to change the exposure time, if it is possible at all...

Jitter:
With String, the rotational jitter is much less then with QCAR. This is most noticeable for large objects in relation to the marker, or when objects are placed far away from the marker. The latest QCAR update fixes this a bit, but String is still more stable at tracking rotation.

Multiple marker coordinates:
QCAR has the ability to have multiple markers in world space while if you want to use multiple markers with String, you can only have their position and orientation relative to the camera. With String, this creates issues if you want to use multiple markers with physics enabled. Your content will be flying all over the place as the world is rotated around the camera. But QCAR has it's own issues with this. The first marker tracked will be the world center so you will have to keep track of the gravity vector as well. It depends what you are using it for I guess.

Marker visibility:
With QCAR it is easier to detect whether a certain marker is visible or not. With String, this requires some code which is undocumented.

Occluded markers:
QCAR supports partially occluded markers (image targets) while the markers from String have to be (nearly) fully in view.

Conclusion:
So in my opinion, technology wise String is superior. However. As I said before, String is very, very expensive if you want to publish you work. But this is not the main problem. The main problem is that you have to pay every year for your license. So you better make sure you have enough people buying your app every year, otherwise you will make a hefty loss. Or make people pay every year to use your app, but you are going to be very unpopular if you do that. And it creates all sorts of licensing issues, like what happens if you cancel your subscription. Do the people who already purchased your app loose their right to use the app?

Because of this silly licensing system, I would still prefer to use QCAR. Mobile devices are becoming faster every year so the speed advantage String has, will soon disappear I believe. And the rotational jitter issue with QCAR will be resolved at some point I am sure.

 

One thing i dont like about QCAR is that you have to put a license agreement stating that Qualcomm can collect your phones personal information. I know that this information is needed because of the device fragmentation but i think it scares a lot of people away. I hope that the iOS version does not need this disclaimer.