Prime31 Augmented Reality Plugin now live and special announcement!

Two quick messages for everyone:

1. Prime31 Studios will be releasing a free plugin in the Unity Asset store in the coming days. It will be the Native Pause and UI Plugin and will consist of an easy to use package that will give you the ability to create menus easily along with a really easy to use pause menu that essentially freezes the entire Unity Mono VM.

2. We have finally released our most requested plugin of all time: the Augmented Reality Plugin! It took a LOT of trial and error to find a solution that was performant enough for us to consider releasing it. That day has finally come!

Here and here are a couple demo videos showing the plugin in action on an iPhone 4. As you can see by the FPS counter, it is blazing fast! It also couldn't be much easier to use. The plugin will give you a Texture2D that you are free to set on as many renderer's as you want. The Texture2D will be updated automatically every frame with the output from the camera.

 

With the AR plugin, is there any included marker or target reading capability? Please expand a bit more on what is included and what we have to roll on our own.

Thanks!

TheLorax

 

I'd suggest, by looking at the two example videos and reading the description, that no marker reading capability is included - but getting the live camera view into unity without too much lag is a big step in the right direction! - well done Prime31.

 

@TheLorax, there is no marker or edge detection built in. There is a single hook point though where you can plug in any type of detection you would like from any of the open source (or closed source) projects out there.

 

Looks Cool!
Damn those black borders!

 

Quick update: the upcoming version of the plugin has some nice performance improvements. The performance test had to be redone with more goodies on screen. Check out a video here.

 

Any tips on what source to use and how to do this? We're considering buying the AR plugin, but have to make sure that we can actually achieve what we want with it

 

@Peter, you can check ARToilkit, ARKit or just Google it to find some neat code to use with AR

 

Hi,

first off, not to be annoying or anything, but I fail to see why it's called an Augmented Reality plugin,...
Basicly, if I'm getting this right, and please tell me if I'm not...
at this moment the plugin does not embed any AR related processing,
but is more like an iphone live camera feed plugin, right?

Of course, I know, once the feed is in, we can use it just to display the feed or do any processing we want, but still.

Anyway, pretty interesting functionality either way,
might come in handy for a project I have coming up

Is there a way to capture a higher resolution image from the iphone camera at specific moments in time?
like at the press of a button, or hardcoded in time?
Or is the access really limited to low resolution imaging?

 

@EnoJoy, the current access is a stream from the camera. You can turn it on/off at will. The plugin seems to fit the definition of augmented reality as defined by Wikipedia:

We provide the live feed and your game provides the sound/graphics/input.

 

Your AR plugin seems to be super!

• Do you have a plan to provide the same plugin for Windows?

-Kaya

 

@Kaya, all of our plugins are currently iOS only right now. We will most certainly be expanding to other platforms in the future but there is no date set yet for when we do.

 

So to get it right, its really just a Live Camera Streaming then, not an AR plugin, as even the Wikipedia entry you cite explicitely mentions the relationship of view in combination with "computer generated sensory input".

Unpleasant thing to find out. Considered getting it for experimentation and prototyping but $75 as introduction price, a price beyond the GK plugin, is definitely outside of what I consider paying for a mere live video streaming plugin that lacks any AR functionality even basic one.
But thanks for the clarification on what it is, was not clear from the vids on the website

 

You should really demo it actually detecting a marker and placing a 3D object

 

does it work with iOS<4?

 

@marianojgs, the required AVCaptureDevice is only available in iOS 4.0+.

 

" the required AVCaptureDevice is only available in iOS 4.0+."

 

I have a application with this plugin that works fine in iPhone 4, but crash in a iPhone 3GS with iOS 4.2.1. I'm using the SKD version 4.2.
any help?

 

My bad...the plugin works fine..the problem was in another plugin

 

sounds like you use more ram than you have at hand (non iphone4 is a max of 70-80mb of usage, iphone4 has 2-4 times that much at hand to use). look into the log xcode provides you in the organizer if it mentions something about memory warning. Generally without knowing the error its hard to help you

 

So what happened to this plug-in? Did it get turned into a true AR plugin or abandoned? (This thread is very quiet for over a year now).

Anyone else recommend an AR plugin for Unity? Not Qualcomm's one though because i think the terms of their free plug in are very suspicious (and I don't want my users having to agree with sharing all personal information and usage)

 

@mango, the plugin is now called the LiveTexture Plugin to clarify its purpose. It does video textures and camera textures but does not come with any marker detection which it sounds like you will need.

 

okay, thanks Prime!

 

Hi

Does the LiveTexture Plugin support video textures with audio?

 

@peter, one texture at a time (if playing multiple simultaneously) can play audio. Note that audio playback is considered a beta feature at this time.

 

Hi, I just purchased the LiveTexture plugin, and the audio playback is causing me a lot of problems. In the demo, it works most of the time, but when using the live texture in a more complex scene, the audio is very choppy, doesn't respond to the stop or pause calls, and when it finally ends and I try to play another video the application always crashes. I'm using the videos that come with the plugin. Setting autoDestructOnCompletion to true doesn't help. Any idea when the audio support will leave the beta status?

Jorgeche

 

@Jorgeche, audio support is currently in beta. The more complex the scene the more difficult it is to keep audio in sync with video. It is a very tough thing to do perfectly with the limited resources of a mobile device. It is already a work of magic to have an entire game running and decoding a video simultaneously. We have no ETA on when audio support will be leaving beta at this time.

 

hi prime,

I have been searching all over the internet for a simple AR plugin that just gives me a fast camera feed and leaves the AR to my app. Looks like this is exactly what I want. However, given the nature of the project I am working on, it would have been great if we could have a very limited demo version of this plugin just to convince the funding co-ordinator that it will work. So I was wondering, is there any option like that? Or any other extensive document that will help us convince the supervisors? I know this sounds ridiculous for the price you guys are offering, but since it's a funded project, my hands are tied. Hope you don't take any offence, thanks for your contributions to Unity!

 

@rockin, we don't offer demos but if you take a look at our YouTube channel (prime31studios) you will find many demos of the plugin.

 

Hi Prime! Today i was trying to put a plane with a window and add the depth mask shared on it in order to have a "window" with a video mapped on a sphere.
The depth mask its not working...i can see the entire sphere with the video playing on it...do you know if its possible to do something like that?

Cheers,
Ale.

 

@iguana, I don't see any reason why that wouldn't work. The live textures material works the same as any other material in Unity.

 

hi prime,

Thanks for your reply! another quick question before buying the plugin, we might switch out platform to one of the windows-based slates. Does this plugin work as the webcamtexture of unity (i.e. will it work on say windows 7 with a webcam attached?).

 

@rockin, it is a native code plugin and will only run on the platform it was coded against (iOS)

 

hi Prime, I have bought the plugin and started testing with it, looks pretty good! I am stuck at one point, I am trying to do some image processing (e.g change all the colors to grayscale) on the texture2D returned by the script in my Update() function (I started from the provided example script as reference). However, it seems like the texture Color information (accessed through texture.getPixel()) is not being updated, I always receive (0,0,0), although the video is looking fine on the plane. Does that mean I am doing something wrong, or is there any other way to access the texture color information? Thanks for your help

 

@rockin Unity doesnt provide access to the colors via script for textures created outside the editor. You will need to use a shader to modify the colors.

 

Thanks, will try that

 

hi Prime,

I am back with another question. When I use your plugin, the camera feed seems to be a bit "zoomed in" compared to the ipad camera app. I am assuming this is because of the resolution difference between ipad2 and ipad3 camera (I am using ipad3). Can there be any other reason for this? Also, do you plan to release an updated version of your plugin for ipad3 (since this plugin can only go up to 1280X720 resolution).

Thanks for your help.

 

@rockin, feel free to adjust the UVs/scale of whatever object you are putting the Texture2D on. You have full control of that. Anything over 1280x720 completely kills performance and requires specific iOS versions to use so we have no plans to add any other resolutions at this time.

 

Hi Prime,

I want to use the "Live Camera Texture Plugin" in my augmented reality iOS app. However, I did not find any documentation on whether one can pass the native texture frame(not unity texture 2D but the frame that we get from ios camera) to the augmented reality library for pose calculations which is available as a native iOS lib using this plugin. An early reply is greatly appreciated.

 

Hi Prime, i just updated the plug in (Livetexture) with the latest version but i got a problem which i can really understand...

Even if i open and publish just the test scenes, once i publish a test app it crashes.

I just click on the start video texture button and the app crashes...HELP!

Ale.

 

@iguana, logs?? backtraces?? device tested on? iOS version? We need some information to work with here.

 

Hi Prime, hehe i guess you right...

- Unity 3.5.6 f4 PRO,
- iOS both 5.1 and 6.0
- XCODE 4.5.2
- New iPad and iPhone 4

On the unity side all fine (of course because the actual plug in is for the iOs side)
On Xcode, the moment i click on the button that activate the videotexture i got immediately a BAD ACCESS.

This is the condole before i click the button:

Code (csharp):

1. 2012-11-12 21:33:40.191 aps[20174:907] -> registered mono modules 0xa60d2c
2. -> applicationDidFinishLaunching()
3. QCAR Extension version 1.5.10
4. 2012-11-12 21:33:40.567 aps[20174:907] INFO/AR(20174) 2012-11-12 21:33:40: QCAR SDK version 1.5.8
5. Mono path[0] = '/var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed'
6. Renderer: PowerVR SGX 535
7. Vendor:   Imagination Technologies
8. Version:  OpenGL ES 2.0 IMGSGX535-73.16
9. GL_OES_depth_texture GL_OES_depth24 GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_float GL_OES_texture_half_float GL_OES_vertex_array_object GL_EXT_blend_minmax GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_map_buffer_range GL_EXT_read_format_bgra GL_EXT_separate_shader_objects GL_EXT_shader_framebuffer_fetch GL_EXT_shader_texture_lod GL_EXT_texture_filter_anisotropic GL_EXT_texture_storage GL_APPLE_copy_texture_levels GL_APPLE_framebuffer_multisample GL_APPLE_rgb_422 GL_APPLE_sync GL_APPLE_texture_format_BGRA8888 GL_APPLE_texture_max_level GL_IMG_read_format GL_IMG_texture_compression_pvrtc
10. Creating OpenGLES2.0 graphics device
11. Initialize engine version: 3.5.6f4
12. Begin MonoManager ReloadAssembly
13. Platform assembly: /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/UnityEngine.dll (this message is harmless)
14. Loading /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/UnityEngine.dll into Unity Child Domain
15. Platform assembly: /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-CSharp-firstpass.dll (this message is harmless)
16. Loading /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-CSharp-firstpass.dll into Unity Child Domain
17. Non platform assembly: /private/var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/System.Core.dll (this message is harmless)
18. Platform assembly: /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-CSharp.dll (this message is harmless)
19. Loading /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-CSharp.dll into Unity Child Domain
20. Non platform assembly: /private/var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/System.dll (this message is harmless)
21. Platform assembly: /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-UnityScript.dll (this message is harmless)
22. Loading /var/mobile/Applications/CC22210C-E68E-4586-92F8-4C7C8D958AFB/aps.app/Data/Managed/Assembly-UnityScript.dll into Unity Child Domain
23. - Completed reload, in  0.118 seconds
24. -> applicationDidBecomeActive()
25. -> force accelerometer registration

after clicking the button that activates the video texture it just add this:

Code (csharp):

1. icall.c:1331:
2. (lldb)

the i got immediately a EXC_BAD_ACCESS and i got many errors, one for example is:
0x51f918: bl 0x64210c ; GetInputManager() at InputManager.cpp:20

This happens with the test scenes that comes with livetexture plug in.

I tried to re-download and re-import again the texture, but no luck...

 

@iguana, try it in a project without QCAR. QCAR will be using the hardware decoder so it won't work simultaneously.

 

Hi Prime, i used the old version called AR plug in and it worked perfectly together with QCAR , and in fact that's why i need it, run a video texture in AR.

The video texture inside QCAR plug in is not good as yours that's why i'm using both plug ins.

Is there a reason why the old "Prime31 Augmented Reality Plugin" works and" Livetexture" not?

 

@iguana, the underlying code is the same. Only the name changed.

 

Thanks for your help man, i will try to figure out what's going on....i mean, its a very weird problem.
I got a old project that works perfectly (and also the audio is in perfect sync on an iphone 4) and a new project that doesn't work with the new live texture plug in...

I can continue to use the old one but i lost the unitypackage and i can only manually copy the AR plug in into a new project.

EDIT: It just jumped in my mind...i actually did test the Livetexture plug in without QCAR and i got the same result...crash...don't really know what to think, also seems that i'm the only one with this problem.

 

@iguana, please include the crash log, a backtrace and steps we can follow using the demo scene to reproduce the issue. We have tested on a huge amount of devices with no issues.

 

I will test it tomorrow morning and post it here, i know its weird...it must be something wrong here because there is no trace at all of this problem in the entire world...

 

Was there any progress on this bug? I am having the exact same problem. I'm running with the latest Video Texture plug-in and no other plug-ins in the project. When I try to run the sample test scene everything works fine until I click one of the buttons to run the video. Then - BOOM. Xcode crashes with an EXEC_BAD_ACCESS

This occurs on the iPad3, iPhone4s and the iPhone 3GS

Unity v3.5.6f4
Xcode 4.5.1
Mobile devices running iOS 5.0.1 and iOS 6

I'll post the crash log and backtrace as soon as I figure out how.

Thanks in advance,

Jason Stark
Disparity Games

 

I've attached the console log.

There is no crash log because the app never properly crashes. It just pauses execution and opens the debugger. I can't step over or continue execution however so the program is frozen until I manually shut it down.

Thanks,

Jason Stark
Disparity Games