QR Code recognition in Unity in 2015: What happened?

Not sure if the thread is right for the forum section, but anyway. I was curious was happened to the state of solutions for the seemingly simple task of QR recognition that's surely frequently used in various games. Over the course of two past weeks I have tried seven different solutions, with all of them either being broken or having major drawbacks, a stark contrast with 2012-2013 where people got it to work easily if cached threads are to believe.

The thing I'm trying to do is very simple: getting a string out of a QR codes in a video feed (I'm making a basic location-based quest where people travel to certain places in the city, find QR prints there and use the app to verify they found the right location). I have two iOS devices (4th generation iPad and iPhone 5S) with iOS 8.1.3 I tested all the following solutions on. To recap, here are ways to recognize a QR code from a video feed on a mobile device I'm aware of:

• ZXing library hooked to WebCamTexture video feed. Found here. Works in the editor, silently fails to recognize anything on an iOS device, no errors to work with.
• ZXing library hooked to Image feed from CameraDevice instance in Vuforia. Works in the editor, silently fails to recognize anything on an iOS device, no errors to work with.
• Antares QR package from Asset Store. According to reviews, abandoned a long time ago
• Simple QR package from Asset Store. Works perfectly in the editor, crashes the application on iOS after a few seconds, silently fails to recognize anything during those seconds.
• Uploading QR images into Vuforia Target Manager, exporting them into a trackable library, using that trackable library on Vuforia AR camera, never actually using augmentation functionality itself, hooking up to trackable states to determine which image was found in the video feed. Horrifyingly limited approach: only a small number of images can be included in the application, every activation of AR camera incurs instantiation cost for complex trackable object created for every marker from the database, most of the functionality wasted for such a simple task.
• Cloud recognition with Vuforia: removes the issue of image library size, but completely unfit for my use case both due to cost and due to the quest taking place in the areas like the metro system where absolute majority of users have no internet connectivity.
• Using Metaio QR recognition functionality. Two hundreds megabytes in library size are a deal breaker when I'm not using anything but a tiny QR recognition portion of that library. Metaio also requires manual changes to XCode project which is unacceptable as it makes it incompatible with Unity Cloud Build.

Can someone recommend an actual working solution? All I'm finding is unresolved threads about ZXing from more than two years ago, abandoned blog posts and abandoned broken examples, so save from going with incredibly inconvenient QR-as-AR-marker route, I'm stuck.

Any help greatly appreciated.

 

I've also been looking for QR Code Reader for Unity. Tried Metaio SDK but there's a signature key issue which no one support me in their forum. If you can get something , please share it here too. Appreciated.

 

@Dinksg
At the moment the only working solution I'm aware of is SimpleQR, but I can't wholeheartedly recommend it as support is nonexistent (no one answered my e-mails since March) and as far as I'm aware, Asset Store version is obsolete. I was only able to get it to work when the author sent me an in-development version back in March (Asset Store version was not working on iOS devices at all before I got it), and by fixing a certain issue in the Objective-C code of that version myself (otherwise even that version wouldn't work on iPad and iPhone 6 devices). There are limited options (no obvious way to change feed resolution, for instance), and out of the box code is not the best example for setting things up (for instance, you can't initialize the camera more than once and can't recognize the same image twice in a row, - I had to change a fair bit to get those things).

But hey, with competition like that, there is no need to bother I guess. Everything else is in an even worse state.

P.S.: If you will still go for it, and if you will be able to procure the latest version that actually works, the iPad/iPhone 6 issue was lack of flash mode availability check in the camera initialization in the Objective-C code. The place should be fairly easy to find by searching for "flashMode" - wrap the direct assignment of a mode there with a check analogous to one used for auto focus few lines above.

 

I'm trying to go for it and have contacted the developer to get this mythical unreleased version you described. If he does not respond soon, would be willing to send over your version you got back in march? I'll provide proof that I bought the asset of course.

Thanks for the help.

Also It looks like the other one that may be working for iOS is
http://forum.unity3d.com/threads/easy-code-scanner-unity-plugin-for-android-and-ios.159666/page-6
but not ilcpp support (yet)

 

Here is the fixed SQRCapturer.m which I believe should fix issues with SimpleQR crashing iOS applications.

Code (csharp):

1.  
2. #import <Foundation/Foundation.h>
3. #import <AVFoundation/AVFoundation.h>
4. #include <string.h>
5.  
6. #define BUF_SIZE (1920 * 1080 * 4 + 1)
7.  
8. static AVCaptureSession *session;
9. //static AVCaptureInputPort *port;
10.  
11. static uint8_t *s_pBuffer = NULL;
12.  
13. static int s_Width = 0;
14. static int s_Height = 0;
15.  
16. @interface SQRCapturer : NSObject <AVCaptureVideoDataOutputSampleBufferDelegate>
17. {
18.     BOOL firstFrame;
19. }
20.  
21. +(void) setup;
22. //-(void) captureDescriptionDidChanged:(NSNotification*)data;
23.  
24. - (void) captureOutput:(AVCaptureOutput *)captureOutput
25. didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
26.        fromConnection:(AVCaptureConnection *)connection;
27. - (id) init;
28.  
29. +(void) start;
30. +(void) stop;
31.  
32. @end
33.  
34. void SQRGetBufferSize(int *pWidth, int *pHeight)
35. {
36.     *pWidth = s_Width;
37.     *pHeight = s_Height;
38. }
39.  
40. void SQRBeginCapture(uint8_t *pBuffer)
41. {
42.     s_pBuffer = pBuffer;
43.  
44.     [SQRCapturer setup];
45.     [SQRCapturer start];
46. }
47.  
48. void SQREndCapture()
49. {
50.     [SQRCapturer stop];
51. }
52.  
53. @implementation SQRCapturer
54.  
55. static SQRCapturer *instance;
56.  
57. -(id) init
58. {
59.     firstFrame = YES;
60.  
61.     return self;
62. }
63.  
64. +(void) setup
65. {
66.     if (!instance)
67.     {
68.         instance = [[SQRCapturer alloc] init];
69.     }
70.     else
71.     {
72.         return;
73.     }
74.  
75.     session = [[AVCaptureSession alloc] init];
76.     session.sessionPreset = AVCaptureSessionPresetMedium;
77.  
78.     AVCaptureDevice *camera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
79.  
80.     NSError *error = nil;
81.     if ([camera lockForConfiguration:&error])
82.     {
83.         if ([camera isFocusModeSupported: AVCaptureFocusModeContinuousAutoFocus])
84.         {
85.             camera.focusMode = AVCaptureFocusModeContinuousAutoFocus;
86.         }
87.         else if ([camera isFocusModeSupported: AVCaptureFocusModeAutoFocus])
88.         {
89.             camera.focusMode = AVCaptureFocusModeAutoFocus;
90.         }
91.  
92.         if ([camera isFlashModeSupported: AVCaptureFlashModeOff])
93.         {
94.             camera.flashMode = AVCaptureFlashModeOff;;
95.         }
96.  
97.         // camera.flashMode = AVCaptureFlashModeOff;
98.  
99.         [camera unlockForConfiguration];
100.     }
101.     else
102.     {
103.         NSLog(@"Lock device failed. %@", error);
104.         return;
105.     }
106.  
107.     AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:camera error:&error];
108.  
109.     if (!input)
110.     {
111.         NSLog(@"%@", error);
112.         return;
113.     }
114.  
115.     AVCaptureVideoDataOutput *videoDataOutput = [AVCaptureVideoDataOutput new];
116.     NSDictionary *newSettings = @{ (NSString *)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) };
117.     videoDataOutput.videoSettings = newSettings;
118.  
119.     dispatch_queue_t videoDataOutputQueue = dispatch_queue_create("sqr_video_ouput_queue", DISPATCH_QUEUE_SERIAL);
120.     [videoDataOutput setSampleBufferDelegate:instance queue:videoDataOutputQueue];
121.  
122.     [session beginConfiguration];
123.  
124.     if ([session canAddInput: input])
125.     {
126.         [session addInput: input];
127.     }
128.     else
129.     {
130.         NSLog(@"Add input failed.");
131.         return;
132.     }
133.  
134.     if ([session canAddOutput: videoDataOutput])
135.     {
136.         [session addOutput: videoDataOutput];
137.     }
138.     else
139.     {
140.         NSLog(@"Add output failed.");
141.         return;
142.     }
143.  
144.     [session commitConfiguration];
145.  
146.     //port = [input.ports objectAtIndex:0];
147.     //[[NSNotificationCenter defaultCenter]
148.     //    addObserver: instance
149.     //    selector: @selector(captureDescriptionDidChanged:)
150.     //    name: AVCaptureInputPortFormatDescriptionDidChangeNotification];
151. }
152.  
153. +(void) start
154. {
155.     NSString *mediaType = AVMediaTypeVideo;
156.  
157.     [AVCaptureDevice requestAccessForMediaType:mediaType completionHandler:^(BOOL granted)
158.     {
159.        if (granted)
160.        {
161.            [session startRunning];
162.  
163.        }
164.        else
165.        {
166.            UnitySendMessage("_sqr_native", "OnGetPermissionFailed", "");
167.        }
168.     }];
169. }
170.  
171. +(void) stop
172. {
173.     [session stopRunning];
174. }
175.  
176. - (void)captureOutput:(AVCaptureOutput *)captureOutput
177. didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
178.        fromConnection:(AVCaptureConnection *)connection
179. {
180.     uint8_t flag = s_pBuffer[BUF_SIZE - 1];
181.     if (flag) return;
182.  
183.     CVImageBufferRef cameraFrame = CMSampleBufferGetImageBuffer(sampleBuffer);
184.     CVPixelBufferLockBaseAddress(cameraFrame, 0);
185.  
186.     GLubyte *rawImageBytes = CVPixelBufferGetBaseAddress(cameraFrame);
187.     size_t bytesPerRow = CVPixelBufferGetBytesPerRow(cameraFrame);
188.     size_t rows = CVPixelBufferGetHeight(cameraFrame);
189.  
190.     s_Width = bytesPerRow;
191.     s_Height = rows;
192.  
193.     size_t length = bytesPerRow * rows;
194.  
195.     memcpy(s_pBuffer, rawImageBytes, length);
196.  
197.     CVPixelBufferUnlockBaseAddress(cameraFrame, 0);
198.  
199.     if (firstFrame)
200.     {
201.         firstFrame = NO;
202.  
203.         UnitySendMessage("_sqr_native", "OnStartCapture", "");
204.     }
205.  
206.     s_pBuffer[BUF_SIZE - 1] = 1;
207. }
208.  
209. @end
210.  

You'll have to create a MonoBehaviour handling stuff like the worker thread by yourself, included example isn't really great (I have mentioned issues with it above) and my own implementation has too much stuff specific to my game to serve as a good reference.

 

Any updates on the search? We are looking for a solution as well. @bac9-flcl , did you ever try the "Easy Code Scanner"?
https://www.assetstore.unity3d.com/en/#!/content/4675

 

Judging from the description it's not supporting camera feed retrieval on Android so nope, I never tried it. And no, I'm not aware of any updates, the field seems to be completely abandoned which is frankly bizarre.

 

Some news about that subject?
I just bought SimpleQR and it doesn't work on iOS with Unity 5 it seems. I'd like to fix it by myself but i'll need the in-development version.

 

Would love some news on this topic too, since I'm about to need that feature.

Strange thing is that using dying-adobe-air, there are solutions that are up to date with iOS 9 for very cheap (30$). How come Unity with its large community doesn't get an up to date asset/package/whatever for this? Strange indeed... And quite problematic.

 

I made this work a week ago with a little help from this old thread. It worked just fine in iOS 8.4 and Android 4.2.2 for a few days but today it's not working in iOS anymore.

 

Well I make it work again.

I've followed this post with some help from this thread.

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.UI;
3. using System.Collections;
4. using System.Threading;
5.  
6. using com.google.zxing.qrcode;
7.  
8. public class qrCam : MonoBehaviour {
9.    
10.     private WebCamTexture camTexture;
11.     private Thread qrThread;
12.  
13.     private Color32[] c;
14.     private Color32[] f;
15.     private sbyte[] d;
16.     private int W, H, WxH;
17.     private int x, y, z;
18.     bool camReady;
19.  
20.     Quaternion baseRotation;
21.  
22.     public Text texto;
23.     public GameObject levelSelector;
24.  
25.     QRCodeReader QR;
26.     string output;
27.    
28.     void OnDisable () {
29.         if(camTexture != null) {
30.             camTexture.Pause();
31.         }
32.     }
33.    
34.     void OnDestroy () {
35.         qrThread.Abort();
36.         camTexture.Stop();
37.     }
38.    
39.    
40.     void Start () {
41.         QR = null;
42.         baseRotation = transform.rotation;
43.         StartCoroutine (Launch());
44.     }
45.    
46.     IEnumerator Launch() {
47.         string rearCameraName = null;
48.         WebCamDevice[] devices = WebCamTexture.devices;
49.         for (int i = 0; i < devices.Length; i++) {
50.             if( !devices[i].isFrontFacing ){
51.                 rearCameraName = devices[i].name;
52.                 break;
53.             }
54.         }
55.         camTexture = new WebCamTexture(512,512);
56.         camTexture.deviceName = rearCameraName;
57.         camTexture.Play();
58.  
59.         while (true)
60.         {
61.             if(camTexture.didUpdateThisFrame && camReady == false)
62.             {
63.                 W = camTexture.width;
64.                 H = camTexture.height;
65.                 WxH = W * H;
66.                 Debug.Log(W + " " + H);
67.                 camReady = true;
68.  
69.                 double flip = camTexture.videoVerticallyMirrored ? -1.0 : 1.0;
70.                 texto.text = flip.ToString();
71.                 transform.localScale = new Vector3((float)transform.localScale.x,
72.                                                    (float)transform.localScale.y,
73.                                                    (float)flip * transform.localScale.z);
74.                 try{
75.                     qrThread = new Thread(DecodeQR);
76.                     qrThread.Start();
77.                 }
78.                 catch (System.Exception e){
79.                     Debug.Log(e);
80.                 }
81.                
82.                 break;
83.             }
84.             else
85.                 yield return null;
86.         }
87.     }
88.  
89.     void Update () {
90.         if (camReady) {
91.             c = camTexture.GetPixels32();
92.             this.GetComponent<Renderer>().material.mainTexture = camTexture;
93.             transform.rotation = baseRotation * Quaternion.AngleAxis(camTexture.videoRotationAngle, Vector3.up);
94.         }
95.     }
96.    
97.     void DecodeQR () {
98.         Debug.Log ("DecodeQR");
99.         while(true) {
100.             if(c == null){
101.                 Debug.Log("c is NULL");
102.                 continue;
103.             }
104.             try
105.             {
106.                 Debug.Log("Decoding...");
107.                 d = new sbyte[WxH];
108.                 z = 0;
109.                 for(y = 0; y < H; y++) {
110.                     for(x = W-1; x >= 0; x--) {
111.                         d[z++] = (sbyte)(((int)c[y * W + x].r) << 16 | ((int)c[y * W + x].g) << 8 | ((int)c[y * W + x].b));
112.                     }
113.                 }
114.                 if (QR == null){
115.                     QR = new QRCodeReader();
116.                 }
117.                 output = QR.decode(d, W, H).Text;
118.                 Debug.Log ("Success");
119.                 texto.text = output;
120.                 print (output);
121.  
122.                 selectLevel script = levelSelector.GetComponent<selectLevel>();
123.                 script.tryToGoToLevel(output);
124.  
125.             }
126.             catch (System.Exception e) {
127.                 Debug.Log("Fail.");
128.                 continue;
129.             }
130.         }
131.     }
132. }
133.  

This code is a bit messy but it works.
Note that the second for-loop in DecodeQR is inverted.

The strange thing about this code is that if I delete the 'Debug.Log("c is NULL")' line then it stops working on iOS. I think that's because some weird thread behavior but I could'n fix that so I just leave it there for now. Any help with that would be really appreciated.

 

Hello!
I have a question...

This is the script for the iOS success with zxing?
I'm hardly trying to integrate this (and a lot of examples from other sources) to an Android project but it doesn't work. Is there someone who get a result with Android devices?

Thank you very much.

 

I tested my code in iOS 8.4 and Android 4.2.2 (a bit outdated maybe).

 

I've got it working on PC and Android (5.1.1) using Unity 5.3.
The 'UnityDemo' from the ZXing.NET source repository worked out of the box for me (it's located at 'Clients\UnityDemo\Assets').

 

Did you just replace the Script they had on the camera?

 

Have to bookmark it and test on some spare time

 

The 'BarcodeCam'-script is all you need

 

Any tutorial, prefab or package to use it ? I Try to add it to my project but with no luck. Thanks a lot ! !
(Creo que hablas español y es mas facil para mi. Intente agregarlo a mi proyecto y no pude hacerlo funcionar. Estaría buno si tuvieras un tutorial o un package con una scene. MUCHAS GRACIAS !)

 

Great, that you did that! I tried to use that demo, but that isn't working. I suppose I could happens because I have 64-bit Win. Could you, please, describe your steps in a few words in the blank scene to create working Android demo?
Thank you.

 

You can find my project in the attached file below. It was created in Unity 5.3.4f1, tested on android 5.1.1.

 

Thank you so much!
If anyone are trying to create a QRCode with the dll included in Chuptys package, the QRCoder.dll, is capable to create too.
Here:
https://github.com/codebude/QRCoder

 

Have you tried this code on an iOS device? Let me explain myself. I'm the developer of the QRCoder library. I wrote it as pure C# library, but I don't have any expertise in Unity. The Unity part was added by some other GitHub users.

Unfortunately some user had reported, that the QRCoder Unity part works on every OS execpt iOS. (See this user report: https://github.com/codebude/QRCoder/issues/32 )

As already said, I know nearly nothing about Unity. Has someone of you an idea, why it is working for every platform but iOS? Or is someone of you active on GitHub and is willing to help me find the source for the problem?

 

Hello.
It really doesn't work on iOS. I have a Mac and iOS devices where I work, but really don't understand Xcode to try debug.

If you get any help on this I can try to help. Sorry about this vague response.

 

Hey, I've been looking for something to read QR too. This looks very promising and appears to work on latest devices with latest version of unity: https://www.assetstore.unity3d.com/en/#!/content/56083
I'll be starting my project soon and will get into trying it when that happens, but in the meantime if anyone gives it a whirl, let us know how it goes!

 

I am also facing this issue, any solution?

 

Cross platform QR code scanner and generator for unity 3d.
http://unitydevelopers.blogspot.in/2017/05/qr-code-reader-and-generator-for-unity3d.html?m=1

Cheap and best for unity developers.

 

Can this be converted to work on a UI canvas and not the On GUI camera image as I need it to work with an Old style TV Tube filter.??

Code (CSharp):

1. /*
2. * Copyright 2012 ZXing.Net authors
3. *
4. * Licensed under the Apache License, Version 2.0 (the "License");
5. * you may not use this file except in compliance with the License.
6. * You may obtain a copy of the License at
7. *
8. *      http://www.apache.org/licenses/LICENSE-2.0
9. *
10. * Unless required by applicable law or agreed to in writing, software
11. * distributed under the License is distributed on an "AS IS" BASIS,
12. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13. * See the License for the specific language governing permissions and
14. * limitations under the License.
15. */
16.  
17. using System.Threading;
18.  
19. using UnityEngine;
20. using UnityEngine.UI;
21.  
22. using ZXing;
23. using ZXing.QrCode;
24.  
25. public class BarcodeCam : MonoBehaviour
26. {
27.     // Readout Text
28.     public Text ReadoutText;
29.     // Texture for encoding test
30.     public Texture2D encoded;
31.  
32.     public WebCamTexture camTexture;
33.     public RawImage rawimage;
34.     private Thread qrThread;
35.  
36.     private Color32[] c;
37.     public int W, H;
38.  
39.     private Rect screenRect;
40.  
41.     private bool isQuit;
42.  
43.     public string LastResult;
44.     private bool shouldEncodeNow;
45.  
46.     void OnGUI()
47.     {
48.         GUI.DrawTexture(screenRect, camTexture, ScaleMode.ScaleToFit);
49.     }
50.  
51.     void OnEnable()
52.     {
53.         if (camTexture != null)
54.         {
55.             camTexture.Play();
56.             W = camTexture.width;
57.             H = camTexture.height;
58.         }
59.     }
60.  
61.     void OnDisable()
62.     {
63.         if (camTexture != null)
64.         {
65.             camTexture.Pause();
66.         }
67.     }
68.  
69.     void OnDestroy()
70.     {
71.         qrThread.Abort();
72.         camTexture.Stop();
73.     }
74.  
75.     // It's better to stop the thread by itself rather than abort it.
76.     void OnApplicationQuit()
77.     {
78.         isQuit = true;
79.     }
80.  
81.     void Start()
82.     {
83.         /*WebCamTexture webcamTexture = new WebCamTexture();
84.         rawimage.texture = webcamTexture;
85.         rawimage.material.mainTexture = webcamTexture;
86.         webcamTexture.Play();
87.         */
88.         // ---------------------
89.  
90.         encoded = new Texture2D(256, 256);
91.         LastResult = "000";
92.         shouldEncodeNow = true;
93.  
94.         screenRect = new Rect(0, 0, W, H);
95.  
96.         camTexture = new WebCamTexture();
97.         camTexture.requestedHeight = H;//Screen.height; // 480;
98.         camTexture.requestedWidth = W;//Screen.width; //640;
99.         OnEnable();
100.  
101.         qrThread = new Thread(DecodeQR);
102.         qrThread.Start();
103.     }
104.  
105.     void Update()
106.     {
107.         if (c == null)
108.         {
109.             c = camTexture.GetPixels32();
110.  
111.         }
112.  
113.         // encode the last found
114.         var textForEncoding = LastResult;
115.         // Display The Results
116.         ReadoutText.text = LastResult;
117.         if (shouldEncodeNow &&
118.             textForEncoding != null)
119.         {
120.             var color32 = Encode(textForEncoding, encoded.width, encoded.height);
121.             encoded.SetPixels32(color32);
122.             encoded.Apply();
123.             shouldEncodeNow = false;
124.         }
125.     }
126.  
127.     void DecodeQR()
128.     {
129.         // create a reader with a custom luminance source
130.         var barcodeReader = new BarcodeReader { AutoRotate = false, TryHarder = false };
131.  
132.         while (true)
133.         {
134.             if (isQuit)
135.                 break;
136.  
137.             try
138.             {
139.                 // decode the current frame
140.                 var result = barcodeReader.Decode(c, W, H);
141.                 if (result != null)
142.                 {
143.                     // Display The Results
144.                     ReadoutText.text = LastResult;
145.                     LastResult = result.Text;
146.                     shouldEncodeNow = true;
147.                     print(result.Text);
148.                    
149.                 }
150.  
151.                 // Sleep a little bit and set the signal to get the next frame
152.                 Thread.Sleep(200);
153.                 c = null;
154.             }
155.             catch
156.             {
157.             }
158.         }
159.     }
160.  
161.     private static Color32[] Encode(string textForEncoding, int width, int height)
162.     {
163.         var writer = new BarcodeWriter
164.         {
165.             Format = BarcodeFormat.QR_CODE,
166.             Options = new QrCodeEncodingOptions
167.             {
168.                 Height = height,
169.                 Width = width
170.             }
171.         };
172.         return writer.Write(textForEncoding);
173.     }
174. }
175.