Question Panorama Image Stitching

Hey everybody!
I am hoping to implement pano image stitching at runtime in Unity using EmguCV plugin, targeting mobile. So far I am able to use the high level Stitcher API (Stitcher.stitch method), but when the source images are too large or too many of them, the process is very resources heavy.
What I've tried:

1. I am basically using 22 images taken by mobile photo camera (resolution approx. 2220x1080 each)
2. I successfully implemented stitching in a separate thread (using Unity Jobs system), but it is still too heavy / takes too long / is unstable. I am using OpenCL acceleration which helps in terms of speed but seems a bit unstable in terms of memory (occasional crashes on subsequent attempts).
3. I thought i would do the stitching pairwise after taking every pair of images, instead of loading them all at once to the stitcher after they are all taken, but that introduces a known issue of left side of the intermediate result getting gradually blurred (as a result of consequnt blending), so the Stitcher is unable to stich more images anymore at some point.
4. I am aware that in OpenCV i could pass optional ROIs along with the input images, which might reduce both processing workload and undesired blurring effect, but i could not make this work using EmguCV (there is no such overload of the Stitcher.Stitch method with optional ROIS). I tried to apply the ROIs to the input Mats before passing them to the Stich call, but that doesnt seem to work either (creating a Mat using IntPtr and ROI from an existing Mat only allows setting a single ROI, while in this scenario it makes sense to set 2 ROIs for each source Mat/Image).
5. I also tried to take all the images first, then recursively separate them into pairs and stitch each pair individually and so on, which seems to work quite well (using adaptively reduced PanoConfidenceThreshold for each next iteration to take the blurring into account), but sadly it crashes after 3rd iteration (at that point i have 5 intermediate stitching results, made out of original 22 images).

I was trying to put the code of my parallel job attempt but it doesnt let me paste save the post edit for some reason:/ I might attach its as a file if necessary.

I am running out of ideas here. I am thinking to try the other Stitcher API using EstimateTransform and then ComposePano, as it seems to have an overload that allows me to pass some image masks along with the input images, but frankly i dont understand how should i specify these for them to be in the required format IArrayOfArrays?? They are not ROIs after all?
But still I am not sure how to make proper use of parallelization either way:/

Obviously I am lacking deeper insight into the computer vision algorithms and / or OpenCV / EmguCV architecture here..

My main questions thus are:

1. Does it even make sense to try to accomplish runtime panorama stitching in on mobile in Unity in the first place? I assume that if native mobile apps can do it, then it should be possible.
2. Does it make sense to try call EmguCV API inside a unity Job?? I suspect this might the main problem here.
3. In case of positive answer any of previous 2 questions, how to optimize the stitching properly? Do I have to dive into low level stitching API to stitch the images one by one and then SOMEHOW(?) build a panoramic projection out of the result?

Any suggestions will be highly appreciated!

 

Quite frankly stitching a 52 Megapixel (22x2220x1080) Pano is quite a job on a full grown Computer already. What speed of your implementation are we talking about here?

 

Hey, thanks for your prompt reply!
Well in a single separate thread it takes from 10 seconds to 5 minutes on my laptops (one is i5 / integrated intel graphics, the other is i7 / 2x GForce GT 650M on SLI), according to stitching params, number of images and also camera angle. On mobile (Samsung Galaxy A40) I can hardly get it under 2 minutes.
Thats why i am trying to use Jobs system to be at least able to display some UI feedback (processing) meanwhile (running the stitching in the main thread obviously halts everything, even in editor). Doing it pairwise recursively using the IparallelJobFor seems a bit more faster but it still crashes and almost never finishes.
I would basically be happy if i can make it up to 30 seconds stable, that would be acceptable i think.
But still, the problem is also with the stability - even if it finishes sucessfully once or twice, it either crashes or just fails (correctly logging the Stitcher error status) next time.
Even though I thing i double or triple checked i am disposing all relevant resources properly (not getting any errors about NativeCollection not being disposed etc), I still suspect there are some memory leaks happening, maybe due to bad architecture of my algorithm, especially and the way i am creating, converting and managing the Mat objects or maybe allocating managed memory inside the jobs themselves..
Perhaps I should use profiller to examine that, but especially regarding Jobs i am kind of lost there^^