Google's RAISR image-upsampling algorithm

This looks potentially interesting for certain game-related problems. Their focus is reducing bandwidth consumption but the algorithm is reportedly fast enough to be used realtime on mobile. They're generating surprisingly sharp images from source material that is just one quarter the size of the output.

It's like a reverse mipmap...

https://research.googleblog.com/2016/11/enhance-raisr-sharp-images-with-machine.html

 

The result looks like vector art. So, why not use vector art in the first place?

 

Not really.

https://4.bp.blogspot.com/-mOlb1GnM...C-0T4fgM9V_OZtNFUwKSEsR5qcJzwCLcB/s1600/4.png

 

That does not look very impressive to me.

Here are few vector art examples:

http://www.amusingplanet.com/2010/01/incredibly-photorealistic-vector-art.html

 

Not sure which is more impressive.. the talent behind making realistic vector art or the algorithm for raisr

 

Those are impressive, but you're missing the point by ignoring two critical factors: performance (mobile or otherwise) and the effort required to create the images in the first place (scaling down a bitmap versus hand-drawing vector wireframes and tweaking a zillion gradients).

 

I read a whitepaper on this a couple of weeks ago. It's decent but has limitations and quality will definitely vary depending on your image so, while cool, I don't see it as something that can easily be used as a general purpose solution.

 

This looks cool, if the system ever becomes available for public use in a functional form I could see it being helpful in texture work.

I doubt you'd get much use of images upscaled more than 2x, simply because the algorithm would probably create details of its own by then. There's only ever so much detail the algorithm could pull from the original image. For example on the image of a face, some of the wrinkles are brought out more, but areas where there isn't much detail to begin with the algorithm seems to just sharpen the image a lot.

Definitely curious to see if this ever sees the light of day.

 

I played with making a custom upscaling algorithm here https://forum.unity3d.com/threads/new-method-for-upscaling-pixel-art.274930/ mainly thinking towards sprite or texture data, but I suppose anything. Results weren't too bad... but would take some significant effort to make it realtime. Small little 1-pixel details are the hardest to deal with. I didn't really try a simple 2x upscale on a basic texture.

I guess upscaling would have applications in image processing software like photoshop etc, if you could get a better quality image out of a smaller one. I would guess an up-scale like Google's followed by a quality downscale might produce a better image.

It does look like some of their texture examples are quite good quality, compared to the 2x2 pixel version. I imagine they're basically heading towards a filter that's better than bilinear/trilinear, where textures up-close etc will look more detailed than they are. Sort of a way to procedurally augment an existing texture.

 

Standard browsers support svg rendering, as far as I know. SVG is vector graphics.

The point is I think it could be a better idea would be to try and find "vectorization" technique. Hence I'm not excited about this algorithm.

This example does not look great:

Spoiler

And reminds me of a bad webcam picture.

It could be interesting if this could be applied at realtime as MAG filter on textures.

However, it looks like it probably won't be really possible.