Question Can I force my plugin to only use Gamma space, regardless to the project color space?

Hello,
I am working on this plugin, and in that plugin I deal with image processing operations. It's pretty much an utility tool.
If the project uses gamma space everything works as it should, with no problems whatsoever, however, if the color space is set to Linear, everything is messed up.

I've done research (and I mean A LOT of research). I implemented gamma correction, it fixed some problems, but there are always more just popping up out of nowhere... I am a bit lost and not sure what to do. I spend so many days trying to fix this issue, but can't... Linear space just complicates things sooooo much, it's ridiculous. I tried everything, things just don't look right regardless to what I do.

So, my question would be, is there by any chance any way I could separate my editor plugin, so that it would use gamma space only? I know the question is very vague, I am not sure how else to explain it..
Or some other way that would help me avoid dealing with this Linear to Gamma nightmare..?

I could just set the color space to gamma, when I launch the plugin, but in bigger project that would end up taking way to much time.. And I mostly work with Linear space when I work on the game project, so this solution would just not work.

Any help/advise on what should I do, would be greatly appreciated! Thank you