Chromakey within AR-Experience (Hololens2)

I am a student of design working on a project in which would like for a Hololens2 AR experience to augment/replace/"key out" areas of my surroundings which are of a certain colour with content such as video and still images in accordance to the principles of classic chromakey technique. The objects in question (food on a table in front of the viewer) are in close proximity (one to two meters), fairly uniform in colour and well lit. This is quite easy to simulate in a non AR environment using a modified chromakey shader. As i am not all to knowledgeable on Unity or development for Hololens i am struggling as to how is can be achieved in AR. Can you guys give me some pointers as to how one might go about this? Do i generate a full colour point cloud of the environment and then stream images for each eye to the hololens' screens to then apply shaders and modifications to suit my needs? Any help is greatly appreciated. This might be trivial to most of you and i apologise for bothering you with this.

best regards
Fabian