Problem with Text in 3D Space

Hello, I am currently working on visualizing some data of my objects using Text that floats above them. I have a prefab for this. The hierachy looks like this:

• Plane
  • Text (Text Mesh)
    
  • Text (Text Mesh)
    
  • Text (Text Mesh)

This works for the most part but sometimes the text changes its color to gray.
This is how it looks when looked straight at:



But when zooming around it looks like this (sometimes):



What is happening here and how can I fix it? It is kinda annyoing.