Is there a way to smooth out the diffuse fast material?

So the light that hits it isn't 100% face dependant. : - )
Add some curves to it basically

 

Yes. Use a mesh that has smooth normals. The Diffuse Fast material doesn't do anything but show you the normals on the mesh you've given it.

I'm guessing you're importing in a mesh from Blender and seeing facetted normals. That's because by default that's what Blender's meshes have, flat surface normals. Look up Blender mesh smoothing / smoothing groups.

 

it's very late here and i am just checking before bed but that something different than smooth or flat shading in blender right? (smooth groups you mentioned)
the thing is if i have regular wall that i added few more loop cuts to, just so its a bit more detailed with diffuse fast, if i go up close with a flashlight, you can cleary see lit or dark triangles.
without those loop cuts , up close its two giant triangles ,one dark one lit

 

i use smooth shading btw, no difference

 

Ah, a flashlight. That changes things.

The solution is to not use Diffuse Fast. It's using per vertex lighting which works best with larger point or directional only lights. Something like a close spotlight will break down. Try using the Legacy Shaders/Diffuse rather than Diffuse Fast, which uses per pixel lighting.