Model size and scaling question

Hi

So I downloaded some free assets, and when I drag the prefab model into scene, it is about 10x too big. I can adjust the X Y Z scaling to 0.1 each and the size is about correct.

Does this method affect memory usage and/or runtime speed? Would it be better to have properly scaled model so do not have to adjust scale, or is scaling not something I should worry about?

Thanks