Palette swapping not working

I've made a character with every pixel in a different color and laid out his clothing to create a template. I want this one to be a caveman, so I used the template and desired outcome should be this (the rest of the palette is for the back):

Here's the script for color changing(map is the colorful template, sheet is caveman clothing):

Code (CSharp):

1.  
2.     public Texture2D map, sheet;
3.     Color[] sheetPixels, mapPixels;
4.     public static Dictionary<Color, Color> Colors = new Dictionary<Color, Color>();
5.  
6.     void Start()
7.     {
8.         sheetPixels = sheet.GetPixels();
9.         mapPixels = map.GetPixels();
10.         for (int i = 0; i < mapPixels.Length; i++)
11.         {
12.             if (mapPixels[i].a != 0)
13.             {
14.                 Colors.Add(mapPixels[i], sheetPixels[i]);
15.             }
16.         }
17.         Texture2D currentTexture = GetComponent<SpriteRenderer>().sprite.texture;
18.         for (int y = 0; y < currentTexture.height; y++)
19.         {
20.             for (int x = 0; x < currentTexture.width; x++)
21.             {
22.                 Color currentColor = currentTexture.GetPixel(x, y);
23.                 if (Colors.ContainsKey(currentColor))
24.                 {
25.                     currentTexture.SetPixel(x, y, Colors[currentColor]);
26.                 }
27.             }
28.         }
29.         currentTexture.Apply();
30.     }

However, when I run it only some of the pixels get swapped. Do you have any idea why that happens and how to prevent that?

 

As I thought, the issue was float precision. I rounded up the RGB values and now it works perfectly.