Question Figuring out Get/SetDetailLayer

Hey, guys. I'm trying to figure out how to work with Unity Terrains a bit better.
My current set up is MapMagic 2, Nature Renderer and VSP.

Right now I'm trying to figure out how I can modify vegetation at runtime. I've kind of got it working but it's super wonky. This is what I'm attempting so far. I have no idea how the coordinates work with the detail layer.
Here it worked perfectly on a random patch :


Here's another spot where it kind of worked but removed only a bit of vegetation not quite in the right spot.



script in it's current form.

I'm using HDRP in 2020.3. I can use colliders to mask out the vegetation but I'd like to be able to modify the terrain during runtime (create holes for example) and remove the existing vegetation.

Code (CSharp):

1. public void RemoveVegetation(Bounds mask)
2.         {
3.             var position = mask.center;
4.          
5.             UpdateTerrainData(position);
6.          
7.             var size = mask.size;
8.             var halfSize = mask.extents;
9.          
10.             Debug.Log($"Attempt vegetation removal at <color=green>{position}</color>");
11.          
12.             for (var i = 0; i < _terrainData.detailPrototypes.Length; i++)
13.             {
14.                 var map = _terrainData.GetDetailLayer(
15.                     0,
16.                     0,
17.                     _terrainData.detailWidth,
18.                     _terrainData.detailHeight,
19.                     i
20.                     );
21.                 for (var x = 0; x < _terrainData.detailWidth; x++)
22.                 {
23.                     for (var y = 0; y < _terrainData.detailHeight; y++)
24.                     {
25.                         var worldPos = new Vector3 //these are weird.... not sure how to set these coordinates up
26.                         {
27.                             x = _terrainPos.x + (float) y / _terrainData.detailHeight * _terrainData.size.z,
28.                             z = _terrainPos.z + (float) x / _terrainData.detailWidth * _terrainData.size.x,
29.                             y = position.y
30.                         };
31.  
32.                         if (!mask.Contains(worldPos)) continue;
33.                      
34.                         Debug.Log($"Map [{x}, {y}] value : <color=green>{map[x, y]}</color> at <color=green>{worldPos}</color>");
35.                      
36.                         map[x, y] = 0;
37.                     }
38.                 }
39.  
40.                 _terrainData.SetDetailLayer(
41.                     0,
42.                     0,
43.                     i,
44.                     map);
45.             }
46.             // Update Nature Renderer's buffers.
47.             _natureRenderer.Refresh(NatureRenderer.RefreshFlags.DetailLayers);
48.         }