Search Unity

Importing GDAL from Nuget Package

Discussion in 'Editor & General Support' started by gilley033, Sep 11, 2019.

  1. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,191
    I am attempting to import GDAL into Unity. I am using the GDAL library within my own custom class library, which is compiled in Visual Studio Community 2019. The GDAL library I am using is installed via some Nuget packages (GDAL, GDAL.Native, and GDAL.Plugins, all of which are basically the official C# ports of GDAL).

    The idea is to expand functionality of my Asset Store Terrain Importer tool to allow for the conversion of virtually any GeoTiff file into a Unity Heightmap.

    Upon installing the Nuget packages, a .cs configuration file is created to help configure some bindings and specify the locations of the various .dll files used by the GDAL library (there are many). I have imported all the .dll files generated by the nuget packages into Unity, and modified the configuration file to reflect the folder structure within Unity. But I am seeing the following error:

    Code (CSharp):
    1. DllNotFoundException: Assets/TerrainImporter/gdal/Editor/x64/gdal_wrap.dll
    2. OSGeo.GDAL.GdalPINVOKE+SWIGExceptionHelper..cctor () (at <42e4fcb1fc7b4dfbadeb49881bd1d5e2>:0)
    3. Rethrow as TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
    4. OSGeo.GDAL.GdalPINVOKE..cctor () (at <42e4fcb1fc7b4dfbadeb49881bd1d5e2>:0)
    5. Rethrow as TypeInitializationException: The type initializer for 'OSGeo.GDAL.GdalPINVOKE' threw an exception.
    6. TerrainImporter.Importer.InitializeImport () (at <6ebd26177eb5472ca39d545c9e7aba84>:0)
    7. TerrainImporter.TerrainImporterObjectEditor.InitializeImport () (at <6ebd26177eb5472ca39d545c9e7aba84>:0)
    8. UnityEditor.EditorApplication.Internal_CallDelayFunctions () (at C:/buildslave/unity/build/Editor/Mono/EditorApplication.cs:209)
    9.  
    I have researched this error and the most likely culprit is that one of the dependencies of gdal_wrap.dll is not being found, resulting in gdal_wrap.dll not being usable. However, using Visual Studios dumpbin command, I have found only these dependencies:

    gdal204.dll
    VCRUNTIME140.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    KERNEL32.dll

    gdal204.dll and VCRUNTIME140.dll are dll's included with the nuget package and have been imported into Unity. KERNEL32.DLL is a windows system file and shouldn't be an issue, while the rest should be present as well on the system. I have also verified the existence of the dependencies of gdal204.dll and VCRUNTIME140.dll.

    Basically, all the .dll files outputted in my bin directory are imported into Unity. Here is the configuration file created by the Nuget packages (edited slightly to account for the Unity folder structure):

    Code (CSharp):
    1. /******************************************************************************
    2. *
    3. * Name:     GdalConfiguration.cs.pp
    4. * Project:  GDAL CSharp Interface
    5. * Purpose:  A static configuration utility class to enable GDAL/OGR.
    6. * Author:   Felix Obermaier
    7. *
    8. ******************************************************************************
    9. * Copyright (c) 2012-2018, Felix Obermaier
    10. *
    11. * Permission is hereby granted, free of charge, to any person obtaining a
    12. * copy of this software and associated documentation files (the "Software"),
    13. * to deal in the Software without restriction, including without limitation
    14. * the rights to use, copy, modify, merge, publish, distribute, sublicense,
    15. * and/or sell copies of the Software, and to permit persons to whom the
    16. * Software is furnished to do so, subject to the following conditions:
    17. *
    18. * The above copyright notice and this permission notice shall be included
    19. * in all copies or substantial portions of the Software.
    20. *
    21. * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
    22. * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    23. * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
    24. * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    25. * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
    26. * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
    27. * DEALINGS IN THE SOFTWARE.
    28. *****************************************************************************/
    29.  
    30. using System;
    31. using System.Diagnostics;
    32. using System.IO;
    33. using System.Reflection;
    34. using System.Runtime.InteropServices;
    35. using Gdal = OSGeo.GDAL.Gdal;
    36. using Ogr = OSGeo.OGR.Ogr;
    37.  
    38. namespace TerrainImporter
    39. {
    40.     public static partial class GdalConfiguration
    41.     {
    42.         private static volatile bool _configuredOgr;
    43.         private static volatile bool _configuredGdal;
    44.         private static volatile bool _usable;
    45.  
    46.         [DllImport("kernel32", CharSet = CharSet.Auto, SetLastError = true)]
    47.         static extern bool SetDefaultDllDirectories(uint directoryFlags);
    48.         //               LOAD_LIBRARY_SEARCH_USER_DIRS | LOAD_LIBRARY_SEARCH_SYSTEM32
    49.         private const uint DllSearchFlags = 0x00000400 | 0x00000800;
    50.  
    51.         [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
    52.         [return: MarshalAs(UnmanagedType.Bool)]
    53.         static extern bool AddDllDirectory(string lpPathName);
    54.  
    55.         /// <summary>
    56.         /// Construction of Gdal/Ogr
    57.         /// </summary>
    58.         static GdalConfiguration()
    59.         {
    60.             string executingDirectory = null, gdalPath = null, nativePath = null;
    61.             try
    62.             {
    63.                 if (!IsWindows)
    64.                 {
    65.                     const string notSet = "_Not_set_";
    66.                     string tmp = Gdal.GetConfigOption("GDAL_DATA", notSet);
    67.                     _usable = tmp != notSet;
    68.                     return;
    69.                 }
    70.  
    71.                 string executingAssemblyFile = new Uri(Assembly.GetExecutingAssembly().GetName().CodeBase).LocalPath;
    72.                 executingDirectory = Path.GetDirectoryName(executingAssemblyFile);
    73.  
    74.                 if (string.IsNullOrEmpty(executingDirectory))
    75.                     throw new InvalidOperationException("cannot get executing directory");
    76.  
    77.                 executingDirectory = executingDirectory.Replace("TerrainImporter\\Editor", "TerrainImporter");
    78.                 // modify search place and order
    79.                 SetDefaultDllDirectories(DllSearchFlags);
    80.  
    81.                 gdalPath = Path.Combine(executingDirectory, "gdal");
    82.                 nativePath = Path.Combine(Path.Combine(gdalPath, "Editor"), GetPlatform());
    83.                 if (!Directory.Exists(nativePath))
    84.                     throw new DirectoryNotFoundException($"GDAL native directory not found at '{nativePath}'");
    85.                 if (!File.Exists(Path.Combine(nativePath, "gdal_wrap.dll")))
    86.                     throw new FileNotFoundException(
    87.                         $"GDAL native wrapper file not found at '{Path.Combine(nativePath, "gdal_wrap.dll")}'");
    88.  
    89.                 // Add directories
    90.                 AddDllDirectory(nativePath);
    91.                 AddDllDirectory(Path.Combine(nativePath, "plugins"));
    92.  
    93.                 // Set the additional GDAL environment variables.
    94.                 string gdalData = Path.Combine(gdalPath, "data");
    95.                 Environment.SetEnvironmentVariable("GDAL_DATA", gdalData);
    96.                 Gdal.SetConfigOption("GDAL_DATA", gdalData);
    97.  
    98.                 string driverPath = Path.Combine(nativePath, "plugins");
    99.                 Environment.SetEnvironmentVariable("GDAL_DRIVER_PATH", driverPath);
    100.                 Gdal.SetConfigOption("GDAL_DRIVER_PATH", driverPath);
    101.  
    102.                 Environment.SetEnvironmentVariable("GEOTIFF_CSV", gdalData);
    103.                 Gdal.SetConfigOption("GEOTIFF_CSV", gdalData);
    104.  
    105.                 string projSharePath = Path.Combine(gdalPath, "share");
    106.                 Environment.SetEnvironmentVariable("PROJ_LIB", projSharePath);
    107.                 Gdal.SetConfigOption("PROJ_LIB", projSharePath);
    108.  
    109.                 _usable = true;
    110.             }
    111.             catch (Exception e)
    112.             {
    113.                 _usable = false;
    114.                 Trace.WriteLine(e, "error");
    115.                 Trace.WriteLine($"Executing directory: {executingDirectory}", "error");
    116.                 Trace.WriteLine($"gdal directory: {gdalPath}", "error");
    117.                 Trace.WriteLine($"native directory: {nativePath}", "error");
    118.  
    119.                 //throw;
    120.             }
    121.         }
    122.  
    123.         /// <summary>
    124.         /// Gets a value indicating if the GDAL package is set up properly.
    125.         /// </summary>
    126.         public static bool Usable
    127.         {
    128.             get { return _usable; }
    129.         }
    130.  
    131.         /// <summary>
    132.         /// Method to ensure the static constructor is being called.
    133.         /// </summary>
    134.         /// <remarks>Be sure to call this function before using Gdal/Ogr/Osr</remarks>
    135.         public static void ConfigureOgr()
    136.         {
    137.             if (!_usable) return;
    138.             if (_configuredOgr) return;
    139.  
    140.             // Register drivers
    141.             Ogr.RegisterAll();
    142.             _configuredOgr = true;
    143.  
    144.             PrintDriversOgr();
    145.         }
    146.  
    147.         /// <summary>
    148.         /// Method to ensure the static constructor is being called.
    149.         /// </summary>
    150.         /// <remarks>Be sure to call this function before using Gdal/Ogr/Osr</remarks>
    151.         public static void ConfigureGdal()
    152.         {
    153.             if (!_usable) return;
    154.             if (_configuredGdal) return;
    155.  
    156.             // Register drivers
    157.             Gdal.AllRegister();
    158.             _configuredGdal = true;
    159.  
    160.             PrintDriversGdal();
    161.         }
    162.  
    163.  
    164.         /// <summary>
    165.         /// Function to determine which platform we're on
    166.         /// </summary>
    167.         private static string GetPlatform()
    168.         {
    169.             return Environment.Is64BitProcess ? "x64" : "x86";
    170.         }
    171.  
    172.         /// <summary>
    173.         /// Gets a value indicating if we are on a windows platform
    174.         /// </summary>
    175.         private static bool IsWindows
    176.         {
    177.             get
    178.             {
    179.                 var res = !(Environment.OSVersion.Platform == PlatformID.Unix ||
    180.                             Environment.OSVersion.Platform == PlatformID.MacOSX);
    181.  
    182.                 return res;
    183.             }
    184.         }
    185.         private static void PrintDriversOgr()
    186.         {
    187. #if DEBUG
    188.             if (_usable)
    189.             {
    190.                 var num = Ogr.GetDriverCount();
    191.                 for (var i = 0; i < num; i++)
    192.                 {
    193.                     var driver = Ogr.GetDriver(i);
    194.                     Trace.WriteLine($"OGR {i}: {driver.GetName()}", "Debug");
    195.                 }
    196.             }
    197. #endif
    198.         }
    199.  
    200.         private static void PrintDriversGdal()
    201.         {
    202. #if DEBUG
    203.             if (_usable)
    204.             {
    205.                 var num = Gdal.GetDriverCount();
    206.                 for (var i = 0; i < num; i++)
    207.                 {
    208.                     var driver = Gdal.GetDriver(i);
    209.                     Trace.WriteLine($"GDAL {i}: {driver.ShortName}-{driver.LongName}");
    210.                 }
    211.             }
    212. #endif
    213.         }
    214.     }
    215. }

    I call the following methods before trying to import a GeoTiff file:
    Code (CSharp):
    1. GdalConfiguration.ConfigureOgr();
    2. GdalConfiguration.ConfigureGdal();
    3. Gdal.AllRegister();
    I have attempted all the following to fix this issue:
    1. Moved all "plugin" dlls to a folder called Plugins, thinking perhaps Unity was not loading them for that that reason.
    2. Setting all "plugins" to "Load on Startup."
    3. Setting the Environment "PATH" variable to include the folder where all the dll's are located.
    4. Building my custom library with x64 as target.
    I have also found this thread:
    https://forum.unity.com/threads/gdal-dllnotfoundexception.107455/
    -user says they were missing dependencies, but doesn't say what those dependencies were.

    4 pre compiled non plugin libraries are also used. The code in these libraries actually call into the plugins. For example, like so:
    Code (CSharp):
    1. [DllImport("gdal_wrap", CharSet=CharSet.None, EntryPoint="CSharp___AllocCArray_GDAL_GCP", ExactSpelling=false)]
    2. public static extern IntPtr __AllocCArray_GDAL_GCP(int jarg1);
    Could there be an issue there? Some kind of incompatibility with how Unity needs things done?

    I feel like this is an issue with my use of plugins, not necessarily with GDAL, but I really have no clue. Does anyone have any suggestions?
     
    Last edited: Sep 11, 2019
  2. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,191
    Bump. Has no one run into any issues with using a plug-in in this way?
     
  3. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,191
    Okay, so in case anyone else comes across this problem, I think I have solved the issue. The problem lies in the configuration file. Basically, do not set any Environment variables related to the folder path where the DLL's are stored, as this screws something up with Unity so that it isn't able to find the Plugins. These particular code lines seem to be not needed:

    Code (CSharp):
    1. string executingDLLDirectory = executingDirectory.Replace("TerrainImporter\\Editor", "Plugins");
    2. nativePath = Path.Combine(executingDLLDirectory, GetPlatform());
    3. if (!Directory.Exists(nativePath))
    4.     throw new DirectoryNotFoundException($"GDAL native directory not found at '{nativePath}'");
    5. if (!File.Exists(Path.Combine(nativePath, "gdal_wrap.dll")))
    6.     throw new FileNotFoundException(
    7.         $"GDAL native wrapper file not found at '{Path.Combine(nativePath, "gdal_wrap.dll")}'");
    8.  
    9. var envPath = Environment.GetEnvironmentVariable("PATH", EnvironmentVariableTarget.Process);
    10. if (envPath != null && !envPath.Contains(nativePath))
    11.     Environment.SetEnvironmentVariable("PATH", nativePath + ";" + envPath, EnvironmentVariableTarget.Process);
    12.  
    13. ////Add directories
    14. AddDllDirectory(nativePath);
    15. AddDllDirectory(Path.Combine(nativePath, "plugins"));
    16.  
    17. string driverPath = Path.Combine(nativePath, "plugins");
    18. Environment.SetEnvironmentVariable("GDAL_DRIVER_PATH", nativePath);
    19. Gdal.SetConfigOption("GDAL_DRIVER_PATH", nativePath);
    Also, if you have a project with this code present and remove it, make sure you restart your Unity application.