Optimized Save to CSV

I'm working on a project that requires to save to CSV file for editing many values at once, from a 2d array string[,] . Currently using the old save to CSV file found on the wiki, and it's extremely slow which is causing problems. Theres about 3500 entries (I know it's big for CSV, but it's required) and it's taking about 3 minutes per save. So I need to find a more optimized method...

I found this post here - http://www.codeproject.com/Articles/685310/Simple-and-fast-CSV-library-in-Csharp - but I can't get it working inside Unity.

I also found another method here - http://www.codeproject.com/Questions/453137/Save-multidimensional-array-to-CSV-file - and Amit Gajjar has a really fast method (maybe 2 seconds) but the problem is, it only saves to 3 columns, and I can't quite figure out how to get it to save as many columns as the CSV file has, rather than a static 3 columns. It seems to have something to do with the fact that it was created for a different form of data than string[,] but I can't figure out how to get it convered to a simple 2d string array.

Could anyone help me out to get either of these samples working inside Unity? Or if theres another method that takes seconds to save, rather than minutes, it would be greatly appreciated.

Thanks!

(Edit: Current method is taking 5 minutes! Any assistance would be much much appreciated! Thanks!)

 

By csv your just talking about comma separated values, right? Like this:

Code (csharp):

1. bob,32,ca
2. jim,55,nv

I assume you want it for some database software such as Postgres?

 

Yes.

Here is the script I'm currently using for saving:

Code (csharp):

1.  
2. static public void SaveOutputGrid(string[,] grid)
3.     {
4.         string textOutput = "";
5.         int upperBound = grid.GetUpperBound(0) - 1;
6.         for (int y = 0; y < grid.GetUpperBound(1); y++)
7.         {
8.             for (int x = 0; x < grid.GetUpperBound(0); x++)
9.             {
10.  
11.                 textOutput += grid[x, y];
12.                 if (x != upperBound)
13.                     textOutput += ",";
14.             }
15.             textOutput += Environment.NewLine;
16.         }
17.         System.IO.File.WriteAllText("CSVData.csv", textOutput);
18.     }
19.  

This is veeery time consuming though. As I mentioned before for 3500 rows it's taking about 3 minutes (Edit: Just tried it out and it's even worse than I remembered... Set it up to debug.log the time and prior to the loops it was 7:21:46 PM, after the loop it was 7:26:42 PM....)

These other methods seem so much faster - taking seconds to save - I just can't get them working for the format needed (input of string[,] )

I'm begging anyone to please help... 5 minutes is just crazy and I KNOW it's possible in seconds, just having trouble getting these other methods converted to strings...

 

I'll be happy to read your code, but it makes it a lot easier with code tags

 

Fixed the tags... and added some more information. Thanks so much!

 

Try it with a string builder.

 

I'm not entirely familiar with C#, and this kind of thing can vary with different compiler optimisations, but things like "text += moretext" under the hood will allocate a new string large enough to hold the result, copy the entire "text" into it, copy "moretext" onto the end, then free the entire original "text". As "text" gets bigger and bigger, the copying takes longer and longer. A quick fix to your existing code would be to build an array of lines and write it out with WriteAllLines(), though Open()'ing a file object and writing line by line would be even better.

 

A string builder worked *amazing*.

Took the time from 5 minutes down to 4 seconds!

Thanks so much!

 

By the way, the code you posted won't work right unless you change GetUpperBound to GetLength.

--Eric