Search Unity

  1. We want you to join us at GDC this year! We've added new sessions to Unity Central, space is limited so sign up now!
    Dismiss Notice
  2. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  3. Unity 2017.3 has arrived! Read about it here.
    Dismiss Notice
  4. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  5. Tell us about your experience here and you’ll get early access to the 2018 Game Studios report + more goodies.
    Dismiss Notice
  6. Be the first to take a peek at upcoming 2D Animation Preview. Drop into the forum and check it out!
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Bug with StringReader speed? Or am I missing something?

Discussion in 'Scripting' started by johot, Sep 27, 2011.

  1. johot


    Apr 11, 2011
    So I have a big textfile with words that I need to parse. To do this I have them all in a TextAsset, it's about 500 000 lines of words, about 8 mb.

    Now I want to parse this file, line by line.

    I've tried three different ways.

    1. String.Split - this is really slow and I can't use it... Using the text property of the text asset here (so a big string).

    2. StreamReader using a MemoryStream and using the bytes property of the textasset (so byte[]). This is pretty fast, about 10x as fast as method 1.

    3. StringReader... this never even finishes. Do we have a bug here?

    Have anyone any experience with the StringReader? Is there a bug in it or something? Because it's really slow! I would think it would be about as fast as nr 2?

    Why would I think it is a bug then? Because if I loop through the characters of the file (using the textasset text property) and using a StringBuilder and basically simulating what the StringReader should do (or so I think anyway) I get performance about the same as method 2.

    You can test this easily by creating a big textasset file and then trying to read it line by line using a StringReader.

    Code (csharp):
    2. StringReader reader = new StringReader(aVeryLongString);
    3. string line;
    5. while ((line = reader.ReadLine()) != null) {
    6. }
  2. EddieCam


    Oct 28, 2009
    Old thread, but for anyone else googling, there is a bug with Mono's current StringReader.ReadLine method, as detailed here

    Basically, if a string is using Unix line endings it will search the entire string each time you call ReadLine, to look for Windows line endings too. O(n^2)!

    As Johot says, you may want to write your own ReadLine type method using stringbuilder, for any text file with >1000 lines.
  3. made-on-jupiter


    May 19, 2013
    Still seems an issue with Mono 4.0.1.

    If all line endings in the string are of the same type, this will do a quick job at replacing Unix line endings with Windows ones (but leave Windows ones be):

    if (Regex.Match(text, "\r?\n").Length == 1)
    text = text.Replace("\n", "\r\n");