Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice

Potential math bug

Discussion in 'Scripting' started by raybritton, Dec 22, 2015.

  1. raybritton

    raybritton

    Joined:
    Apr 3, 2015
    Posts:
    4
    I'm sure I'm wrong and it's a simple mistake but:
    I'm using the formula: x + (y * width), to convert from a multi to single dimension array.

    In this case the values were
    x = 3
    y = 1
    width = 10
    and the result should have been 13 but it came out as 12.

    I've attached a screenshot of the process in debug (ignore the exception type) and the code below.

    Width is a constant set to 10

    Code (csharp):
    1.  
    2. public int GetCoord(Vector3 vector){
    3.    int value = (int) (vector.z + (vector.x * width));
    4.    if (vector.x * 10 + vector.z != value) {
    5.       throw new System.IndexOutOfRangeException();
    6.    }
    7.    return value;
    8. }
    9.  
     
  2. StarManta

    StarManta

    Joined:
    Oct 23, 2006
    Posts:
    8,744
    Use Mathf.RoundToInt instead of casting to int. I bet you a dollar that "(vector.z+(vector.x* width))" is returning something like 12.99999, and simply casting to an int makes floats round down.
     
    raybritton likes this.
  3. raybritton

    raybritton

    Joined:
    Apr 3, 2015
    Posts:
    4
    I tried
    Mathf.RoundToInt(vector.z+(vector.x* width))
    and
    ((int)vector.z) + (((int)vector.x) * width)

    but neither work, I should mention it only seems to happen most of the time.
     
  4. StarManta

    StarManta

    Joined:
    Oct 23, 2006
    Posts:
    8,744
    If you're judging whether it works by this:
    Then you're having the exact same issue twice. It'll once again give you a result like 12.9999, even if 'value' is now correctly 13 it'll report that it's wrong.
     
    raybritton likes this.
  5. raybritton

    raybritton

    Joined:
    Apr 3, 2015
    Posts:
    4
    Yes, I removed that and checked by just logging the results of calling GetCoord() repeatedly
     
  6. raybritton

    raybritton

    Joined:
    Apr 3, 2015
    Posts:
    4
    Ah, nevermind, apparently I implemented your fix incorrectly the first time. It's working fine now that I've written it again.
    Thank you.