Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Read data frequently from a huge csv file

Discussion in 'Scripting' started by Jophiel17, Oct 24, 2021.

  1. Jophiel17

    Jophiel17

    Joined:
    Mar 14, 2021
    Posts:
    1
    I am making a game that needs to access a few lines (10 lines) of data from a huge (2GB) CSV file frequently (around twice a day). I don't want to add all the data in the game, so I am thinking of uploading it to Google sheets and reading it from there.

    My questions are:
    1. What are the best ways to get my task done?
    2. If uploading it to Google sheets is ok, how to read data from Google Sheets and use it in the game? (Hoping to get the script to read the data)
    Thankful for any help.
     
  2. exiguous

    exiguous

    Joined:
    Nov 21, 2010
    Posts:
    1,749
    So you want your user to download 4GB per day? Thats around 120GB per month assuming he/she is playing daily. Thats quite a strain for any internet connection. Don't you think?

    Is the data changing? If it is static just put it in the build. Also if user only needs a small amount (10 lines) of the data let your app only download these. There is no need to load the whole stuff if you discard most of it anyway.
     
  3. PraetorBlue

    PraetorBlue

    Joined:
    Dec 13, 2012
    Posts:
    7,890
    I think your needs have evolved beyond a csv file.

    This is the realm of relational databases. If you put your data in a table in a SQL database and index it properly, you will be able to very quickly and efficiently query the appropriate small number of rows as needed from the table.