Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Is there a solution for big entities?

Discussion in 'Entity Component System' started by JakHussain, Jul 6, 2019.

  1. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    I've just started trying to use Ecs for the first time and in my case, I need some of my entities to contain a collection of 8760 shorts each. The issue is that this apparently exceeds the 16kb memory limit for individual entities but I can't use a data type smaller than a short and I can't have any less than 8760 values. How can I get around this without over complicating this by splitting the data across 2 entities?

    Also, its seems that a component data can't use a native array to store collections which I find really odd. How should I be storing my large collections in entity components if not with a native array?
     
  2. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    1,154
  3. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    I came across dynamic buffers but setting the buffer capacity to 8760 gave me errors that my data was too large.

    By setting the capacity to zero, you say the values I add will allocate outside of the chunk but then where will they go? And how does that affect performance if ecs is designed around processing data in chunks?
     
    Tony_Max likes this.
  4. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,753
    I quote 2 posts on DynamicBuffer and NativeArray

    I just bring 2 quotes,
     
    digitaliliad and Tony_Max like this.
  5. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    1,154
    I don't know this for sure, but I would assume that you have to do 1 memory jump for each buffer not allocated in the chunk (non linear), but after that it should be linear access again. So pretty much neglectable and the only way to do it anyway. I was asking a long while ago if they consider user defined chunk sizes and I think Joachim said, that they might in the future.
     
  6. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    That makes sense, so I can fill my chunks to the absolute brim with my buffer size to get the most performance I can at let the memory jump to the heap happen for the elements at the end. That's super flexible!
     
  7. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    1,154
    this is not how i understand it though. I think it is black or white.
    1. whole buffer fits in chunk -> allocated in chunk
    2. whole buffer does not fit in chunk -> chunk has pointer to memory location outside of chunk & whole buffer resides there (i.e. if it starts in the chunk but then outgrowth the chunk, the whole buffer is copied to a new memory location outside the chunk)

    But that should be fine, when you access the buffer, instead of reading the first element, you jump to a memory location, but then it should be linear again...
     
  8. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    1,154
    possibly best for someone at Unity, to provide this kind of info in the documentation, it's repeating FAQs
     
    JesOb and mannyhams like this.
  9. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,752
    It's also worth noting that if you allocate buffer inside chunk, as soon as you resize the buffer, even just adding 1 element, the entire buffer is moved outside of the chunk. The only way to keep it in your chunk is to allocate it and treat it like an array.
     
  10. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    318
    So to summarise, it isn't a solution but a compromise. An entire big entity will be allocated on to the heap but will still be in a tight and ordered memory layout so that accessing the data is still faster than not using Ecs, but not quite the physical speed limit of the DOTS system running on that hardware. Is that correct?
     
    NotaNaN likes this.
  11. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,752
    Only the buffer will be allocated outside of the chunk.

    From my own personal testing, I can't measure any notable performance difference in real world conditions in any of my dozens of systems when the buffer is allocated within or outside the chunk.

    There will obviously be certain cases or benchmarks where you could notice a difference, but I'd be shocked if there wasn't a magnitude of other optimizations you could make before it'd be a consideration in your coding. i wouldn't even give it a second thought.