Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Resolved How to use a Data Blackboard?

Discussion in 'Scripting' started by John_Leorid, May 18, 2020.

  1. John_Leorid


    Nov 5, 2012

    Last few days I've written a behaviour tree system with the new viewGraph tools and almost everything is set up except for the variables.
    Seems like blackboards are the way to go, but I've no idea how to use them.

    Just in case you don't know: Behaviour Trees only exist once in memory, so all the data of a tree is stored in the agent who is executing it, this data is called "Context" - so when the tree gets executed it looks like this

    Context, Blackboards? Where is the connection? Local data is not the only data needed, for example: I need to know where the player is located, this is global data, then if the enemy fortress/base is alerted, I call this "group data", because it only affects a group of enemies (so if one enemy fortress is alerted, enemies on the other side of the game world are not alerted, only the ones of this specific base).

    Currently my blackboard looks like this:

    Code (CSharp):
    1. public class DataEntitiy
    2. {
    3.     object storedValue;
    5.     public T Get<T>() {
    6.         T result = (T)storedValue;
    7.         return result;
    8.     }
    9.     public object Get() {
    10.         return storedValue;
    11.     }
    12.     public void Set(object value) {
    13.         this.storedValue = value;
    14.     }
    15. }
    17. public class DataBlackboard
    18. {
    19.     protected Dictionary<string, DataEntitiy> data = new Dictionary<string, DataEntitiy>();
    21.    public DataEntitiy this[string key] {
    22.         get {
    23.             DataEntitiy result;
    24.             if (data.TryGetValue(key, out result)) {
    25.                 return result;
    26.             }
    27.             result = new DataEntitiy();
    28.             data[key] = result;
    29.             return result;
    30.         }
    31.     }
    33. // Other GET and SET methods
    34. }
    Using it like this:

    Context has 4 of these Blackboards
    • BehaviourTree-Node specific data (e.g. a timer, only relevant for the node executing)
    • local data (data about the executing agent e.g. agentHealth)
    • group data (data of one enemy fortress e.g. lastTimePlayerWasSeen, isAlerted, NrOfAliveAgents)
    • global data (e.g. playerPosition, currentDaytime)
    Now it just feels wrong to set the data in an update loop, especially if I use Update() to set the Data like PlayerPosition and also Update() to execute the behaviourTree and therefor read the same data - agents could miss shots because they have the wrong information, which was set in the previous frame.
    Overall I am not happy setting the data like this, when it may not be needed at all ... is this even the correct approach or am I doing something wrong here?

    Note: Some VariableNames are created in the Editor, so I can't just write them all down in code and access them directly like
    , also it would break the decoupling and I may want to reuse the behaviour trees in another project.
    Last edited: Feb 10, 2021
  2. Cannist


    Mar 31, 2020
    I am sure the experts will have something to say on this. I myself am quite new to C#, Unity and game development. But I have been programming in other languages for ages.

    I wonder if perhaps you could use delegates to compute the information on demand if and when it is needed. The idea would be to store a computation in the dictionary that knows how to get the required information from the world. The computation would only need to be stored once and could be executed in the framea where the corresponding information is needed. Perhaps like this:
    Code (CSharp):
    1. public class InfoStore
    2. {
    3.     public delegate object DataProvider();
    5.     private class Data
    6.     {
    7.         internal DataProvider Provider;
    8.         internal object Value;
    9.         internal int LastUpdated;
    10.     }
    12.     private Dictionary<string, Data> dict = new Dictionary<string, Data>();
    14.     public void Add(string key, DataProvider provider)
    15.     {
    16.         Data data = new Data()
    17.         {
    18.             Provider = provider,
    19.             Value = null,
    20.             LastUpdated = Time.frameCount - 1
    22.         };
    23.         dict.Add(key, data);
    24.     }
    26.     public object Get(string key)
    27.     {
    28.         if (!dict.TryGetValue(key, out Data data) {
    29.             throw new InvalidOperationException("Cannot get data for non-existing key " + key);
    30.         }
    31.         // Is the data up to date?
    32.         if (data.LastUpdated != Time.frameCount)
    33.         {
    34.             data.Value = data.Provider.Invoke();
    35.             data.LastUpdated = Time.frameCount;
    36.         }
    37.         return data.Value;
    38.     }
    39. }
    (Add generics as needed.) You could then populate the
    instance with something like
    Code (CSharp):
    1. info.Add("PlayerPosition", () => thePlayer.transform.position);
    Depending on your needs there could also be variants that pass additional parameters to
    that would be passed on to the delegate call.
  3. Baste


    Jan 24, 2013
    Don't make it that generic. Don't make a blackboard for "any game". Make one for your game.

    Look, you will know what data your blackboard needs to contain. Don't have a Dictionary<string, object> that you cast to Vector3 just to store the player's position. Store the player's position in a variable named playerPosition.

    That's a lot more readable, and actually quite a bit faster (you're boxing the position a lot in the current implementation).

    Also don't be too worried about what the "correct" implementation of a BT is. Have your own implementation, and let it just have the data it needs, rather than wrapping it in a super-generic context object. That'll just make your code very cumbersome to use.
    sonmium, affablemattress and RogDolos like this.
  4. John_Leorid


    Nov 5, 2012
    Yes this works :) I actually ended up using Func<object> in my DataEntity class, whenever the Data is requested and if "useFunc" is TRUE, so it can be a storage OR a method pointer, which will return the desired result and it works fine ^^

    Problem here is that the user (thats me in this case xD) can create variables in the Editor, outside of Visual Studio, so there has to be this string/object based solution for now. I don't want to hop into visual studio every time I need need a new variable. Because one node can write to a variable, that another node can read. GoToPosition is probably the best example. If I search for various positions, e.g. patrol position, healing spot, ammo position, and then decide where to go in the tree, these are all variables, which do not exist in code - thats why I need a solution that works for both cases.

    But unless I run into perfomance issues, I'd consider this case closed. ^^ Thanks for the help :)
  5. John_Leorid


    Nov 5, 2012
    Well, here we are, 9 months later, on the day where our second gameplay-test-phase should start.

    I had the idea to use a string/object dictionary from someone who wrote this in C++, which probably works fine but accessing variables, hundreds of calls per frame, via string in C# causes massive Garbage. Every time a string is passed into the Get() method or into the dictionary[] indexer, the string is actually copied - and a string is basically a List, so it is saved as reference type on the heap memory, so the garbage collection has to clean all that clutter.

    The 30 enemies in our game create so much garbage per frame, almost 10 milliseconds, and it's only 30 enemies with rather simple behaviour - the amount and the complexity of their behaviour will increase, so I basically have to rewrite the whole system.

    To anyone stumbling over this - do not use string/object dictionaries for things you call a lot of times every frame

    So, back to refactoring, just thought I'd warn whoever comes by. ^^
    mopthrow likes this.
  6. Baste


    Jan 24, 2013
    I don't think a Dictionary<string, whatever> should allocate on the getter. It's just calling GetHashCode on your key. Passing a string instance around should also not copy it.

    strings are immutable, but that only means that methods that modify them return a copy. Any use that does not change it should be safe.

    Here's a test script:

    Code (csharp):
    1. using System.Collections.Generic;
    2. using UnityEngine;
    4. public class TestScript : MonoBehaviour {
    5.     public string s;
    6.     public int i;
    7.     public int i2;
    8.     private Dictionary<string, int> dict;
    10.     void Start() {
    11.         dict = new Dictionary<string, int> { {s, 5} };
    12.     }
    14.     void Update() {
    15.         dict[s] = Mathf.RoundToInt(Time.time);
    16.         dict["foo"] = Mathf.RoundToInt(Time.fixedTime);
    18.         i = dict[s];
    19.         i2 = dict["foo"];
    20.     }
    21. }
    The profiler results are:


    So your allocations are probably not coming from the accessor.
    kayroice and mopthrow like this.
  7. John_Leorid


    Nov 5, 2012

    Ok, thats interresting - but I don't have any new() or delegate calls in the System, strings are the only thing that is passed around - maybe it has to do with the calls?

    Maybe I don't have to rewrite the whole System? We'll see, further investigation is needed.
  8. Baste


    Jan 24, 2013
    6,342 allocates.

    This is because the name lives in the c++ engine, and has to be copied over to the C# runtime every time you get it.

    In general, do not get the names of things. If you're using the ScriptableObjects as keys in a Dictionary, use them directly instead of using their names, as their HashCode is just their instanceID, which is stored in a field, so that's both faster than using the strings to begin with, and does not allocate.
  9. John_Leorid


    Nov 5, 2012
    Yes, thanks a lot for the info, I will try this - more or less, maybe I cache the name-strings in a seperate dictionary and access them by the ScriptableObject-Reference because right now I have both cases, some where the blackboard variable is referenced via a ScriptableObejct in the BehaviourTree Inspector and some where they are references directly inside the code (for local variables of one node or lazy writing, when I just want the variable somewhere without having to assign the scriptableObject in the inspector, their names are unlikely to change).
  10. John_Leorid


    Nov 5, 2012
    I tracked it down - the result is ... well, unexpected, after about 8 years of game dev, how could I never know about this?
    Ok, so it took me about 4h to deconstruct the method, bit by bit, out-commenting everything and it turns out that assigning a struct to an object field, creates garbage, massive garbage to be more precise xD

    Here is the script I used for testing:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    5. public class GarbageTest : MonoBehaviour
    6. {
    7.     Vector2 _var1;
    8.     object _var2;
    10.     [SerializeField] bool _startTest;
    12.     void Update()
    13.     {
    14.         if (_startTest)
    15.         {
    16.             _startTest = false;
    17.             DoTest();
    18.         }
    19.     }
    21.     void DoTest()
    22.     {
    23.         TestMethod1();
    24.         TestMethod2();
    25.     }
    27.     void TestMethod1()
    28.     {
    29.         for (int i = 0; i < 100000; i++)
    30.         {
    31.             _var1 =;
    32.         }
    33.     }
    34.     void TestMethod2()
    35.     {
    36.         for (int i = 0; i < 100000; i++)
    37.         {
    38.             _var2 =;
    39.         }
    40.     }
    41. }
    And here is the screenshot of the profiler, set to deep profile

    you can test it yourself, if you don't beliefe this, I think this is quite crazy ...
    As you can see, the GC Alloc Field says:
    Method1 = 0 B
    Method2 = 2.3 MB

    we just created 2.3 Megabyte of garbage, with only 100.000 calls.

    Setting the iteration count to 1, returns 74 B Garbage for TestMethod2(), quite a bit for a Vector2 which should only have 8 Bytes and create no Garbage at all ...

    This basically breaks the whole system, I've absolutely no idea how to avoid this.
    30 Agents call this method, getting and setting variables about 2.500 times a frame, creating ~20 Kilobyte Garbage per frame.
    Thats pretty bad.

    Does anyone have any ideas how to deal with this?
  11. bobisgod234


    Nov 15, 2016
    John_Leorid and mopthrow like this.
  12. John_Leorid


    Nov 5, 2012
    Finally solved it by making
    generic ->
    and deriving from a class with the same name, just without the generic argument.
    No more garbage on a per Frame basis - well that was quite a ride. (hadn't time to fix it until now, since all the other changes I made, already stabilized the framerate - there were quite a few performance leaks in the code)

    Thanks @bobisgod234 for the keyword "boxing" ^^
  13. egemencelik


    Oct 25, 2019
  14. John_Leorid


    Nov 5, 2012

    Code (CSharp):
    1. public class DataBlackboard
    2. {
    3.     protected Dictionary<string, DataEntitiy> data =
    4.         new Dictionary<string, DataEntitiy>();
    7.     public DataEntitiy<T> Get<T>(string key)
    8.     {
    9.         DataEntitiy<T> result;
    10.         if (data.TryGetValue(key, out DataEntitiy resultRaw))
    11.         {
    12.             return resultRaw as DataEntitiy<T>;
    13.         }
    14.         result = GenericPool.Get<DataEntitiy<T>>();
    15.         data[key] = result;
    16.         return result;
    17.     }
    18. }
    Code (CSharp):
    1. public abstract class DataEntitiy
    2. {
    3.     public abstract void ReturnToPool();
    4.     public abstract DataEntitiy GetCopy();
    5. }
    6. public class DataEntitiy<T> : DataEntitiy
    7. {
    8.     T _storedValue;
    10.     public T Value
    11.     {
    12.         get => _storedValue;
    13.         set
    14.         {
    15.             _storedValue = value;
    16.             OnSetEvent?.Invoke(this);
    17.         }
    18.     }
    20.     public delegate void onSet(DataEntitiy<T> entity);
    21.     /// <summary>
    22.     /// returns the entity when set, so Get<T>() can
    23.     /// be used to get the prefered type
    24.     /// </summary>
    25.     public event onSet OnSetEvent;
    27.     public DataEntitiy() { }
    28.     public DataEntitiy(T value)
    29.     {
    30.         _storedValue = value;
    31.     }
    33.     public Type GetTypeOfValue()
    34.     {
    35.         return _storedValue.GetType();
    36.     }
    38.     void ResetValue()
    39.     {
    40.         _storedValue = default;
    41.     }
    42.     public override void ReturnToPool()
    43.     {
    44.         ResetValue();
    45.         OnSetEvent = null;
    46.         GenericPool.Return(this);
    47.     }
    49.     public override DataEntitiy GetCopy()
    50.     {
    51.         DataEntitiy<T> copy = GenericPool.Get<DataEntitiy<T>>();
    52.         copy._storedValue = _storedValue;
    53.         copy.OnSetEvent = OnSetEvent;
    54.         return copy;
    55.     }
    56. }
    Code (CSharp):
    2. public static class GenericPool
    3. {
    4.     private static Dictionary<Type, object> _pool = new Dictionary<Type, object>();
    5.     public static T Get<T>()
    6.     {
    7.         if (_pool.TryGetValue(typeof(T), out object value))
    8.         {
    9.             Stack<T> pooledObjects = value as Stack<T>;
    10.             if (pooledObjects.Count > 0)
    11.             {
    12.                 return pooledObjects.Pop();
    13.             }
    14.         }
    15.         return Activator.CreateInstance<T>();
    16.     }
    17.     public static void Return<T>(T obj)
    18.     {
    19.         if (obj == null)
    20.         {
    21.             return;
    22.         }
    23.         if (_pool.TryGetValue(typeof(T), out object value))
    24.         {
    25.             Stack<T> pooledObjects = value as Stack<T>;
    26.             pooledObjects.Push(obj);
    27.         }
    28.         else
    29.         {
    30.             Stack<T> pooledObjects = new Stack<T>();
    31.             pooledObjects.Push(obj);
    32.             _pool.Add(typeof(T), pooledObjects);
    33.         }
    34.     }
    35.     /// <summary>
    36.     /// only required when fast play mode options are active
    37.     /// </summary>
    38.     [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)]
    39.     private static void DomainReset()
    40.     {
    41.         if (_pool != null) _pool.Clear();
    42.     }
    43. }
    Last edited: Mar 24, 2022
    chrisall76 and xenotime like this.
  15. lilbud_evan


    Dec 16, 2017

    This is really helpful! I think I got a better understanding of it but there's still some parts that I don't get.
    Could you explain why you're using a "GenericPool" and what that does in the context of the rest of the code?

  16. John_Leorid


    Nov 5, 2012
    To avoid garbage. Further up this thread I talk about boxing - a valid way to avoid it is by using a custom wrapper. And I have to pool the wrapper to avoid generating garbage.

    The real DataBlackboard class has a remove method and there I return the classes to the pool. Given the code here, it wouldn't matter if I would create the DataEntities with new(), as all of them are saved in the dictionary anyway.