BG Database (inMemory database | Excel/Google Sheets syncing | CodeGen | Save/Load support)

I encountered an error with the settings file that may be a bug. We store the banshee settings file in our git repo (to share google sheet import and other settings). After I moved the settings file using the banshee settings UI, and checked it into the repo, other clients can no longer open any settings file. In Settings/Main/Settings file, there is a single error message that says "There was an error while loading settings file: Editor settings file does not exist at path (original location)." They no other options to open any settings file, or generate a new settings file.

The fix is easy, though clumsy, which is to manually move the new file to its original location, which banshee notices immediately, then use the banshee UI to "open file" with the new location.

It seems that if the settings file cannot be found, the various file option settings should be shown by default.

What is the best practice for sharing the settings file, and where should it be optimally located?

 

Indeed, there is a bug, which does not let you choose setting file location if it was not found
We will redesign GUI in the next release, thank you for letting me know about this issue!
To be honest, I think the current algorithm for resolving setting file location is not good, but we can not come up with a better alternative
Currently, we store setting file location in editor prefs locally, so once you open a project on a new computer, setting file location can not be resolved.
It seems like there is no way to find the setting file location, cause we do not have any information about which file to use (there could be multiple settings files in a project)
We also provide an option to store settings file outside of the project folder, cause it can have sensitive information(Google Sheets or machine translation settings)
Probably we could implement a search for all settings files inside the project folder so a user can choose which file to use to mitigate this problem, but we can not figure out how to auto-find it.

 

just started trying to use the Database system,
i get this errors
Cannot convert type 'BansheeGz.BGDatabase.BGEntity' to 'D_SkillDmgShots'
Cannot implicitly convert type 'D_SkillDmgShots' to 'BansheeGz.BGDatabase.BGEntity'

iam using the monobehavior codegen
and i have a table that has a relationSingle to another table
(skill stats referencing dmgshots)

 

Hello, just wanted to let you know that there's no overhead when reading assets on runtime in build, probably only in editor as far as i can see. I'm using the preloader for reading the assets, why do you don't recommend using it?

 

Hello,
Most probably you use the same D_ "Class names prefix" for both CodeGen addon (Addons->CodeGen) and for MonoBehaviour classes generator (Settings->Code generation), please, see the screenshot below under spoiler tag

Spoiler: D_ classes names prefix screenshot

so generated classes conflict with each other, cause they have the same names

To fix it, please, assign a different prefix to one of the generators ( E_ for example)
Also, if you choose to change the prefix for MonoBehaviour classes generator (Settings->Code generation), please, locate previoulsy generated C# files with D_ prefix in Project window and delete them (see the screenshot below)

Spoiler: Locate generated classes screenshot

We will add a check in the next release, so the error will be raised if two code generators have the same classes names prefixes, sorry for the inconvinience

 

As far as I know, Addressables system can behave differently inside Unity Editor and in builds.
It has a "Play mode script" parameter.
If it's set to "Use Asset Database", different API is used to load assets

To emulate realtime behaviour in Unity Editor you need to set it to "Use Existing Build" (not sure about Simulate Groups).
This also requires you to build/update Addressables built groups if you change any addressables settings, using Build->NewBuild->Default Build Script or Build->Update a Previous Build

The main concern about preloading technique is that it relies on internal Addressables cache, which returns an asset synchronously after the asset was loaded for the first time.
It seems like it works as needed (judging by the source code of the current version) and loaded assets can not be removed from the cache (at least until reference counter reaches zero), but still there is some risk involved when you use a preloader script.
On the other hand, using BGAddressablesGetAsset asynchronous action https://www.bansheegz.com/BGDatabase/Downloads/RuntimeAddressablesPlaymaker does not involve any additional risk and it should be as robust as Addressables system is

Also, a couple of minor concerns:
1) If you use preloader script, you need to wait until assets are loaded and proceed only after assets were loaded.
It means you need to wait until "Completed" method is called to proceed further.
It's not enough to simply attach the preloader script to GameObject you also need to wait until assets are loaded
2) By default preloader script loads all assets from the database.
You can filter out non required assets if you want to (using "IsIncluded" methods), but it involves some C# scripting and probably additional setup (you need to know which assets to filter out)
I'm not sure if loading all assets at once makes sense- cause probably it has the same effect as using Resources folder (I'm not sure about it)

 

@BansheeGz

I don't really get how the BG Addressables Get Asset action work. For example, i have an Enemies Meta, and the field has EnemyPrefab, EnemyElement1, EnemyElement2 and so on as well as Squadrons Meta with a lot of fields too, is there a way to iterate through all the fields of the meta? I see that it is not for preloading but just for getting data asset by asset, field by field. That's kinda over the top for my kind of game. I mean it's not small, but it's kinda too late now to reimplement that kind of resource loading for every prefab.

As for preloading GameObject (that type of object is the only thing i use addressables for), i just use Addressables.LoadAssetsAsync<GameObject> ("LabelName", null); (don't need to put in a callback, it goes quick) and it works in build provided i use existing build for addressables which, yeah, can be a kind of drag when you change stuff.

Also, there's a problem i have. I have some arrays that i populate by iterating through the rows with Get By Index action when i enter the level.

When i load scene regularly (good old fashioned SceneManager.LoadScene), takes a while, but everything is ok.

But when i do it the fast way with firing up async loading the level in the main menu (SceneManager.LoadSceneAsync) and then activating the scene i get nothing when i try to get something from the database. I tried with both yours and my preloader.

Any ideas? I thought addressables were going to be a bit more high level than this

 

cool i got it sorted, i was a little confused by the documentation, i guess yeah i can have both codegens huh
i was confused thinking i had to choose one or the other and being like WUT ... got it sorted now though, good database tool
would be cool if there was a modifier system built into it
like databox has a modifier system and runtime editing
but this excel styled system is waaaay better imo

 

It's just a wrapper around Addressables.LoadAssetAsync method
It takes an address from the database and calls Addressables.LoadAssetAsync to load an asset

Most probably preloading technique should work fine
I've also attached "BG Addressables Get Assets" (note 's' at the end), which allows to load multiple fields at once to an array just in case you might be interested

Spoiler: screenshot

Probably using a callback is a safer option

Code (CSharp):

1.         Addressables.LoadAssetsAsync<GameObject>("LabelName", null).Completed += handle =>
2.         {
3.             //proceed further
4.         };
5.  

You preload the assets and then use generated Playmaker actions to read them, correct?
And when you use SceneManager.LoadSceneAsync assets fields return null when your code is executed?
Maybe your code is executed before assets are preloaded?
This is the most obvious reason
Could you please ensure that your code is executed after preloading is finished?
When you call Addressables.LoadAssetsAsync, it may take several frames to load the assets, you can use a callback to determine the moment all assets are loaded:

Code (CSharp):

1.         Addressables.LoadAssetsAsync<GameObject>("LabelName", null).Completed += handle =>
2.         {
3.             //assets are loaded at this point and can be accessed
4.             Debug.Log("Assets are loaded!");
5.         };
6.  

 

Hi! I found probably a bug, If I call adressables Get Asset on Bolt, inside a "for loop", I get this error, If I put a "wait for next frame" in the loop, then its all fine,...

 

Thank you a lot for your support and suggestions!
We could try to implement these features as plugins
I will contact you as soon as they are ready

 

Hi, thank you for reporting the issue!
Could you please, let me know:
1) Which Unity version you use?
2) Which Bolt version you use?
3) Would you mind sending me a full screenshot of your flow graph?

 

@BansheeGz

Yeah, i read them with generated Playmaker actions.

Wll, you were right old chap, i overestimated the speed the addressables are loaded, so i needed that callback after all. The thing i didn't know is that the async loading of the scene that uses the addressables must be AFTER the addressables that it uses are fully loaded.

 

Yes, if you use preloading technique
I'm glad the issue was solved!

 

Thanks for the help and the additional action!

 

Unity 2019.4.11f1
Bolt 1.4.13
DGDatabase 1.5.11

I´m dealing with this error in more than one place in my project and solved it with similar solutions, like waiting for next frame, or avoiding to call addressables Get Asset on several GOs at the same time. But I´m afraid the project is too complicated to show one or two graphs as example, I´v tried to reproduce in a new scene, but with no luck.

 

Thank you for the information!
We also were not able to reproduce it in a new scene
I've asked our dev to investigate this issue- there should be some reason why it occurs
I will contact you as soon as I have new information

 

That's great news, I should mention that the "addressables Get Asset" is being called inside flow macros in both scenarios I´v had the issue.

 

1) We investigated the issue and it turned out to be quite easy to replicate.
To replicate it we accessed the asset before the unit is executed, e.g. before "exit" out control is executed (please, see the screenshot below)

Spoiler: screenshot

The reason why value can not be fetched dynamically is because this unit is asynchronous (just like Addressables.LoadAssetAsync method it uses), and it may take several frames to load the asset.
"Exit" out control is executed right after the asset was loaded- so asset should be accessed after that moment

Spoiler: correct workflow

2) There is an alternative method, which allows reading assets synchronously (using generated classes for Bolt and a preloader script). It seems to work correctly but we have concerns about its reliability

Please, let me know if you have any questions

 

Hello:

When using Excel extension, is it possible to import images from a spreadsheet to the database (as a UnitySprite for example)?

Also, I'd like to suggest the ability to communicate with column that have spaces and special characters in their titles. Perhaps a workaround may be to designate each field a special index value that maps to a column. So, meta may have field with name "Color" and some special index of "3". It would then always import from the third column in the spreadsheet, even if title of that column were something like "Pantone #". Perhaps there are other, better, ways.

Thank you for your time and your creations!

 

@ForeverSublime Thank you for your suggestions!

We could try to implement it, but it may require special setup

1) First of all you'll need to ensure that each image is properly anchored to correct cell.
Upper left corner should be inside target cell, so we could map it to the target row/field (please, see the screenshot below)

Spoiler: image anchor

2) We do not store any assets inside the database, we store the references to the assets
It can be a file path (for Resources loader) or Addressables address or GUID (for Addressables loaders)
So when you import a new image- we will have to update Excel cell value with corresponding asset reference to keep a link between excel image and the target asset, so next time you import the data- existing asset will be updated instead of creating a new one
So after the first import the excel file content will be like this (assuming Resources loader is used):

Spoiler: after first import

With addressables, we need to know the target addressables group as well
Also, probably, you will need to provide the target directory inside your project to create/search for the sprites

3) These asset references can be changed. For example, you can change file name/path, change addressables address or delete meta file to generate new GUID (database does not keep track of these changes). In this case, we could skip the sprite and add an error to the log or generate a new sprite

4) Excel does not seem to keep original file names, so we will need to generate new ones (using name field for example)

Please let me know what you think about it

Every field has a unique ID, which can be used to reference it

Spoiler: unique field ID

Maybe we could use a comment to assign field ID to the excel header cell?

Spoiler: comment to store field ID

In this case, you can use any name as a header without a need to concern about breaking naming convention.

 

Wonderful! having those two exits will be amazing, thanks again. Will let you know if I got any more issues.
Those are available in the latest release?

 

Thank you for your reply, @BansheeGz. I will think on the solutions and problems a bit more.

Regarding the field IDs as a comment: For me, personally, I don't know if it would be a solution I'd use, because I am trying to avoid as much data input as possible. Oh, I get my spreadsheets from other vendors, so that is why they are not named acceptably in the first place. So, I'd probably opt to rename the cells if I'm going to input data. In fact, I think I'll save a file where each row is a template of acceptable column names - and whenever I get a spreadsheet from a vendor I'll just copy and paste the appropriate template. It's not ideal, but for now would be fine. Perhaps other solutions will come to mind between now and then.

The comment idea is very good in general, though. If it's a solution you can only attribute to a single issue, maybe it's a good idea to hold onto that solution for a bigger problem. Ha!

Thank you again.

 

Please, contact me immediately in case of any issue
Two exits are available in the latest plugin release https://www.bansheegz.com/BGDatabase/Downloads/RuntimeAddressablesBolt/
Also, latest BGDatabase package has builtin notifications about available updates for installed plugins

Spoiler: plugins update notification

 

Sure, please, contact me when you are ready to discuss it

We could create a mapping config for a data source, which maps database table/fields to excel sheets/headers names
Database table->Excel sheet name
Database field->Excel header cell value
This solution does not require an excel file editing (if excel file has a proper layout)

Spoiler: mapping config for excel

 

^That looks amazing! With a feature like that, I think a lot of people could benefit for different types of projects.

From my perspective, the benefit of mapping without editing is it would likely help people more often use this little tool we're building. If all our coworkers have to do is save their spreadsheets in a particular folder and give the data source a certain name, it is very easy for them to use weekly/monthly, and they avoid having to ask someone to modify their spreadsheets for them if they don't feel comfortable.

Thanks for reading my little story.

 

I agree- we will include it to the next release, thank you for your suggestions!

 

After Upgrading my plugins I´m having this error:

 

Two different methods to fix this error:
1) If you do not use a preloader script- delete it Assets\BansheeGz\BGDatabaseAddressables\ScriptsNoDll\BGAddressablesPreloader.cs
2) Upgrade Addressables package via Package Manager (Window->Package Manager) to the latest version (v.1.16.7)

 

Perfect !

 

@LuiBroDood

Hello,
We've released runtime editor https://www.bansheegz.com/BGDatabase/Downloads/RuntimeEditor/

 

Hi, I´m using the new version and overall the issues disappeared, but still, sometimes randomly this still happens:



Sorry for not having how to give a better idea of the reasons.

[EDIT] I believe its related to calling AdressablesGetAsset in several Objects at the same time
[EDIT2] I´m avoiding the issue, using an "WaitUntil" unit to assure the execution of each object before the next.

 

Hi, indeed, the current implementation does not support simultaneous invocations
We will try to revisit it and create a better alternative using events.
Unfortunately, Bolt does not support delegates/callbacks, which would be an ideal solution https://support.ludiq.io/communities/5/topics/1120-delegates-callbacks
I will contact you as soon as I have any new information

 

Hi
We've implemented an alternative method for loading assets using events, which is better suited for loading multiple assets simultaneously and it does not require coroutine mode.
I've attached an upgraded plugin package to this message.
A new version is available for download here: https://www.bansheegz.com/BGDatabase/Downloads/RuntimeAddressablesBolt/

Loading is split into 2 phases:
1) Requesting an asset loading (using BansheeGz/BGAddressablesGetAssetViaEvent unit)
2) Once the asset is loaded, an event is fired (Events/BansheeGz/BGAssetLoadedEvent) with the asset as an argument. Additional information is also provided so the origin of the asset can be identified (table, field, entity, optional eventData object)

Spoiler: BGAddressablesGetAssetViaEvent screenshot

BGAssetLoadedEvent has optional eventName parameter, which can be used to filter out events with different names (similar to how CustomEvent works)

Spoiler: EventName usage screenshot

I hope we did not miss anything

 

@BansheeGz Hello! I can't work because have error in BG Database after import asset package:

Code (CSharp):

1. Assets\BansheeGz\BGDatabase\Scripts\Misc\Util\BGUtilForUi.cs(35,40): error CS0121: The call is ambiguous between the following methods or properties: 'BansheeGz.BGDatabase.BGRectTransformExtension.GetWidth(UnityEngine.RectTransform)' and 'BansheeGz.BGDatabase.BGRectTransformExtension.GetWidth(UnityEngine.RectTransform)'

How I can solve this problem?
Unity 2020.1.14f1

 

Hello!
Most probably you extracted the package(s) with C# source code files, and now they conflict with the classes inside DLL file(s). Source files packages contain C# classes, which were used to build DLL files

Two different methods to fix the issue:
1) Method #1: Delete all subfolders (if any) of the following folders (please, see the screenshots below):
1.1) Assets\BansheeGz\BGDatabase\Scripts
1.2) Assets\BansheeGz\BGDatabase\Editor\Scripts

Spoiler: folders to delete

2) Method #2: Do clean import of BGDatabase asset, i.e.
2.1) Delete BansheeGz\BGDatabase folder
2.2) Import BGDatabase asset from the Asset Store

 

@BansheeGz Thank you!

 

Hi,
I'm using a Excel file with 3 different metas with same configuration. I will be changing the meta at run time and picking the values from there and assigning it to my GameObjects. I have properly loaded plugin "Excel file monitor for runtime".
I have a gamobject with BGExcelImportGo script with Monitor file option ON.

I have written the code something like this.
private BGMetaEntity table;
private BGRepo repo;

repo = BGRepo.I;
excelSheetName = System.IO.Path.GetFileNameWithoutExtension(_videoFiles[curVideoIndex]);
table = repo[excelSheetName];
//I'm incrementing calculatedExcelRow
PoleNumeratorText.text = table[calculatedExcelRow].Get<String>("OHE_Numerator").ToString();

I'm getting the expected values but all runtime changes are not getting reflected.
I don't want to use any binder and not using CodeGen also.
Please guide me on this.

 

Hi,

1) We have added OnAfterImport Unity event to BGExcelImportGo component which can be used to run your own code after import
I've attached updated BGExcelImportGo.cs script to this message
Use this event to transfer the values from database to your GameObjects

Spoiler: New OnAfterImport event

2) I've also attached an example project (ExcelRuntime.zip), which demonstrates how OnAfterImport event can be used
Import BGDatabase package to this project and run Assets\Scenes\SampleScene.unity scene
Excel file location is Assets\StreamingAssets\test.xls
Changing excel file data and saving it results in the following actions:
2.1) Excel runtime monitor detects file change and transfer data to the database
2.2) After importing OnAfterImport event is fired and InjectText.cs Inject() method is called (cause it was registered for OnAfterImport event as a receiver)
2.3) Inject() method read the value from database and transfer this value to text component

Code (CSharp):

1. using BansheeGz.BGDatabase;
2. using UnityEngine;
3. using UnityEngine.UI;
4.  
5. public class InjectText : MonoBehaviour
6. {
7.  
8.     // this method is called after import
9.     public void Inject()
10.     {
11.         GetComponent<Text>().text = BGRepo.I["watchme"].GetEntity(0).Name;
12.     }
13. }
14.  

Spoiler: New OnAfterImport event

Please, let me know if you have any questions

 

Extremely sorry for delay.
Your solution works like a charm.
Thanks a ton!
Another Query: Can I write the data in excel at runtime?
I created another column Month, set up in database configuration.
I created another function, added it in OnAfterImport() runtime
I tried writing
public void WriteData()
{
Debug.Log("Inside writeData");
repo = BGRepo.I;
table = repo["watchme"];
table[0].Set("Month", "July");
repo.Save();
}
But couldn't save the data.

 

I have attached my project with few modifications.

 

If you want to transfer data from the database to Excel file, you can use BGExcelImportGo.Export method (BGExcelImportGo component is the component, attached to ExcelRuntimeManager GameObject)
There is one caveat though: it requires exclusive file access- so if the file is opened in an external editor (like Microsoft Excel or LibreOffice Calc) you won't be able to write the file and get IOException

So you need to ensure the file is closed at the moment you write to Excel file

I've attached an updated project, which demonstrates how to export data to Excel file
The list of changes:
1) I turned off BGExcelImportGo.ImportOnStart toggle
2) I've added InjectText.Manager variable
3) I call InjectText.Manager.Export() method after changing the month to "July" in the database

Here is how to trigger exporting:
1) Run SampleScene scene in Unity Editor
2) Open StreamingAssets/test.xls file in your Excel editor
3) Change Excel data, close the file, and confirm you want to save the changes. Closing file instead of simply saving it is a key for releasing file lock (otherwise you won't be able to override this file)
4) Once you saved and closed the file, BGExcelImportGo component detects the file was modified and import all data to the database.
5) After importing InjectText.WriteData() method will be called, the month will be changed to "July" and the data will be saved to Excel file (using Manager.Export() method)

Please, let me know if you have any questions

 

Thanks for quick reply.
It's working properly.

 

Hi,
I have a excel file in .xlsx format.
While setting up and importing datasource I faced this error.

In log file it shows that File is read successfully. but no data imported.

 

Hi!

There was a bug in one of the earlier releases, which broke xls/xlsx (binary/xml) Excel file format detection, and it was fixed in one of the later releases.

Please, take the following steps:
1) Make sure you use the latest BGDatabase version (1.6.3)
You can find BGDatabase version under Settings->Main (as shown on the screenshot below)

Spoiler: BGDatabase version check

2) Download and import the latest Excel Editor plugin (version 1.3) here: https://www.bansheegz.com/BGDatabase/Downloads/EditorExcel/
You can find the installed version of this plugin under Plugins->Excel integration for Editor

Spoiler: Excel Editor plugin version check

Please, confirm that the issue has been resolved

 

Awesome! issue resolved.
Thanks for quick response.

 

Hi BansheeGZ,

I'm trying to use the asset for a particular use-case, and I'm trying to follow the tutorials but I am getting stuck because there are a few slight differences that I haven't figured out.

My program is made to test multiple AI-driven racecars that are all simultaneously on a track, and log the progress of each car over time. I want the scene table to have one nested RaceCar table for each car (there will be anywhere from 6-20 cars at a time), and each RaceCar table should have a nested Telemetry table. Each row in the Telemetry table is a timestamped recording of car speed, position, etc. So, we are using the asset only for datalogging, and all table values are empty at startup.

One issue is that we randomize the number of cars at startup, so we want to attach a RaceCar script to each car and have that script generate the RaceCar entry within the scene table when the scene starts. Then, every tenth of a second, I want the RaceCar to add a new row to its Telemetry table. I'm trying to follow the 2D game example, but it uses "E_Scene.ForEachEntity(scene =>....." on startup - we will only have one scene and really want to directly address that single scene. What is the best way to get a handle on our single scene, so we can call E_RaceCar.NewEntity(scene) on it? I thought that the BGMetaEntity API's first option (this[int entityIndex]) would do it, but that doesn't compile.

What am I overlooking, or what is the best way to do this? Thanks!

 

Hi,

1) You can access the first scene entity by using this code
E_Scene.GetEntity(0)
or
E_Scene.GetEntity(new BGId("{Insert you ID here}"))
So you can create a race car with this code:
var newCar = E_RaceCar.NewEntity(E_Scene.GetEntity(0));
But if you have a single scene- there is no need to create a RaceCar table as a nested table to the Scene table- create it at the top level.
If you will need some scene reference later- you can create a relationSingle field to the Scene table
Also, I think the Telemetry table would better be a top-level table with a relationSingle field to RaceCar

2) I have serious concerns about using BGDatabase in your particular use-case.
You said you want to create a telemetry row every 1/10 of a second.
So what will happen if the car stops or if it's moving at very low speed?
How many records could be created?
I do not think BGDatabase is well-suited for this task, cause it's an in-memory database, meaning it holds all data in memory and does not flush it on disk- so it has some limits in terms of the size
I think you need some tool, which has some kind of maximum memory parameter, which can limit how much memory data can take before it's flushed to the disk and memory is released.

 

Hi again! I´v just bought Localization addon, and so far it seems very straightforward. I´d like to know if there is a Bolt node/way for changing the language?

 

Hi, thank you!
The new unit can be downloaded here: https://www.bansheegz.com/BGDatabase/Downloads/Uncategorized/
It is located under "BansheeGz->BG Bolt Set Locale".
Pass locale name as a parameter

Please, let me know if something is wrong

Spoiler: screenshot