BG Database (inMemory database | Excel/Google Sheets syncing | CodeGen | Save/Load support)

@Autarkis

1) To filter rows with Playmaker, please, use for loop to iterate the rows and find target rows.
No need to use BGEntityGo component.
I've sent you a working example in private message.
Please, test it and let me know what you think.

2) BGEntityGo component is actually a pointer to the target row.
So if you need your GameObject to be connected to some database row- you could add it.
No need to add it otherwise.
Once it's setup, you can read/write values directly to a row without the need to find/locate this row first.
And it's much better to use generated classes (Settings->Code generation->MonoBehaviour Classes) instead of generic BGEntityGo component, cause these classes, apart from connecting GameObject to a row, also have additional helper properties generated out of database schema.

3) The reason, why you have empty rows in Chose Entity popup of BGEntityGo editor, is because your row names are empty. Probably we need to implement some alternative method to identify rows with empty names.

 

Hello,

I just bought BG Database and tried to export the existed sample database into a Spreadsheet in order to see how working from spreadsheet could ease the process by allowing using on online tool. So I followed the tutorial and succeeded to export to a Google Spreadsheet. But I was kind of disappointed when I noticed that the generated sheets does not contain any data validation rule and that some columns that should reference other table/sheet/meta row used concatenated values.

Example with the Collectable :

In Unity the column "type" reference an entry in the ColletableType table.



Inside the Spreadsheet:



The column "type" seems to be the concatenation of the type value and the id of the CollectableType row of that type.

So, just taking a quick look at that Spreadsheet it is not clear for me how it is possible or easy to use it without making errors. Is there something I did not notice ? Is it possible to have data validation rule automatically generated in order to constraint what data a columns can have ?

Thanks

 

@Kiupe , Hello

Relation fields (like Collectable.type) stores the id of related row.
It's just a pointer to one of the CollectableType rows.
The formats are "Name_Id" or simply "Id" without a name (both formats are supported).
The reason we included "Name" token is caused "id" is just a meaningless set of bytes and without "name" tag is not clear which row is actually referenced.
On import "Name" tag is completely ignored and only "id" is used to set the value.

Regarding validation:
The current strategy is to ignore individual errors (errors related to incorrect cell value) but print them to the Log, which can be viewed by pressing on the "Log" tab.
Theoretically, all the errors should be printed to this log, and if it's missing it should be considered as a missing feature/bug.
Also, we could add "Stop on first error" parameter to Import/Export job, which would cancel the import process if any error is encontered and report this error.
Please, let me know what you think.

 

Ok, I will test as it is and see how it feels.

One more note: BG Database seems to create issue with Unity package. It's not clear what is happening but after imported BG Database and moving it to the Plugins folder suddenly Unity packages like Timeline are not recognized anymore.

 

Ok, I will test as it is and see how it feels.

One more note: BG Database seems to create issue with Unity package. It's not clear what is happening but after imported BG Database and moving it to the Plugins folder suddenly Unity packages like Timeline are not recognized anymore.

Here what happened after reloaded all assets. On a side note I have to uncheck Google API assemblies because I already have them in my project.

 

Thank you for letting me know, I will test it

 

Hope you be able to find out what's happening because I actually can't make my project working anymore without removing BG Database.

Unity 2019.1.3f1

 

1) I've tested moving BGDatabase to Plugins folder with Unity 2019.1.3f1 and it did not result in an error.

2) I'm almost 100% sure the problem with GoogleSheets API DLLs (gdata).
You said you already have these DLLs in your project, so you disable DLLs from our package.
Most probably your DLLs from a earlier gdata package or some DLL is missing (we use the last version: 2.2.0), so Unity can not load BGDatabaseEditor.dll
Could you, please, zip these 5 DLLs you use and send the archive to me so I could check them? (Google.GData.AccessControl.DLL, Google.GData.Client.dll, Google.GData.Documents.dll, Google.GData.Extensions.dll, Google.GData.Spreadsheets.dll)

Also, I think the problem probably can be solved if you disable existing DLLs and enable our DLLs instead.
If you need them at runtime and not only in Editor, you can move them from "Editor" folder to any other folder

 

Hi, congratulations for this fantastic asset !
Maybe a newbie question already answered :
I use the set by Index PM action, data is written, but as soon as I close the runtime, this data is lost.
Any way to "keep" datas generated during runtime in the database ? For instance a score, an inventory ?

 

Hi, Philippe, thank you for your support!

The database works exactly as Unity does- it's rolling back all the changes which are made during play session.

There are 3 ways to save/load play session changes:

1) By clicking "Save" button while playing (in Unity editor only)

2) Saving and loading the whole database with these lines of code:

Code (CSharp):

1. //to save
2. byte[] bytes = BGRepo.I.Save();
3. //to load
4. BGRepo.I.Load(bytes);

3) Saving and loading only part of the database with SaveLoad addon (we recommend this method)

Code (CSharp):

1. //to save
2. byte[] bytes = BGRepo.I.Addons.Get<BGAddonSaveLoad>().Save();
3. //to load
4. BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(bytes);

More details about how to use it and why it's better than saving the whole database can be found here: http://www.bansheegz.com/BGDatabase/Addons/SaveLoad/

Once you called Save method and obtained an array of bytes, you can save this array to the file on disk (in the Application.persistentDataPath folder https://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html) and use it in the future to restore database state (by calling corresponding Load method and passing this array as a parameter).
Also, please, note, saving and loading is a relatively expensive operation and it's not meant to be executed every frame. Usually, you want to call save/load functions when the player saves/loads the game (or at autosave/autoload).

If you have any difficulty implementing these save/load functions, please, let me know, I will send you a working example.

 

Perfect support, thanks a lot !

 

Hi,
Fantastic Asset!
I am newbie for this one. I have a specific task in hand.
I have a Excel file kept outside Unity in specific folder in C drive. Lets say C:\MyExcelInput\Sample1.xls
I will have to establish a live link in Unity such that after building EXE, user will open excel file and update changes at runtime. User will change certain cell (B2 column heading) Speed to (C4 column) 3 to 7.
Only when there is any update, after receiving that value I will have to change animation speed of GameObject A.
Please explain steps I will have to do in order to achieve the effect.
Thanks in advance.

 

Hi, thank you for your support!
This is possible with one of our extension packages ("Excel file monitor for Runtime"), which can be downloaded here http://www.bansheegz.com/BGDatabase/Downloads/
It works on PC/Mac/Linux only.
We can not integrate it into our package, cause it requires one of the libraries to be moved to Runtime space from Editor space
This package has an example scene (please, read instructions at the bottom of the "Downloads" page and use an empty project to run it)
Once you understand how the example works, it would be much easier for you to integrate this package to your project.

To integrate the package to your project you will need to:
1) Import BGDatabase to your project,
2) Create your own database http://www.bansheegz.com/BGDatabase/Setup/
3) Export required data to Excel http://www.bansheegz.com/BGDatabase/ThirdParty/Excel/
4) Move NPOI library to Runtime space (from Assets\BansheeGz\BGDatabase\Editor\Libs\NPOI\ to Assets\Libs\NPOI for example)
5) Import downloaded BansheeGzExcelRuntime.unitypackage (Assets->Import package->Custom package)
6) Add "Assets\BansheeGzExcelRuntime\BGExcelImportGo.cs" component to your scene. (to support Excel->Database data transfer)
7) Add one of the binders (http://www.bansheegz.com/BGDatabase/DataBinding/) to a target component (to support Database->TargetComponent data transfer). In our example we use BGDataBindingFieldGo. If you can not inject the value directly, you could use a very simple proxy script (please, take a look at Assets\BansheeGzExcelRuntime\Example\GoldInjectionTarget.cs as an example)
8) There are 2 options to set up excel file location and some additional parameters- a) you can set them as BGExcelImportGo component parameters (in Editor) or b) By running a scene and pressing on "Excel>>" button (in Editor or at Runtime). If you save runtime settings to a file they will override Editor settings.
9) Once you setup Excel file location, you can open this file and change the data. Please, note, you will need to save the file by pressing Ctrl+S to trigger data transfer. Also, the scene should be run at this moment (in the Editor or in the build).

Please, let me know if you have any questions.

 

Thanks for your quick reply. I have sent private mail with details. Pls have a look at it.

 

Im very intrested in this asset. But were it not for my dedication I would have never found out that your database supports addresables. It should be in bold letters in the middle of you asset store listing! I think you have the only DB asset that does this!!!

PS: Correct me if Im not understanding it correctly but I can have a database reference an addresable asset(SO with a prefab reference) (NOT in the resources folder) and then access that asset through code/identify it with an ID/instantiate the prefab etc... and only when accessing it will it be loaded into memory. (I'm concerned with resources because they can get out of hand quickly when developing for mobile)

 

Hello, thank you for your advice!

Regarding addressables- one restriction applies: you need to preload assets before reading the values.
The reason why- is because our API is synchronous and addressables API is asynchronous.

We provided the script here: http://www.bansheegz.com/BGDatabase/Downloads/#addressables (at the bottom of the page) which can be used to preload all values from the database.
But it can be modified to preload only required part.
For example, you may have 100 levels and each level has its own set of sprites and you can use both database values and addressables system itself to filter and preload only required assets on level loading.

If you are not sure if you want to use our asset or not- please, email to this address: support@bansheegz.com
and we will provide a demo version

EDIT: "Regarding addressables- one restriction applies: you need to preload assets before reading the values." - this is not totally true. You can actually retrieve asset address from database and use it with addressables system directly, no preloading is required.

 

I am trying this asset, and a bit confused on creating my own database. It asks to do this in resources folder. But I don't want to do that because that means every time I change the data in database, I will need to rebuild the game. This presents problem for patching games because I will need to update entire game. I thought the database source can be read from other than resources folder?

 

Hello, I've sent you a beta version with support for loading database from StreamingAssets folder in private message. https://docs.unity3d.com/Manual/StreamingAssets.html

 

I have been integrating the DB in our game today and on friday. And I must say it is amazing. Got maybe two complaints/suggestions.
ad 1) I have spend like 4 hours coming up with a solution on how to serialize and save your classes with our saving system. Main problem was a circular reference that is created when you added a relation-single to a table. Had to resort to creating my own JsonConverter. I don't really know how you can solve this easily. Maybe an addon/tutorial on how to do that on your website would be enough? I doubt many people will even need to serialize it on their own and will just use your saving system.
ad 2) Maybe a special Scriptable Object type so I don't have to add it myself. Also some "OnAddToDatabase" check if that SO is a subclass of some other abstract SO. (Basicly what Im doing is adding a relation-multiple to my item database where each item can provide skills which all inherit from ASkill so some user level protection against designers assigning random objects or even not having to cast my objects) Again. Don't really know how to easily solve this. Just giving you my experience. Anyway gonna definitly use this asset as it is amazing.
You guys did gods work with this one. Especialy the code gen.

PS: A different way to iterate over all the entities in a table would also be helpful. Something a bit faster that sending Actions. But thats just a nitpick.

 

@GrandBOOM , thank you so much for your support!

1) Could you please provide more details about the issue and saving system you are using?
How does your saving system work? Does it save GameObjects along with their components?
If yes, which our components you use (generated MonoBehaviour classes/BGEntityGo/DataBinders)?
2) There is SaveLoad addon, which saves/loads database state http://www.bansheegz.com/BGDatabase/Addons/SaveLoad/
It should work with any external saving system.
On Save you call Save() method, obtain byte array and save it along with other information you want to save.
While loading you pass this array to Load() method and the database state is restored.
But it works with database only, it does not save any GameObjects-related information

Good idea, thank you for the suggestion.
I've sent you a version with scriptableObject field added in private message.
It also has class/subclass constraint

You can use :

Code (CSharp):

1. var entities = YourGeneratedTableClass.FindEntities(null);
2. for (var i = 0; i < entities.Count; i++)
3. {
4.     var entity = entities[i];
5. }
6.  

Not sure about the speed though.

EDIT:
even better:

Code (CSharp):

1.  
2. var count = YourGeneratedTableClass.CountEntities;
3. for (var i = 0; i < count; i++)
4. {
5.     var entity = YourGeneratedTableClass.GetEntity(i);
6.    
7. }
8.  
9.  

 

Amazing work on that feature. Nothing feels better than a type safe database.

Thank you for this. Probably more performant that creating anonymous methods. Gonna test it out later.

PS: One tiny suggestion. When using the Code Generator a button next to the Help button or Save All button to generete would be lovely. (Might even light up when you edit the Configuration) Offten times I forget to generate and later wonder why I cannot access my variables. Im sure I will get used to it either way.

 

I've sent the updated package in private message

 

Hey. I encountered some "trouble".
So after building my game I found out that sprites do not load. "Currently, Sprite/SpriteAtlas fields can be loaded with method #2 only" <- from documentation. But it's super easy to go around it as with the Scriptable Object field I can create an object called "SpriteSO" and just have a public Sprite MySprite or w/e.
Only functionality I lost is that the database does not preview the image I think. Or am I missing something?

 

Hello, we have a fix for it (Sprite/SpriteAtlas not being loaded).
I will send it to you in private message very soon.
This fix allows loading Sprite/SpriteAtlas fields just like any other field.
Sorry for the inconvenience

 

Hi, is there a way to duplicate the structure of a ROW with lot of fields ?

 

Hi, could you please give me a link to this RAW file structure description?
I found this while searching for it https://en.wikipedia.org/wiki/Raw_image_format#File_contents , but I'm not sure it's the same RAW file format you are referencing to.
So you want to save binary files on disk- and have their metadata in the database for quick access, correct?

 

No, not a raw file, what you call a row in your database, I mean duplicate the structure of a data table. I send you a picture if unclear.
Thanks !

 

Sorry, I did not notice your post edit by some reason
I sent you a package with table structure duplication in private message

 

Hello @BansheeGz
Prospective BG Database user here.
Could you please give some pointers on what the bottle necks are stopping live updates from working with WebGL and potential workarounds if at all possible?

Would also like to know very broadly, and forgive me if a previous post has addressed this somewhat, if and how BG Database could be used to satisfy a use case where (using Addressable perhaps, would this make sense?) apart from providing a database that's based on spread sheets, a content management system can also be integrated such that project admin who are not game devs can upload content to a server, with automatic updates to the database spreadsheet and by extension, the game clients as well.

Thanks.

 

Hello @DADA_universe ,

Thank you for your interest in our asset.
I've sent a demo version of our asset to you by email along with some comments

Regarding live updates from WebGL builds- we had some technical difficulties implementing synchronous cross-domain HTTP request on this platform. Probably this issue can be resolved with some third party HTTP client

 

Thank you @BansheeGz, you've been most helpful.

 

Hi Dear

I am new to use BGDatabase

so I generated User class.

then I write code


var newUser = User.NewEntity();
newUser.Age = 30;
newUser.Set("Age", 30);

byte[] bytes = BGRepo.I.Addons.Get<BGAddonSaveLoad>().Save();


like this,

when I run with unity play mode
Database row is added as shown

but I stop unity play mode then database rollback to before I run code.

how can I make a change to the database with code?

thank you for your support

 

Hi,

This line is excessive:

Code (CSharp):

1.  newUser.Set("Age", 30);

it has the same effect as this line:

Code (CSharp):

1.   newUser.Age = 30;

Since you are using code generation, it's preferable to use the second method (newUser.Age = 30; )

Once you obtain a byte array from SaveLoad addon, you can save it to a file.
Use Application.persistentDataPath folder, cause your application has write access to it on mobile devices
Once you have it saved to a file, you can load it later any moment you need it
You can load it at the application start up or when user choose to load a save file
You can create multiple save files and let user chose which file he wants to load.

Here is a minimalistic example, which uses one single save file and loads this file at startup if it exists:

Code (CSharp):

1. using System.IO;
2. using BansheeGz.BGDatabase;
3. using UnityEngine;
4.  
5. //this is minimalistic save/load example
6. public class SaveLoad : MonoBehaviour
7. {
8.     private string FilePath
9.     {
10.         get { return Path.Combine(Application.persistentDataPath, "save.dat"); }
11.     }
12.  
13.     //this method loads data from a saved file at startup
14.     void Start()
15.     {
16.         Load();
17.     }
18.  
19.     //this method saves current database state to a file save.dat
20.     public void Save()
21.     {
22.         File.WriteAllBytes(FilePath, BGRepo.I.Addons.Get<BGAddonSaveLoad>().Save());
23.     }
24.  
25.     //this method loads database state from a file save.dat
26.     public void Load()
27.     {
28.         var filePath = FilePath;
29.         if (File.Exists(filePath)) BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(File.ReadAllBytes(filePath));
30.     }
31. }

Also, because you add new rows at runtime, make sure you included User table to SaveLoad addon settings with "Add missing"/"Remove orphaned" toggles turned on, as shown on the picture



More information about SaveLoad addon is available here: https://www.bansheegz.com/BGDatabase/Addons/SaveLoad/

There are also other examples, which use SaveLoad addon:
1) Example, bundled with a package. You can find save/load script at this path (Assets\BansheeGz\BGDatabase\Examples\Scripts\BGSavingLoading.cs)
2) Inventory example is available at our download page (Inventory example) https://www.bansheegz.com/BGDatabase/Downloads/#inventory

Please, let me know if you have any questions

 

Hi.
Thank you for a quick reply.

now I can save and load the database.

this is a great asset I’ve never met !!

just one more question can I make changes to the default static database at runtime?
(I want to maintain data with GUI at edit mode after I add data at playmode.

Thanks.

 

@chooser thank you so much for your support!

Yes, you can press either "Save All" or "Save Repo" buttons while running a game (at playmode) and confirm you want to save runtime changes

 

ok, so I can not save a static database with code.
only I can do is a GUI button. right?

thank you for everything.

 

You can save database with code, but it will work only inside Unity Editor (not in builds)
This should work (replace pathToDatabase value with the relative path to your database):

Code (CSharp):

1. //deprecated

UPD: To save database from Unity Editor assembly, use this code

Code (CSharp):

1. BansheeGz.BGDatabase.Editor.BGRepoSaver.SaveRepo();

 

very thanks!!

 

Hi @BansheeGz

first of all, thanks for this great asset, the concept is great and functional, i've been looking for something like this since i started working in Unity.

Of course, i've got a bug report too!

#1

Every time i hit play, stop the game and then return to the BGDatabase window another instance of ResourceManagerCallbacks is created in the scene. I'm using lastest BG Database version, Unity 2018.3.7 with addressables package, let me know if you need some more info. The project is quite large so i'm unable to share unfortunately.

#2

When i add addressable prefabs to database i get flooded by:

Code (CSharp):

1. Exception encountered in operation UnityEngine.ResourceManagement.ResourceManager+CompletedOperation`1[UnityEngine.GameObject], result='', status='Failed': Exception of type 'UnityEngine.AddressableAssets.InvalidKeyException' was thrown., Key=a6b7b730602f6464aa6a90009c626266
2.  

3#

I'm probably doing something wrong, but i've got a gap between rows when i export data to sheets

 

ResourceManagerCallbacks .. oh that is where it was coming from...

 

@Oshigawa
@castor76
Hello, I've sent you an updated package with addressables issues got fixed
Sorry for inconvenience

Export does not erase empty rows, so if you have empty rows before export, they will remain intact.
Or these empty rows are created during export?

 

Thanks No sweat, glad we could catch those

I don't think i've put those empty rows there, i'll check some more and try to reproduce it or see if i messed something up.

 

We ran some tests (adding/removing rows, exporting multiple times), but was not able to reproduce it
Maybe it appears only under certain conditions
If you encounter this issue again, please, let me know

 

@BansheeGz

Thanks, it didn't happen again, so i've might screwed something unknowingly. Patch works fine, no more error or duplicating ResourceManagerCallbacks prefabs.

While we're at it, is there a way to reorder rows in the Configuration view?

It would be nice (if it's not already implemented, but i can't seem to find it) if you could select the default loader for prebabs (resources/assetbundle/addressables/addressablesbyguid) since it's a bit tedious to set every row and there's a chance of hitting remove row when repeating the procedure many times.

Ah yeah, one more thing, i know this is not excel, but maybe freezing some of the columns (or setting them as selected) could be cool, like the way you can freeze header, things can get complicated when you have tens of columns and you need to manually copy some stuff from the inspector.

 

Yes, by using "M" menu (which is shown next to field's ID)

I've sent you an updated package in private message

I'm not sure if it's possible to implement properly with Unity IMGUI, we will look into it.
UPD: pinned columns are implemented in version 1.8.1

Thank you a lot for your feedback and suggestions

 

@BansheeGz

Thanks, works great

Now, another question

It may be out of the scope (i wouldn't normally ask, but the unity resources on addressables are quite scarce for now), but the addressables are not being loaded in the build. I turned off the lazy mode and i've got the BG Addressables Preloader in the first scene, but when go into the game scene i get errors

Code (CSharp):

1. Exception encountered in operation Resource<GameObject>(Prefab.prefab): Unknown error in AsyncOperation

No prefabs are being loaded that way. I've set up my database loader to resources, which i thought was not a good idea, but i see that the database in AddressablesQuickStart is also set to resources, so i guess that's all right, so i'm not quite sure where i'm wrong.

 

@Oshigawa
There are 2 ways to load assets using Addressables system and our database:

1) Asynchronous (using Addressables asynchronous API and asset's address, which is stored inside the database). This method does not have any prerequisites- you can load any asset at any moment without any condition- but you may need to wait for several frames until the asset is loaded
Example:

Code (CSharp):

1. var loadOperation = Addressables.LoadAssetAsync<GameObject>(address);
2. loadOperation.Completed += handle =>
3. {
4.     //asset is loaded
5.     var asset=handle.Result;
6. };
7.  

2) Synchronous (using a preloader, which preloads required assets). Addressables system does not provide a way to load asset synchronously, so preloader preloads required assets asynchronously (using the same API from method #1). After assets are preloaded you can access them synchronously using synchronous BGDatabase API.
Example:

Code (CSharp):

1. var imageSprite = Entity.Get<Sprite>("icon");

I do not know which method is better (EDIT: probably the first, asynchronous method, because it does not rely on extra code(preloading) to be executed correctly and it's as robust as Addressables system is )
Database location (Resources or StreamingAssets) does not matter.
I've sent you an example project which uses both methods
It works fine for me inside Unity and inside Windows build.

Could you please, tell me-
1) which Unity version you use?
2) which Addressables package version you use?
3) Could you please try to load the same asset(s) asynchronously (using method #1)?

 

@BansheeGz

Hi and thanks for the PM, i'll try out loading a single asset, for now i've been trying to load all assets that share the same label.

Unity 2018.3.7, Addressables 1.5.1., trying method #1.

 

Code (CSharp):

1. Exception encountered in operation Resource<Object>(Provoker.prefab): Unknown error in AsyncOperation

Code (CSharp):

1. Exception encountered in operation UnityEngine.AddressableAssets.Initialization.InitializationOperation, result='', status='Succeeded' - Chain<Object>: ChainOperation of Type: UnityEngine.Object failed because dependent operation failed
2. Unknown error in AsyncOperation

Code (CSharp):

1. Error: can not load asset using address=Assets/Prefabs/Enemy Prefabs/Pirates/Provoker/Provoker.prefab

Well that went bad. But nevermind, it doesn't pose a problem for now. I crossposted on the addressables forum to see what's going on.

 

Hi, I just imported JSON.NET (newtonsoft) from the Asset Store, but it seems to have a conflict with the Newtonsoft.Json.dll included with BGDatabase:

Code (CSharp):

1. PrecompiledAssemblyException: Multiple precompiled assemblies with the same name Newtonsoft.Json.dll included for the current platform. Only one assembly with the same name is allowed per platform. Assembly paths: Assets/BansheeGz/BGDatabase/Editor/Libs/GDataV4/Newtonsoft.Json.dll, Assets/JsonDotNet/Assemblies/Standalone/Newtonsoft.Json.dll

What is the best way to resolve this conflict? My workaround is to disable the "Editor" platform on the newtonsoft dll included with BGDatabase. Everything seems to work fine after that change.

thanks!