BG Database (inMemory database | Excel/Google Sheets syncing | CodeGen | Save/Load support)

Yeah, I caught the import wizard. It's a separate "DB construct process". Though I'd prefer the spreadsheet to be more visibly informative, it's not too bad. If I move to custom IDs, I can use those like GDEditor.

A couple of questions about Save/Load -

I'm not sure what this means. Does that mean I can't do what I'm asking for above, have references to tables/fields, if using dynamic data?

Also is there a 'backup' of the original data so that I can restore to 'default' value? Or would I clone a table for dynamic data and restore from a predefined source table if I want to restore values?

Okay, I see saving happens to another file and the main DB viewable in the editor isn't updated. So there's two DB files, the one created at build as part of the distribution package, and another runtime copy with saved data, right?

One last point, how do I check if a row/entity exists? eg.Cards table has 3 entries and I want to test if item number 5 exists.

Code (CSharp):

1.         DB_Cards.GetEntity(1).f_name = "splooky";
2.         if (DB_Cards.GetEntity(5) != null) {
3.             DB_Cards.GetEntity(5).f_amount = 999;
4.         }

This doesn't work!

Of course testing a particular row number isn't useful, but I'd want to see if an ID already exists ahead of creating one. I guess it's a case of some Contains() test for an ID?

 

It means that if you:
1) get database object (table/field/row) from database
2) store it in some C# variable or field
3) Load the data with SaveLoad add-on
4) Trying to use variable/field from step #2 results in an error
The reason why - is cause the whole database is reloaded

SaveLoad add-on creates a temporary database in memory when Save method is called.
It uses the default database (BGRepo.I) as a source of data and add-on merge settings as a guide which data needs to be saved and which data should be skipped.
After that, it serializes temporary database to a byte array, destroy temporary database and return serialized byte array

Code (CSharp):

1. if (DB_Cards.CountEntities > 5)        

 

Hi,

I already own BGDatabase, but I've not used it much, nor do I know much about databases.

I was wondering if you could advise me about whether or not BGDatase is the right plugin to use.

I have 100 'enemies'. I need to make a record of up to 10 variables (ints, floats, strings and bools) each time players kill an enemy. I need to record each instance of a player killing an enemy, essentially appending the data to each field. A typical play session would involve 100 of the variables to be added to 100 times. It's important to understand that each time any variable is added to it is appended, not overwritten. There is no requirement that it be stored remotely.

Upon level completion, and also from the Main Menu, a player can review this data in a chart. These charts need to reflect progress over time and draw from data hundreds (possibly thousands) of instances deep. There is no need for the charts to be relational or searchable.

I guess what I'm asking is if keeping track of 1000 variables as they each accumulate data and keep adding ot the size of the recorded files is best done in BGDatabase? I don't need all the bells and whistles of a database, but it seems to do the best job of organising it.

I also own Easy Save and am researching JSON, but I could use advice from a data expert.

Thanks!

 

I added list<Vector3> as column and filled it with 10 000 values. After it BGRepo.Editor repaints for a long time with second breaks. I can't even get out of unity

 

Hi,

If you have a lot of data, and you want to process it using complex queries/rules, using regular SQL database is a better solution.
1) SQL database should have some parameters to limit memory usage. BGDatabase is in memory database, keeping all the data loaded
2) SQL queries can have complex rules for result data calculations, which can result in better performance if proper indexes are created to avoid full scans. BGDatabase only has keys and single field indexes, which are limited compared to SQL queries optimizations

Maybe even better, specialized solutions exist, I do not know

 

Could you, please, write to our support email, so I could send you an upgraded package with a fix?
I can not DM you
Or send me a message using "Start a conversation" button

 

having an issue adding a google sheets data source - I have the service setup in the google console, have entered my client id / secret etc but when I try to get tokens it fails with an error:



[edit] - aha - had a stupid trailing space on one of the fields that was breaking the parsing apparently.

 

Thank you for the update, we will add a check for space characters inside parameters in the next release

 

Hello!

I'm using BGDatabase to store my prefabs and I'm trying to implement a tool to automate the way I add multiple prefabs into the database with an Editor Window. I can create a new row in my table and set the fields with primitive types (string, bool, int) but can't find a way to set asset type fields.

I found some documentation on the website saying that I can import the Editor assembly to access a "SetAsset" function inside the BGField but it doesn't show up for me.

Can anyone help me with this please?

 

Hello!

SetAsset method is defined on the base Unity asset field, which is BGFieldUnityAssetA<T>, so you need to downcast the field to this base class

Code (CSharp):

1. BGFieldUnityAssetA<GameObject> field = (BGFieldUnityAssetA<GameObject>) table.GetField("prefab");

I would highly recommend you to use CodeGen addon, which gives you direct access to the field without need to downcast the field https://www.bansheegz.com/BGDatabase/CodeGeneration/ExtensionClasses/
I've attached an example project, which implements both approaches (with and without CodeGen)
With CodeGen addon- you can access the field directly using {TableClass}._{fieldName} notation
Here is the comparison of both approaches:

Spoiler: SetAsset example: CodeGen vs No CodeGen

Code (CSharp):

1. using BansheeGz.BGDatabase;
2. using BansheeGz.BGDatabase.Editor;
3. using UnityEditor;
4. using UnityEngine;
5.  
6. public class SetAssetExample
7. {
8.     //example assets
9.     private static GameObject[] Assets => Resources.LoadAll<GameObject>("/");
10.  
11.     [MenuItem("Tools/SetAssetWithCodeGen")]
12.     public static void SetAssetCodeGen()
13.     {
14.         foreach (var asset in Assets) CreateNewDBEntry(asset.name, asset);
15.     }
16.     [MenuItem("Tools/SetAssetNoCodeGen")]
17.     public static void SetAssetNoCodeGen()
18.     {
19.         foreach (var asset in Assets) CreateNewDBEntryNoCodeGen(asset.name, asset);
20.     }
21.  
22.     private static void CreateNewDBEntry(string assetName, GameObject asset)
23.     {
24.         DB_Objects newRow = DB_Objects.NewEntity();
25.         newRow.name = assetName;
26.         DB_Objects._prefab.SetAsset(newRow.Index, asset);
27.     }
28.  
29.     private static void CreateNewDBEntryNoCodeGen(string assetName, GameObject asset)
30.     {
31.         BGMetaEntity table = BGRepo.I["Objects"];
32.         BGEntity newRow = table.NewEntity();
33.         newRow.Name = assetName;
34.  
35.         BGFieldUnityAssetA<GameObject> field = (BGFieldUnityAssetA<GameObject>) table.GetField("prefab");
36.         field.SetAsset(newRow.Index, asset);
37.     }
38. }

The version with CodeGen addon is much less verbose, no boilerplate code, compile-time checks, IDE autocompletion support

Spoiler: CodeGenAddon settings

Here are the rules for this method to work properly:
1) The code should be located in Editor assembly (under Editor folder)

Spoiler: example

2) The assets should be accessible at runtime with chosen loader:
2.1) If you use Resources loader, assets should be located under Resources folder
2.2) If you use Addressables, assets should be included into Addressables settings

Spoiler: Resources assets

I've attached an example project just in case
Open it and import BGDatabase asset
The C# script location for creating new rows is Assets\Scripts\Editor\SetAssetExample.cs
Generated C# script for CodeGen addon is Assets\Scripts\CodeGen.cs
To run example code, use "Tools/SetAssetWithCodeGen" or "Tools/SetAssetNoCodeGen" menu items

Please, let me know if you have any questions

UPD: we have updated our docs to include more information https://www.bansheegz.com/BGDatabase/Misc/#editor

 

I sent an email to support but received no response, so I took the plunge and purchased the product.
However, I am beginning to regret that perhaps I will not be able to achieve what I have in mind...

What I want to accomplish:.
Read data from Google SpreadSheet like QuickSheet and generate a C# source file with it as a data class.

Is it possible to achieve the above with BGDatabase?

Also currently (http://www.bansheegz.com/BGDatabase/ThirdParty/GoogleSheets/) trying to load GoogleSpreadSheet while viewing this page!
However, I can't import it because the Merge setting does not exist as shown in the attached image on the job creation screen.
Is there any information on how to prepare a Merge Setting?

Also, when I set Merge Mode to transfer, I get an error like the one in the attached image.
I thought it might be a problem with Google's rights settings, but do you know how to solve this problem?

 

We received your email, but we were not online, so we could not answer it
Currently, we have 24 hours response time deadline, we can answer immediately if we are online, but if we are not, it can take several hours usually

Certainly, it's possible
We have CodeGen addon https://www.bansheegz.com/BGDatabase/Addons/CodeGeneration/, which generates C# data access classes.
Basically it's the same stuff, but internal implementation is different
If you are looking for exactly the same C# classes, QuickSheet produces, we could try to create another code generator to produce these classes, just let me know

1) It looks like GoogleSheets API is disabled in your project.
Please, follow this guide's 3rd step ("Enable GoogleSheets API if it's disabled") to enable it https://www.bansheegz.com/BGDatabase/ThirdParty/GoogleSheets/setup-oauth.html
2) We take a little different approach to importing data from GoogleSheets
Our import job does not change database structure (table/fields) it transfers rows only
Export job creates missing sheet/fields(headers), but import job does not
It means these tables/fields should be created before running import job
Probably we should add an option to create tables/fields during import job execution in the next release, we will look into implementing this
Currently, the interactive "Data Extraction Wizard" can be used to transfer tables/fields from Excel/GoogleSheets to the database https://www.bansheegz.com/BGDatabase/ExportImport/#dataExtractionWizard ("Using Data Extraction Wizard" section)
Your data should have proper data layout (i.e. the first row is the fields names), more information is here https://www.bansheegz.com/BGDatabase/ExportImport/#dataLayout ("Data Layout" section)
Run this wizard and import tables and fields
After importing, these tables/fields should show up under "Database" and "Configuration" tabs (ignore the rows, there will be no rows in your case)

Spoiler: tables under Database and Configuration tabs

and also under job's "Merge settings".
To include all tables/fields into job's settings, you can toggle on global "Add missing"/"Remove orphaned"/"Update matching" parameters (tables/fields will be highlighted in green)

Spoiler: tables under Merge Settings

To sum up:
1) Enable GoogleSheets API in your project in Google's console
2) Run Data Extraction Wizard to create database tables and fields using data from Google Sheets spreadsheet. Save database
3) Include all tables and fields by toggling on global parameters In the job's merge settings
4) Run import job to transfer rows. Save database.
5) Enable CodeGen addon, save database and run code generator (use "CodeGen" button on the toolbar). Here is the example settings you could use for CodeGen addon

Spoiler: CodeGen addon settings

6) If you used the same CodeGen addon settings as on the screenshot above, the file with data access C# classes sources will be located in Assets/Scripts/Gen.cs (you can use any location under your project). All classes names start with DB_ prefix (for example, DB_Items class is the class for accessing Items table)
The full list of all generated properties/methods can be found here: https://www.bansheegz.com/BGDatabase/CodeGeneration/ExtensionClasses/#whatIsGenerated ("What is generated, exactly?" section), you can use these methods/properties to access table, fields, rows, cells, add/delete rows etc.

If you have any questions, please, let me know

 

Thank you for your reply.
I am very impressed with your very detailed reply.
I have a few additional questions.

・I would like to know if it is possible to specify "enum" in the "Data Extraction Wizard"?

・I tried exporting and it made destructive changes on the GoogleSpreadSheet side.
（When I tried to export the data, GoogleSpreadSheet made destructive changes to the data.in my opinion, it initialized all the data entry rules).
Since we only plan to make changes in GoogleSpreadSheet, we would like to seal the export function.
There is a risk of accidental pushing by collaborators, which we want to prevent.
Any such features or ideas?

 

Yes, it's possible, select "Show all fields" menu option after clicking on the type button and standard file type selection dialog will be shown with all options

Spoiler: Add field dialog

after that, you need to select enum and provide full enum type name (including namespace)

Spoiler: enum type selection

If you are not sure, what is your enum full type name, use this C# code to print it to the console (replace MyEnum with your enum)

Code (CSharp):

1. Debug.Log(typeof(MyEnum).FullName);

Also, keep in mind, that you can still add fields manually one by one (under Configuration tab), you don't have to use "Data Extraction Wizard" for adding fields, it's just a utility for adding several tables/fields at once

As far as I know export can not overwrite existing data (at least with default settings and without special data setup), it can only add new rows/columns
If you noticed that unwanted changes were made to your spreadsheet, you can revert them by using versions history ("File->Version history" in GoogleSheets GUI)

We have added an option to disable data exporting via data source setting
I will send you an updated package in private message
Please, let me know if it does not work as expected

Data sources/jobs settings are stored locally in the settings file, so they are not shared by default
If you want your teammates to be able to run import jobs, you can share your data source settings with them by using data source JSON view

Spoiler: data source JSON view

Send JSON formatted string using secure communication, cause it contains sensitive data
Ask them to create an empty Google Sheet data source and update it with JSON data (via "JSON view->Open update window")

Please, let me know if you have any questions

 

I just wanted to inform you.
The position of the popup window seems to change depending on the row of the selected Type.
As shown in the attached image, I encountered a problem where the input window went to the top of the window and I could not input.
The top of the window is completely hidden, so the window cannot be moved.
Is this a problem with my settings?

Also, the error text is difficult to read and cannot be scrolled, so it is hard to tell what the error is.
(I'm self-resolving on this error.)

 

Thank you for letting me know about these issues!
I've sent you a package with a fix, both issues should be fixed, now it's possible to hover a mouse over an error message to see the full text in tooltip

 

I have confirmed that both have been corrected.
Thank you very much for your prompt response.

 

Thank you this worked perfectly! I haven't used the CodeGen yet but I'll check it later.

Thanks a lot!

 

Hello!

I've been loving the BG database, although I had a question about database update callbacks.

I'm using Playmaker and am trying to send a callback event when a database field is updated. Is there any way to do this?

example:
gameobject "DoorA" has a generated component on it with a tracked save field bool called 'unlocked'.

the database value is changed from false to true, and an updated event is sent to that gameobject telling it to check its value and open the door.

cheers!

 

Hello!

1) If you have a reference to the target GameObject, you want to send an event to, sending an event to it directly after changing database value would be more efficient solution, then using database callbacks

2) If you have a database component, attached to the target GameObject, use the attached PlaymakerEventSenderEntityComponentBased script to send an event to Playmaker FSM
This script uses database events to identify the moment the field value was changed
You need to provide Playmaker event name and database field name

Spoiler: component-based

3) Use PlaymakerEventSenderCellBased script if you do not have a database component attached
You need to provide event name and database cell, you want to listen to
I will send you this script in a private message, cause BGDatabase asset should be upgraded for this script to work properly

Spoiler: cell-based

Please, let me know if you have any questions

 

Just a friendly thank you here as well

 

Hi, I just tried this in Unity Editor is working fine.
But after build to Android. the value is not received in google sheet

Any other solutions for build android?

 

Hi,

Sorry, I can not remember any alternative solution, but I've checked the original guide, and it still works properly for me on Android device
Which Unity version do you use? I used Unity 2022.2.10 and a new project with default settings to check it

You could try the following steps to identify the problem:
1) Open your Player settings, select "Android" tab, "Other Settings" and under "Configuration" group, set "Internet Access" parameter to "Require" just in case.
For me, the default setting="Auto" works ok.

Spoiler: Internet permission

2) Surround your HTTP call with try/catch block and add debug messages
To read debug messages, install "Android Logcat" package from Package Manager (if you did not already) https://docs.unity3d.com/Packages/com.unity.mobile.android-logcat@1.3/manual/index.html
Here is a short tutorial how to use it

Spoiler: Youtube tutorial

You will need to enable Developer options (development mode) and USB debugging on your phone/tablet
Alternatively, instead of using "Android Logcat"- output your debug messages to some UI text component

3) Add Google Forms response check to make sure the response code is ok (200)
Unfortunately this check is not 100% reliable, cause if you turn off the form the response code would still be 200, despite the fact that data were not added. Maybe the better check exists, I do not know.

Spoiler: debug messages and response code check

I've attached the updated GoogleManager script with above-mentioned changes to this message.
We had to switch from WebClient to HttpClient for easier response code access (UnityWebRequest can also be used, probably)

 

Hello,
We found a saveload issue again. Version 1.7.12.
Load the database from saveload, some data lost,such as hashtable,default value.
Our project is preparing to be released, so I hope to reply as soon as possible, or is there any alternative solution?Thanks.
Test code:

Code (CSharp):

1.          
2.             //to save
3.             byte[] bytes = BGRepo.I.Addons.Get<BGAddonSaveLoad>().Save();
4.  
5.             var testHaseTableSaveData = SettlementData.GetEntity(0).ForceRelations;
6.             var testNormalSaveData = SettlementData.GetEntity(0).EntityID;
7.          
8.             //to load
9.             BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(bytes);
10.          
11.             var testHaseTableSaveDataAfterLoad = SettlementData.GetEntity(0).ForceRelations;
12.             var testNormalSaveDataAfterLoad = SettlementData.GetEntity(0).EntityID;
13.          
14.             //Result: testHaseTableSaveDataAfterLoad is null, and the every entity default value setting lost too

 

Great. It's works now by using HttpClient.
I dont know why but it's work when using HttpClient and not work using WebClient
Thankyou for your response.

 

Hello,

1) Regarding lost values: we do not save empty lists and empty hashtables.
We assume, that if hashtable is empty, it means there is no value, so empty hashtable==null
We implemented it like this to minimize resources usage and to make GUI easier (there is no option to define empty hashtable, so if you do not add any value to it- it will be null).
Maybe we were wrong- but we can not change it now anyway, because it can break existing code

In case you do not want to check for null, you can mitigate this issue by creating a new partial C# class with the same name and namespace as the generated class (SettlementData) and adding another property.
This property checks for null and if value==null it assigns an empty Hashtable as a value
Use this new property instead of the generated one.
We could add an option to generate such properties with code generator if it can help

Spoiler: New property

2) Regarding default values: this issue is probably not related to SaveLoad add-on.
We were not able to figure out how it could affect default values.
I will send you the latest build by email, please, let me know if the issue were fixed or not

 

Thanks, The default value issue has been fixed.But i found some weird,and send you a new email to report.

 

Hi,
When you support the merge on save? I really need this feature right now.
Or How can i modify to support this feature?
Thank you

 

Hi,
My current game need several save files.
one for game data, one for game setting.
as far as i know, the BGDatabase only can save all the table marked merge settings to one file.
but i need some tables save to one file, other tables save to another file.
how could i do?
Thank you

 

Hi,
Do you mean filtering rows when SaveLoad add-on's "Save" method is called to exclude some rows from the saved data?
I was sure we already support this feature via custom C# RowController class ( https://www.bansheegz.com/BGDatabase/ExportImport/#rowlevel "Row-level control for merge settings" section), however it looks like it works for Load method only for some reason.
Sorry, we will look into it and add support for Save method within a day or two

Sure, we will add additional configurations to SaveLoad add-on
I will send you an updated package along with a fix for row controller

 

@infinitudegame hi,
We have added new features, I will send you an updated package in a private message.
If something is missing or does not work as expected, please, let me know

We have added a new interface BGMergeSettingsEntity.ISaveLoadAddonSavedEntityFilter with method bool OnSaveEntity(BGEntity entity). If the row controller implements this interface, this method will be called for each row and if it returns true, the row will not be added to saved data.
Here is an example:
Let's say we have a table "Test" with "save" boolean field, and we want to save the row only if "save"==true
Here are the add-ons settings and controller class:

Spoiler: SaveLoad and CodeGen add-ons settings and controller class

and here is the result of running a test:

Spoiler: Database data before and after running test code

The first row's "someValue" did not change, because it was not saved

Additional optional configs are added to SaveLoad add-on settings

Spoiler: SaveLoad additional configurations

1) Saving is quite simple, you just need to provide configuration name to Save method

Code (CSharp):

1. var settingsSave = BGRepo.I.Addons.Get<BGAddonSaveLoad>().Save(new BGSaveLoadAddonSaveContext("Settings"));

2) Loading is tricky, because by default SaveLoad add-on reloads the default database first and only after that it merges default database with saved data
So, if you loaded your settings first and after that you load saved game, the loaded settings will be lost and will be replaced with the values from default database
We have added an option to skip default database reloading, so you can load your settings on your application startup without database reloading like so:

Code (CSharp):

1. byte[] settingsData = SomeMethodToLoadSettingsByteArray(); //load settings data
2. BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(new BGSaveLoadAddonLoadContext(new BGSaveLoadAddonLoadContext.LoadRequest("Settings", settingsData)){ReloadDatabase = false});

But if you try to load saved game session data after that

Code (CSharp):

1. BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(gameSessionData);

all your loaded settings values will be lost
You can also try to load your game session data without default database reloading

Code (CSharp):

1. BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(new BGSaveLoadAddonLoadContext(new BGSaveLoadAddonLoadContext.LoadRequest(BGAddonSaveLoad.DefaultSettingsName, gameSessionData)){ReloadDatabase = false});

but this will work properly only if
1) You do not create/delete rows in runtime, only update the values
2) All values, which can be changed in runtime, are saved with either settings data or game session data
In other scenarios, it will not work properly.

So, the only reliable approach for now is to load both game session data and settings data on game loading.

Code (CSharp):

1. BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(new BGSaveLoadAddonLoadContext(
2.     new BGSaveLoadAddonLoadContext.LoadRequest(BGAddonSaveLoad.DefaultSettingsName, gameSessionData),
3.     new BGSaveLoadAddonLoadContext.LoadRequest("Settings", settingsData)
4. ));

In this case database will be reloaded first and after that it will be merged with gameSessionData and settingsData one by one and all the changes should be preserved
We do not know, how we could get rid of this limitation, if you have any suggestions, please, let me know

UPDATE: We have added preserve requests.
Preserve request means, that the data for provided config will be saved internally before default database is reloaded and merged with default database after reloading.
So the following code should also work fine:

Code (CSharp):

1.         BGRepo.I.Addons.Get<BGAddonSaveLoad>().Load(
2.             new BGSaveLoadAddonLoadContext(new BGSaveLoadAddonLoadContext.LoadRequest(BGAddonSaveLoad.DefaultSettingsName, gameSessionData))
3.             {
4.                 //the data for "Settings" config will be saved before database is reloaded and loaded after database reloading
5.                 PreserveRequests = new List<BGSaveLoadAddonLoadContext.PreserveRequest> { new BGSaveLoadAddonLoadContext.PreserveRequest("Settings") }
6.             }
7.         );

 

hello. I created a character StatTable and added the following to receive stat information in the form of a dictionary.
However, it does not change even if the key value of enum is changed. help.

 

Hello,

It looks like the popup window (in the 3rd screenshot) does not belong to our asset
How do you open it?

Spoiler: popup window

When I click on the hashtable field cell button, the following window pops up

Spoiler: hashtable editor window

 

Hello, I posted a video on my blog because there is room for confusion with the image. I hope it was helpful to analyze.

An additional finding during testing is that values are not added to the Dictionary .

For your information, the asset is the latest version.

https://blog.naver.com/samplecode/223119046020

 

Hello,

Thank you for the video, it was really helpful!

This is a GUI issue, the issue with dropdown window specifically
A dropdown window is closed once it loses the focus.
The default control for enumeration does not transfer the focus,

Spoiler: default control

but probably you have some Editor extension installed (or maybe Unity has an option to replace the default enumeration control), and this extension replaces the enumeration control with another window and the original dropdown window is closed as soon as another window is opened.

We have added an option to switch from dropdown window to a regular window

Spoiler: switch to regular window

A regular window is not closed automatically, it needs to be closed manually
The hashtable value is updated when this window is closed (either by clicking on x icon or on "Apply" button) or when another cell is selected
You can also revert the changes by clicking on "revert" button
Also, keep in mind, that hashtable can only have a single value corresponding to a specific key, so if you create multiple rows with HP key for example, only one, single value will be stored

I will send you an updated package in a private message
Please, let me know if the issue was solved or not

 

Oops, I was mistaken for writing on a forum, and I suddenly came up with a post.
The modified package worked just fine! thank you so much.
Thanks to you, I'm freed from worrying about whether I should make two lists.
BGDataBase will continue to be with my indie game development.
thank you !

 

Hi @BansheeGz
Can you explain how can we do this :
"You will need to trigger LiveUpdate plugin to reload the data from GSS (for example with button click). No need to restart the game"

Thanks

 

Hi,

By default, LiveUpdate addon ( https://www.bansheegz.com/BGDatabase/Addons/LiveUpdate/ ) is executed automatically one time on database loading
If you enable "Manual load" toggle

Spoiler: Manual load toggle

addon will not be executed automatically, and you need to execute it manually (from C# script) using this code

Code (CSharp):

1. BGRepo.I.Addons.Get<BGAddonLiveUpdate>().Load();

This code will load data from GSS and merge it with local database data
You can invoke this code when some event occurs (for example, when a user pressed on a button etc.)

Please, let me know if you have any questions

 

Thanks @BansheeGz for replying,
i have one more concern, i'm not able to find "Text.Text" property in case of System.single & System.Int32 as it is available inn System.String. I have to showcase number in my text Field. can you suggest how can i do that.
i'm attaching the images for your reference.

 

Yes, I think we need to improve Field Binder by adding options to convert values
We will implement it, I will keep you updated on the progress, thank you for bringing this to my attention!

There are some workaround solutions, but unfortunately, they are not that easy to use compared to the Field Binder
1) You can use a graph binder instead of field binder.
https://www.bansheegz.com/BGDatabase/DataBinding/#graphBinder

Spoiler: graph binder

2) You can create a proxy C# component with a property, which has a proper type, and inject the value from database to this proxy component. The downside of this approach is an additional unnecessary component

Spoiler: injecting value to proxy component

3) You can also use a template binder, but this option is probably the worst one
https://www.bansheegz.com/BGDatabase/DataBinding/#templateBinder

Spoiler: template binder

 

Thank you so much @BansheeGz Really appreciate the support.

 

@BansheeGz my spreadsheet is still not loading at runtime, even after build. if i'm importing at console then its updating otherwise it's not, I even tried your button approach which you suggested. but that is also not working

 

Hi,
We have added convertible values for field binders
The only supported function is "any object to string", but if you need other functions we can add them
Custom functions are also supported
The function is applied to the database field value
I will send you an upgraded package using direct message (conversation)

Spoiler: conversation

To enable the function, turn on "Show convertible fields" toggle while selecting a field

Spoiler: enabling conversion

1) Do you see any exceptions in the Unity console? If yes, please, post it here
2) Also, could you, please, send me LiveUpdate add-on log?
We have added BGDatabaseMonitorGo component for easy access to LiveUpdate add-on log
Update BGDatabase with a new package, set LiveUpdate add-on log level to "Summary by metas"

Spoiler: Log level

and add BGDatabaseMonitorGo component to your scene
Once you run the scene, this component produces a floating window
You can minimize/maximize, resize, move or close it
It should work both in Editor and in the builds
Select "LiveUpdate add-on status" to access LiveUpdate add-on log
Copy the log and paste it here (or DM it to me)

Spoiler: LiveUpdate log

 

Hello,

i have a field called RefNumber

the way I add to my database is using
DatabaseDump.NewEntity(entity =>

#1 how can i prevent a duplicate with RefNumber from being added,

#2 how can i mass clean up any duplicate entries?

Thank you

 

Hello,

Create a unique key RefNumberKey and add RefNumber field to it
Run code generation after that

Spoiler: Unique key

If you want to review the rows, violating the key in Editor database GUI, turn on validation and the rows will be highlighted

Spoiler: Validation

How to avoid adding new rows with the same RefNumber:
If you want to create an Editor script and performance does not really matter, then there are 2 options:
1) Iterate over all possible values and select the first value, which is not used
Use the key to determine if the row with specified value exists

Spoiler: iteration script

Code (CSharp):

1.         var count = DatabaseDump.CountEntities;
2.         int uniqueRefNumber;
3.         for (uniqueRefNumber = 0; uniqueRefNumber <= count; uniqueRefNumber++)
4.             if (DatabaseDump.GetEntityByKeyRefNumberKey(uniqueRefNumber) == null)
5.                 break;
6.         var newEntity = DatabaseDump.NewEntity(dump =>
7.         {
8.             dump.name = "NewRow";
9.             dump.RefNumber = uniqueRefNumber;
10.         });

2) Calculate maximum value and use max+1 as a value for the new row

Spoiler: max+1 script

Code (CSharp):

1.         var max = int.MinValue;
2.         DatabaseDump.ForEachEntity(dump => max = Mathf.Max(max, dump.RefNumber));
3.         max = Mathf.Max(max, 0);
4.         DatabaseDump.NewEntity(dump =>
5.         {
6.             dump.name = "NewRow";
7.             dump.RefNumber = max + 1;
8.         });

If you want to run this code automatically when you add a new row, you can use table's controller
Add the following class to your runtime assembly:

Spoiler: table controller

Code (CSharp):

1. using BansheeGz.BGDatabase;
2. public class DatabaseDumpController : BGControllerOnEntityAdd
3. {
4.     public void OnEntityAdd(BGMetaObject source, BGEntity entity)
5.     {
6.         var dump = (DatabaseDump)entity;
7.         var count = DatabaseDump.CountEntities;
8.         int uniqueRefNumber;
9.         for (uniqueRefNumber = 0; uniqueRefNumber <= count; uniqueRefNumber++)
10.             if (DatabaseDump.GetEntityByKeyRefNumberKey(uniqueRefNumber) == null)
11.                 break;
12.         dump.RefNumber = uniqueRefNumber;
13.     }
14. }

and assign it as your table controller

Spoiler: table controller assignment

After that, RefNumber value will be automatically assigned when you add a new row

Above methods will work fine if performance does not matter, if you need a fast solution for runtime, please let me know

Here is an example script (using previously created key)

Spoiler: batch delete rows with duplicate values

Code (CSharp):

1.         var toRemove = new HashSet<BGEntity>();
2.         DatabaseDump.ForEachEntity(dump =>
3.         {
4.             var result = DatabaseDump._RefNumberKey.GetEntitiesByKey(dump.RefNumber);
5.             for (var i = 1; i < result.Count; i++) toRemove.Add(result[i]);
6.         });
7.         DatabaseDump.MetaDefault.DeleteEntities(toRemove);

 

If I could leave another review on the asset store I would.
Still believe this is one of the most superB supported asset on the asset store.

Regarding the RefNumberKey, any way I can ignore all keys with "0" ?

#2

Code (CSharp):

1.    var toRemove = new HashSet<BGEntity>();
2.         DatabaseDump.ForEachEntity(dump =>
3.         {
4.             var result = DatabaseDump._RefNumberKey.GetEntitiesByKey(dump.RefNumber);
5.             for (var i = 1; i < result.Count; i++) toRemove.Add(result[i]);
6.         });
7.         DatabaseDump.MetaDefault.DeleteEntities(toRemove);

If i use that code, with a slight modification:
for (var i = 1; i < result.Count; i++)
{
Debug.Log($"Found duplicate Number: {DatabaseDump.GetEntityByKeyRefNumberKey(dump.RefNumber).RefNumber} - {dump.RefNumber}");
}
i get hundreds of debugs with duplicates even though they are not? something isnt right with this code?

#3 Can i make the RefKey also be connected to Time, so if both are the same its duplicate if not its not?

 

Thank you so much!

Do you mean when validation is turned on in database GUI?

Spoiler: validation GUI

Currently, it's not possible, but we could add an optional C# interface to add support for skipping certain rows while validating and to add some customization to the validation process in general.
Please, let me know if you are interested.

Modified code prints all the rows except the first one
GetEntityByKeyRefNumberKey method returns the first entity with provided RefNumber value or null if no such entity exists

You can change this script slightly to make it work

Spoiler: modified script

Code (CSharp):

1.         var result = DatabaseDump.FindEntities(null);
2.         for (var i = 0; i < result.Count; i++)
3.         {
4.             var dump = result[i];
5.             //the first entity with RefNumber==dump.RefNumber
6.             var firstEntity = DatabaseDump.GetEntityByKeyRefNumberKey(dump.RefNumber);
7.             //if dump is the first entity with RefNumber==dump.RefNumber, we can skip it because it's not a duplicate
8.             if (dump == firstEntity) continue;
9.             //dump is duplicate
10.             Debug.Log($"Found duplicate Number: {dump} - {dump.RefNumber}");
11.         }

the same code, but ignoring rows with RefNumber==0

Spoiler: modified script (ignoring rows with RefNumber==0)

Code (CSharp):

1.  
2. var result = DatabaseDump.FindEntities(null);
3. for (var i = 0; i < result.Count; i++)
4. {
5.     var dump = result[i];
6.     //ignore rows with dump.RefNumber==0
7.     if (dump.RefNumber == 0) continue;
8.     //the first entity with RefNumber==dump.RefNumber
9.     var firstEntity = DatabaseDump.GetEntityByKeyRefNumberKey(dump.RefNumber);
10.     //if dump is the first entity with RefNumber==dump.RefNumber, we can skip it cause it's not a duplicate
11.     if (dump == firstEntity) continue;
12.     //dump is duplicate
13.     Debug.Log($"Found duplicate Number: {dump} - {dump.RefNumber}");
14. }
15.  

Another alternative code, which does not use the key and also skips the rows with dump.RefNumber==0

Spoiler: removing duplicate rows without key

Code (CSharp):

1.  
2. var toRemove = new List<BGEntity>();
3. //stores already processed RefNumber values
4. var processedKeys = new HashSet<int>();
5. //iterate all rows with dump.RefNumber!=0
6. DatabaseDump.ForEachEntity(dump =>
7. {
8.     //remove the row if there was the row with the same RefNumber value
9.     if (!processedKeys.Add(dump.RefNumber)) toRemove.Add(dump);
10. }, dump => dump.RefNumber != 0);
11. //remove rows
12. DatabaseDump.MetaDefault.DeleteEntities(toRemove);
13.  

Yes, it's possible,
1) You can use int/long field to store time(date) and add this field to the key
Code generator will generate a method, which accepts 2 parameters (RefNumber and time)

Spoiler: 2 fields key and code generator

2) We could add time/date fields, probably they will be much better than int/long fields for time/date values
Please, let me know if you are interested

 

Thanks but I think i need to keep the OrderDate and OrderTime as strings for this,

I really like this last code seems easiest to read:

Code (CSharp):

1.  
2. var toRemove = new List<BGEntity>();
3. //stores already processed RefNumber values
4. var processedKeys = new HashSet<int>();
5. //iterate all rows with dump.RefNumber!=0
6. DatabaseDump.ForEachEntity(dump =>
7. {
8.     //remove the row if there was the row with the same RefNumber value
9.     if (!processedKeys.Add(dump.RefNumber)) toRemove.Add(dump);
10. }, dump => dump.RefNumber != 0);
11. //remove rows
12. DatabaseDump.MetaDefault.DeleteEntities(toRemove);

I think I am going at this wrong though, my mental isn't prepared for this warfare lol. Allow me to try to explain my problem better



When I import a file it goes through each RefNumbers and adds them to the database, however under the same RefNumber there can be multiple entries with the same OrderDate and OrderTime with sometimes different StockQty and StockPrice,

I don't know how to make sure if this 'order' with the same refnumber and times is entered in the database already to find it as its a duplicate and do the debug log,

Please let me know if i need to explain the situation better to make this work,

Thank you

 

1) If you want to determine if the row with the same RefNumber+OrderDate+OrderTime values already exists in the database, you can use the database key
As far as I understood, you want the combination of RefNumber+OrderDate+OrderTime values to be unique, so you need to add all 3 fields to the key

Spoiler: The RefNumber key

Run code generation after key is created
A method DatabaseDump.GetEntityByKeyRefNumberKey with 3 parameters (RefNumber,OrderDate,OrderTime) will be generated

This method returns the first row with provided RefNumber+OrderDate+OrderTime values or null if there is no such a row.
So if this method returns null, it means there is no duplicate row in the database

Here is an example, let's say you have some data (the list of C# objects), declared in the Start method, and you want to insert this data into database in the Process method

Spoiler: C# code

Code (CSharp):

1.  
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class Temp : MonoBehaviour
6. {
7.     public class DataModel
8.     {
9.         public string OrderDate;
10.         public string OrderTime;
11.         public string Type;
12.         public int RefNumber;
13.         public string Description;
14.         public int StockQty;
15.         public string Symbol;
16.  
17.         public DataModel(string orderDate, string orderTime, string type, int refNumber, string description, int stockQty, string symbol)
18.         {
19.             OrderDate = orderDate;
20.             OrderTime = orderTime;
21.             Type = type;
22.             RefNumber = refNumber;
23.             Description = description;
24.             StockQty = stockQty;
25.             Symbol = symbol;
26.         }
27.     }
28.  
29.     private void Start()
30.     {
31.         var data = new List<DataModel>
32.         {
33.             new DataModel("July 20, 2023", "1:45 pm", "Some type", 1, "Some desc", 1, "Some symbol"),
34.             new DataModel("July 20, 2023", "1:50 pm", "Some type", 1, "Some desc", 2, "Some symbol"),
35.             new DataModel("July 20, 2023", "1:45 pm", "Some type", 1, "Some desc", 2, "Some symbol"),
36.         };
37.         Process(data);
38.     }
39.     private void Process(List<DataModel> data)
40.     {
41.         for (var i = 0; i < data.Count; i++)
42.         {
43.             var model = data[i];
44.             var row = DatabaseDump.GetEntityByKeyRefNumberKey(model.RefNumber, model.OrderDate, model.OrderTime);
45.             //if duplicate row is found, print a warning
46.             if (row != null) print($"Found duplicate at position {i}");
47.             else
48.             {
49.                 //no duplicate so we add the data to the database
50.                 DatabaseDump.NewEntity(dump =>
51.                 {
52.                     dump.RefNumber = model.RefNumber;
53.                     dump.OrderDate = model.OrderDate;
54.                     dump.OrderTime = model.OrderTime;
55.                     //todo transfer all fields values
56.                 });
57.             }
58.         }
59.     }
60. }
61.  

The first and the third objects have the same RefNumber+OrderDate+OrderTime values ("July 20, 2023", "1:45 pm", 1), so you need to detect it in the Process method
Process method uses GetEntityByKeyRefNumberKey method to detect if the row with the same RefNumber+OrderDate+OrderTime values already exists in the database
If such row is found, the warning is printed, if there is no such a row, the new row is added to the database
The execution result


2) Also, if you simply want to find duplicate rows without adding new rows to the database, you can modify Process method a little bit by adding a hashset to store processed values instead of inserting new rows

Spoiler: checking for duplicates without adding rows to the database

Code (CSharp):

1.     private void Process(List<DataModel> data)
2.     {
3.         //this hashset will store all processed RefNumber+OrderDate+OrderTime combinations
4.         var processedValues = new HashSet<Tuple<int, string, string>>();
5.         for (var i = 0; i < data.Count; i++)
6.         {
7.             var model = data[i];
8.             //the duplicate row can be in the hashset or in the database
9.             var rowIsFound = !processedValues.Add(Tuple.Create(model.RefNumber, model.OrderDate, model.OrderTime))
10.                              ||  DatabaseDump.GetEntityByKeyRefNumberKey(model.RefNumber, model.OrderDate, model.OrderTime) != null;
11.             //if duplicate row is found- print warning
12.             if (rowIsFound) print($"Found duplicate at position {i}");
13.         }
14.     }

Please, let me know if you have any questions

 

After some testing Super Happy to report - Both work wonderfully thank you