Games [WIP] Hovercrafts Sandbox Prototype (Postponed)

Hi,

Roughly two weeks ago I started that little side project as prototype, about sandbox hovercrafts.
See quick vid in next post.

This project is a prototype, for my other bigger project, in which I want to test options, possibilities and limitations.

Project aim is, to allow players build own simplistic and futuristic hovercrafts, which can travel around the planets. Yes, planets as plural. See video, where hoercraft flies between one and other planet, by escaping gravity pull of one planet and entering into gravity of other planet. In fact Hovercraft as construct, is under gravity influence of all planets, at any given time.


Player Goal
One of player goals, is to capture hexagonal tails of planets.
These tiles will generate score and probably will be source of resources, for which will permit to build more etc.
Hence more captured tails = more resources.


I would like see, where this project may go.

I may add more description over the time, if I find a time.

 

Some work over past week has been done.

Added some additional telemetry.
Latitude and Longitude on local current planet.

Player can move panels freely in the screen.

Nested joints structures.



Also, since I like data driven designs, I converted Json based saving system of construct, into SQLite format.



Adding very brief Player Goal to description.
One of player goals, is to capture hexagonal tails of planets.
These tiles will generate score and probably will be source of resources, for which will permit to build more etc.
Hence more captured tails = more resources.


I really like to get to the point of damage system. But I need have everything stable and possibly no bugs present.

 

Second update on the prototype, featuring wide range of improvements from propulsion control, multiple joints linking, to GUI with drag & snap, based on HTML5.

Also, added for now very crude Gyro GUI panel. To be improved.

For more, see time table below.

0:13 Two Worlds
0:21 Custom Designs
0:35 Flexible Propulsion
01:03 Hover Thrusters
01:20 Hovering Based On PI Controller
01:30 Test Drive
01:42 Manual Hovering Toggle on/off
01:48 Capture Tails
02:41 Go To Another Planet
02:59 Low Gravity
03:14 Spin Out Of Control
03:22 Multiple Camera Modes
03:48 Expand Builds
04:22 Add Multiple Linked Joints
05:04 Stability Test
05:41 Stargazing
05:50 Drag & Snap GUI Panels
06:10 HTML5 Based Moddable GUI

 

My prime project was using lua for modding blocks. It works great as expected.
However, I see limitation of such approach, if I want to use ECS. I know how I could go round the problem, to satisfy lua and ECS, but that way, I won't be able to utilize full potential of ECS.

Therefore, I decided to roll in my own pseudo language for modding, which will be significantly different in terms of design architecture, compared to previous modding tool.

This will be quite a bit of work. But is better start early, instead of later tinkering, how to redesign and modify code, to fit modding mechanics.

The ultimate goal, is so mods can be stored in files, like csv for example.
I got concept of modular approach, so could be shareable across players.
Now getting into prototyping.

I hope some time in far future to get multiplier going, therefore, modularity is critical, to keep shareable files in small chunks.

To keep mods enclosed in safe sandbox, just like lua, no dll based mods are planned at this stage.

 

I must say, for the short development time the result is impressive. Especially the ability to put your craft together in-game and the idea of modding and data-driven design.

 

Thx.
Yep implementing modding can be quite tricky.
And I know, as perhaps many do as well, if not thought about from the beginning of development, later can be difficult to implement.

However, I tested and did prototypes in past of multiple approaches, weighting which would be most suitable. So that helps a lot.

I aim, to allow modding accessibility for wider audience. Hence as I mentioned, I rejected for now dll based modding.
While some surely will prefer dlls approach etc. with later implementation of ECS, that would be potentially very difficult for many modders, due to low familiarity with Data Oriented Design. I may consider at some point tho.

 

Initially posted this in my wrong topic.
So moved here

----

Just quick update.

At the moment I work on stuff behind a scene.
Partially related to moddding, building core system.

But one of my main focus in general is a performance.
What I found common in games like Kerbal Space Program, Space Engineers, Rawbots, Robocraft, is a poor optimization. This may be because of the core design, or used older physics engine, only single core utilization, or maybe even other factors.
Specially issue become apparent,w hen player is trying go beyond relatively (for a game) big amount of components (100s-1000s).

On the opposite site, I find From The Depths, Where building flying machines made of 10 of thousands of components can run quite smooth. I am inspired by amount of optimization there.

Here I hope, my past few months of study on ECS, with some initial prototypes, including writing octree, will help to achieve this goal.

Other things in my scope are determinism and mentioned earlier (for far future) multiplier.
This things I need be thinking from the start, during design and developing.

I think major problem with making game optimization, once is advanced developed, or even released, is that developers either don't bother about optimization, or hard to figure out how to optimize, or simply too complex to optimize, as it would require significant rework, of part of the game.

How and if I will achieve my goal, is yet to see day light. But is on the scope never the less.


All this of course will extend significantly the dev time. But I am not in rush to release. So I will take time, as long it needs to be.

 

Update #4
Jut brief on part of modding core.
So initially I did quick mocap, in OOP paradigm just to prove a concept.
Then started prototyping DOD system.
I think I got significant part working, but is not finished.
I did number of debugging along the line.

So now I am transiting part of the system to ECS, with multi threading support, as it will be easier from there, to expand further.

My nearest goal, is to get to the point, where I can use modding tool, to further develop game mechanics.
I will be happy, when that part will be functional. Or will be my failure
This way, I hope I should be able, to catch most of its quirks and make it as much user friendly as I will be able to.

Surely I got no lack of ideas and concepts, to keep me going.

 

Update #5

Not going into details, unless someone specifically is interested, but running initial stress testing of modding core on ECS, with 100k entities.



The result is promising. But I need run more complex stress testings at some point.

 

Ah, for curiosity I decided to turn off Burst compiler, on exact same code.
I did expect some improvement with burst, but that is just massive.

No burst


With burst

 

Just a quick update.

For modding purpose and to comply with my ECS core, I was looking for most suitable way, to parse expressions from strings, into executable code.

I aim, to be able execute similar to web JavaScript code like. For which, I got visual scripting solution mostly working and ready, from my older prototype project.

However, for now, I have no object oriented solution in mind yet. I may drop it for simplicity. But I want to run predefined and custom functions.
Therefore, I run few small prototypes, to test best feasibility.

Following this, I created topic for that purpose
Ordering math equation from pseudo code.
I decided to use Shunting Yard algorithm.
So far, so good.
But much more work is still needed.
Not that I haven't expected.
I will get there eventually.

Anyway, that is it for now.

 

So far, I got working modding parser, which can interpret such functions:

Code (JavaScript):

1. function Start() {
2.   i = 50;
3.   x = 5;
4.   If100 = 20;
5.   if (i > If100 + 7) {
6.     x = -i;
7.     i = x + (If100 - 5) * 4;
8.   } else if (i < If100) {
9.     x = i + If100;
10.   } else {
11.     x =0 ;
12.   }
13.   j = x;
14. }

Now I need implement recognition of functions inside functions.
Both built in functions like sin, cos, pow, time, etc math access to construct, and world data; and custom functions. For example x = CustomFunction Start ( i );

The thing is, because I need to ensure determinism, I will allow only operations on ints.
Most of floats can be represented by ints, by multiplying 10^x. In worse case scenario, I can round floats to int, or to n decimal points, then multiply by 10^n. Yet I need to think about best solution, and make it as fool prove as I can, without over engineering.

As I mentioned in Ordering math equation from pseudo code. ECS at current state, do not support strings. Hence I have every string representation as int value. Best, is to think about enumerators, with assigned value. For example = is 0, + is 1, - is 2, * is 3 etc.

 

Manged to make custom and built in functions going.
For example:

Code (CSharp):

1. function Start() {
2.   j = 1000 ;
3.   i = Test(j, 7) ;
4.  
5.   j = 1200 ;
6.   i = Test(j, 8) ;
7.  
8.   j = 1400 ;
9.   i = Test2(j, 9) ;
10. }
11.  
12. function Test(m, n) {
13.   k = Math.sin(m);
14.   return k;
15. }
16.  
17. function Test2(m, n) {
18.   k = Math.sin(m);
19.   return k;
20. }

Now I just need complete 'for' and 'while' loops.
Then more tests.

Also I did quite few optimizations, as I got quite few debugging logs, to track what is going on.

For me feels exciting

I want get to the point, where I can start program blocks

 

So it now appears, I got working core for all custom and built in functions, if, if else, for and while loops.
I need to say that dusting off some ancient knowledge about assembly language did help.
However, I haven't used assembly, but I borrowed few techniques.
There is few things, I could approach slightly different, but I am happy so far.

I will be keep testing.

On technical side, I built part of this system, based of few modules.

• First importing code as text.
• Then is parsing of text based code, into lines of code.
• After that, first pass decomposition separates relevant terms, including loops, variables and operators.
• Also it adds special jump lines, to skip, or go back to relevant line of code. For example 'else if', or 'for' loops.
• Then second pass, updates relevant skip lines, with correct line index and applies initial Shunting Yard algorithm.
• Then next modules decouples text based decomposition, into numeric representation.
• Then finally linearizing all data, into continuous arrays, to be compliant with ECS format.

From this point, I can run evaluator, which executes all code.

I could perhaps compress few steps into one, but since is easy to make mistake, this way I can easily track, what is going and where, since for most steps, I have debugging points, represented at enumerators, or strings. So kept readability. (These are only called at script initialization.) At least until linearization point, beyond which, all data is represented as numbers for performance, which makes difficult to debug, while is hard to be readable for human.

Ok, that it for now.

 

Next update #6, which involved prototyping of modding tool.

Commented in topic
GameObject Tweening: Prove of concept using Native Arrays.

So video briefly shows, ECS working with modding tool, where you can code it externally.
Multi threading now is supported.

 

So small thoughts to put away of my mind.
While currently focusing specifically on data driven side of project, such tasks are quite demanding. Is not complaining, but just saying, while passing by

Requires to think and rethink concepts, to ensure, systems and sub systems work as desired, yet not to fall, into spaghetti boloneze code like.

ECS is definitely helping, to keep code in tidy manner. (At least less messy )
Progressing in small chunks, bits by bits.

For sure, making all working in classic OOP would be lighting fast, just as I did first prototype, in comparison to ECS + OOP. But I hope, ECS will be the main strong hand later, in this project.

In the end, I am still working around the core of the game.

I think, while finishing what I am doing now, I should be able to implement multiple source gravity ECS systems, by the end of week. Unless, I will pick up different part of project to do instead. My to-do list is plenty.

 

Another quick update #7.

I hope all had and having good winter break, full of rest, getting ready for next year

This time some gravity implementation, which took me actually less than a very next day, from previous post. Now ECS side supports multiple gravity sources, as much wish for.

Then I worked on improvement of modding tools and bugs resolving.
Further to that, I have implemented built in PID controller.
This also should be possible, to be written with modding tool.

Then some further improvements, bit of cleaning and some small optimization.

Ah, and now I can executed major modding changes at runtime. At the moment quite in crude way, by triggering manually GameObject, to command ECS side.
Not as much, as would normally do, but here it is.

Just a bit more on video, there are 4 hover propulsions, to keep construct above the ground.
They are purely PID driven. Due to strong close gravity source and not fine tuned PIDs, hovering is rather hectic. When craft is moving sideways, the gravity force is acting toward gravity source, which is just few meters below the surface. Means pulling vector is not straight down, but mostly at some angle. This makes PID acting weird, as it looks.

Ok, now lets focus, I got few days untill end of the year, to finish a game

Edit:
One of update also I did, Improved time syncing, between systems Entities and GameObjects, with emphasis on determinism.

I got few directions to go as next. Inputs, octrees, or rebuild ECS building mechanics, as I had shown in Classic OOP on first videos.

 

Welcome back in 2019.

So for time being, after holidays, I was sitting on my side part of project [WIP] Octree In Pure ECS.

Spoiler

I plan to implement it into the main project some time soon.
More about it is on its own thread, if anyone is curious.


On other side, I looked also into Utility AI, and I had great input from community so far.
Utility AI (Discussion)
This is something on my mind, do derive into ECS for as well, and combine with modding tools.
I got brief concept how I want to do it.

And I got few ideas of utilizing it already.

 

Quick ECS based block building prototype, using Octree system.
Hence one step closer, to get to the point, of initial OPP based prototype.

 

Further testing ECS based octree system. This time supports multiple shapes.
It was quite tricky to get to this point however.

 

It's amazing work!! Good Job!


I see that we have some similar thoughts about creating a game as I also have begun with a data-driven architecture, the modding feature and very early optimization.


There are still some differences, such as the fact that I have decided to go with dlls for modders instead of a text file interpreted/compiled at runtime.
I know that the dlls do have the major problem of forcing modders to exit the game, modify their code, compile their mod and then start again the game, which takes some time, whereas your solution is more efficient from a modder point of view.

I must admit that I have chosen the easy path with the dlls...
That said, the thing wich took me the most time wasn't giving modders access to the code, it's giving them access to all the assets.
Being able to remove/modify/replace/add any asset in the game from a mod (including modifying/removing/replacing the assets of other mods) takes a lot of time to do, especially when you are insane enough (as I am) to allow them to modify the animations, shaders or the worse of all, the UI panels...

Modding the UI is the most difficult thing I have seen so far (don't worry, I still have a very long road to go, I'm sure I'll see more difficult things).
I have a lot of limitations, modders can only change the skin of all the UIs or add new panels, no removing or replacing there, it would be just too much to do for me.


As for the data-driven architecture, have you looked at Scriptable Objects?
I find them extremely useful to configure the game using data instead of code. Most of the hard-coded values and tiny behaviours can be replaced by Scriptable Objects, and so they can become configurable directly from the Unity Editor.
As a bonus, if the modders do use hte Unity Editor to create their mods, they will also have access to that feature and will just have to create SO, configure them in the Untiy Editor, and drag/drop them where needed.


Also, I want to thank you for the tests you do with the ECS and burst.
I planned to use an ECS about 2 years ago, so I developed my own ECS framework in order to see the viability, and 6 months later Unity announced their own version...... lucky me....
I am still unsure if I will use my own version (designed to be easier to use than Unity ECS, and very easy to mod) or the Unity ECS.
The Unity ECS seem to be more flexible, but I was unsure about how easy it would be to mod it, I guess you answered that question for me, thanks!

In the end, I guess the choice will be made by the performances. My own implementation do not rely on allocating new arrays every time as I think Unity does (still unsure, I haven't taken a close enough look at it yet), but my version has the disadvantage of not maintaining the entiies in the correct order, except when manually re-ordered.
Anyway, I shouldn't have a use for it before the middle/end of the year probably, so I still have some time to decide.


Well, anyway I'll continue to follow your work, as you seem to have some similar needs than me, and you are actually doing some amazing demos and performance tests.

 

Hi, Thank you your interest and insight on similar modding approach.

I shall go from the back.

Unity ECS do not maintain entities order as well. But I do explicitly, to keep them in relevant arrays (Buffer Arrays in case of ECS octree). So if you approach similar way, that wouldn't be an issue for you.

Yeah. Burst is amazing piece of thech. for massive performance bust.
There is also Entitias. Did you looked at it before? Now I think is released for free, since Unity introduced own ECS.

While still way to go, I am sure, Unity team will pack ECS with many features, which can become very handy.
So if I were you, I would massively consider shifting to Unity ECS. Also we know, they focus on super performance. But any bugs, they will handle, rather you would need divide your work, with your own ECS. That in case, you find any bugs. But I think, with Unity ECS, you will be better of in long term.

To be honest, I never have been convinced to use SO. Is just me. Maybe I don't appreciate them enough . I read about them much, and I see people use them here and there. There are even some examples from community, using SO with ECS. But I try focus on data from files, as you indicated earlier. Hence SO may be good for me in editor, but I would convert them eventually to read files / Data Base.

For that I use HTML5, which takes all burden away from Unity based UI. Also, is much easier to mod and test, without even running a game. Then I can potentially make similar layout for web page and in game, if I will fill alike. And much more.

I wouldn't necessary say is an easier way. Surely is different. My approach is surely much more time consuming to do so. I do consider in future, potentially allowing dlls as well. With ECS it can be apparently quite easier, since is more about hooking up right system. And you good to go.

But because I need stay focus on performance, I enforce certain approach, ensuring that mods will not hinder it. As they tend to do, when start plugin many. Also, ECS may be less friendly / familiar for modders.

In case of my approach, I got additional tool to help modding. But for now I do not discuss it.

There are way of loading dlls at runtime. Or even compile the code at runtime. These options may be wort for you to look at.

 

That's good to know, then maybe I have the same performances than them then.

It is the thing which made me go for my own ECS, mine is a lot more performant, there's 0 garbage collection and no runtime allocation (provided the initial arrays are large enough, or you have to enlarge them, as for a List).

Yes, that's the main thing that will probably make me go swtich to Unity's ECS.
Currently, my ECS lacks multi-threading, but it's already planned in its architecture, I am only looking for a good way to indicate that a system must be executed before/after another one.

I saw that!!! I'm very very thrilled by Burst, the only downside to me is I absolutely need to use Double for a lot of things, as my game takes place in a star system I need the extra precision for the orbital calculation.
But I hope that they will make Burst available for Double in the future, after all it's probably nearly exactly the same code.

It took me a while to find out how to to use them in my game.
But now I have made them the central place for nearly every data I need to manage. I have even retrofitted some old code in order to get rid of my old configuration system and use SO instead.
In fact, you can consider them as a class with nearly no code, nearly structs except for the way they are allocated.
The huge incentive to use them is that they are automatically serialized by Unity, so they are automatically integrated in the Unity Editor, meaning that you can configure them directly in the Editor, and drag/drop them easily.
For example, I have all the types of ships in SOs (one SO per type of ship), I also have all the factions.
When I create a ship, I just have to drag/drop its type and faction and it's configured!
And the modders can do that too!!

There I am lost. HTML is a technology I simply do not even want to try to go any near to (even reading its name hurt my eyes).
I know it's irrational (and probably very dumb), but it's like that.
And honestly, I have spent so much time already trying to figure out how Unity UI works that it would hurt me badly to stop using it...

Here I have the exact opposite approach. I will probably integrate in my game a tool for the modders to modiy/replace any of my own methods, opening completely the game to them.
Any bug/performance issue is their own responsibility, I have decided not to try to ensure any kind of safety as it would drastically limit the extent of their creativity.

On my end, I have decided that at first it will be with Unity, and after the release, if I still have the resources, then I'll create an editor.
But honestly, Unity's Editor is quite efficient and can be used for modders provided you add a few custom inspectors and a window with a few buttons to create the mod automatically.

Yes, I know that, but I have made a decision very early in the dev which forbid the use of dynamically loaded dlls.
I have decided that everything must be loaded during the initial loading screen of the game.
I made that decision in order to reduce the loading time between menus/scenes, and also in order to have a database containing every asset/data. That way I can access anything very easily.
As some data need to look into the code in order to be created (the ECS components and systems for example), I need to do that at startup.

Anyway, it's a design choice, and as always it has some good things and some drawbacks, I guess we just have to accept the drawbacks of the decisions we made before and go forward.
I know that modders will have to exit/restart the game very often, and that players will have to exit/restart the game when they add a new mod or disable one. In exchange for that they will spend nearly 0 time loading data less while playing, and it's also easier for me to manage the resources.

 

I was modding in past. This was one of major draw,backs for me, since I was trying to crack the game and implement feature, for which most community was asking for long time. I was progressing and had working results. But making fully functional was requiring tons of debugging. Which means restarting game all the time. Massive time and motivation killer, if wanting making something more in depth.

For that reason, I have decided not go for that route. Rather expose what need to be exposed, and allow modding within a game. this way, I left for a purpose, control over the very game core. But I can amend that later.

I was thinking about it many times over. And indeed, using Unity as modding tool has massive advantage. But just because to above, I decided to not go that route. Weather right, or wrong. Well, time will tell

Yes there is risk on that part. And can be tricky.
However, myself to deal with that issue, I decided, that most of game I will be writing as mods. Other than core mechanics. Which means, I will ensure, that everything is exposed and tested from the beginning.

Some time over year ago, I had been also developing UI system. But I wasn't happy with it. Eventually, I scrapped month of work, and turned into HTML5, which was much more progressive for me, since I know it, quite ok.

With current ECS approach, I got very little game objects in scene. Most stuff is generated. But maybe I will convince my self one day, to take a look more into SO.

That would be major candidate for switch to be honest. Unity ECS, or even jobs run multithreading out of the box. All safety and race conditions are dealt for us, behind scene. You could really focus on game dev, rather trying figure out and test threading mechanics and syncing. Which can be pain in back side.

Unity ECS has mechanics, to handle after and before. These are not ideal at current stage, but option exists, and better system is on the road map.

Myself I haven't been using Entitias, but didn't expected, it generates GC.

If you good at it, and got direct access to Unity game engine with C++, there is that probability. But if you work on C# level, I don't think you could directly beat it, since all the chunks and cache and memory management etc. Whole magic in general. Just beyond me

 

Yeah, I remember that, which is a shame, you were doing quite well modding From the Depths.
At some point Nick optimized the starting sequence, dividing the loading time by 3 approximately, but it's still a huge loss of time for modders (and devs!!) to wait for the game to start.

Maybe at some point I will let me convinced to add a way to load dynamically the dlls in order to avoid that massive problem, but I will not do it for the whole mod, it's really handy to load all the assets at startup.
And beside that, for now the game (with about 500Mo of assets) loads in a bit less than 12s on a HDD (including 3.4s before the very first call to 'Update()', I don't know what Unity does during that time...), so I hope that the full game will not take more than 25-30s to load.

SO aren't necessarily linked to a GameObject, I also use them to define/configure the keys, contexts and actions, in that case they are just configuration.
As a SO is a file, it is in the Unity Editor, so you can configure it. That way I can configure the keys, actions and contexts easily, and they are loaded at startup as I would for any file (except that it's automatic because of the Unity serialization).
That said, I am still at the very beginning, maybe I will not find them so useful in the core of the game.

I don't thing it generates garbage, from the code it's using a lot of pools.
But when I looked at it about 1.5 years ago, it was using classes, not structs, that means that it made poor use of the data cache memory which remove most of the performance advantage of the ECS.

Well, from my tests, my framework is consuming a bit less that 100 cycles to process an entity, which is about nothing and nearly comparable to just loop over a list.
I achieved that by placing all the data in arrays of struct, ensuring that the data are packed in memory.

Honestly, the very thing that I like about my ECS framework is the ease of use.
I tried to look into the Unity ECS (well, honestly I haven't spent more than one hour on it, so...) and it seemed a bit verbose to me.
Here is a small example of how to define components, systems and entities with my framework:

Code (CSharp):

1. struct Data1 : IQComponent  // Just inheriting a struct from 'IQComponent' is enough to make it a component
2. {
3.     public Int32 i1;
4.     public Single f1;
5.     public Health TheHealth;
6. }
7.  
8. struct Data2 : IQComponent
9. {
10.     public Vector3D Pos1;
11.     public Quaternion Rot1;
12. }
13.  
14. class TestSystems  // Name unimportant, the systems only need to be static and decorated with the 'QSystem' attribute, they can be anywhere in the code
15. {
16.     [QSystem(QSystemExecutePoints.Update)]  // More execution points are available
17.     [Triggers(Trigger.OnChange, typeof(Data1))]  // More triggers are available
18.     static public void TestSystem2(Single DeltaTime, Data1 D1)
19.     {
20.     }
21.  
22.     [QSystem(QSystemExecutePoints.Update)]
23.     static public void TestSystem3(Single DeltaTime, out Data1 D1, ref Data2 D2)
24.     {
25.         // 'out' components are write-only, 'ref' components are read-write, and the others are read-only
26.     }
27. }
28.  
29. // Entities are easier (and faster) to create when they are built from a blueprint, but it is possible to add/remove any component at runtime to/from any entity, even if based on a blueprint
30. interface Blueprint1 : IQEntity  // Just inheriting an interface from 'IQEntity' is enough to create a blueprint
31. {
32.     void InitComponents(Data1 D1, Data2 D2);  // This is optional, but it makes it easier to initialize the components
33.  
34.     Data1 D1 { get; set; }
35.     Data2 D2 { get; set; }
36. }
37.  

I honestly think that it's not possible to make it less verbose. In fact, when I designed it I asked myself: "what is the absolute minimum which is needed to define a component, a system and an entity?".
And then I tried to make it so the user only has to write that, and nothing else.

Of course, behind the scene there's a lot of automatically generated IL code in order to generate the 'InitComponents()' methods for example.
The components also have an automatically generated 'Equals()' method in IL code in order to make the comparisons a bit faster compared to the standard 'Equals()' method).

So, the ease of use is the main reason I have cold feet about Unity's ECS. Even if I know that they will continue to add very well optimized features in their ECS over the next few years, and I will never be able to compete with that.
I guess that once I try Unity's ECS, I will get used to it and it won't be a problem anymore.

 

I am positively impressed on your work. Yes that is definitely less verbose than Unity ECS. Not sure, how far they want to shrink the boiler plate code. They did already quite a bit progress on that. But is far from what you got.
Job systms may be similar in length.

That is nice an tiny. Don't know how long takes to iterate each entity in ECS, but since they mention about indexes, I would expect just a bit fewer cycles.

But hey, if it works for you why not

But you need safe multi threading for sure.

Since you got experience anyway, it wouldn't be any big change for you. General logic shouldn't change, to port it at some point, if you will feel about it. Other than adding more boiler plates

I am glad you brought it up. If I recall, you had some useful mods there too. And I even looked up back on their forum, so I see you are quite active I remember something about speed up of loading time. But not sure if it was after, or before I "left".

Imagine FTD was on ECS

But definitely FTD is one of few inspirations, for this project.

How is FTD standing this days btw? Still very active?

If you manage to keep asset loading to low, I think may be feasible. I think Unity collects hardware data, or other checks / telemetry.

These are links, you may be interested
https://forum.unity.com/threads/double-support-for-burst-compiler-and-new-math-library.520394/
ECS halve year old thread on floating origin.
https://forum.unity.com/threads/ecs-floating-origin.546472/#post-3606188

I was trying find more recent post regarding double and burst, which was just mentioned. But I failed.
I think content was saying, that burst already support doubles, but performance is not as that great as floats.

 

Well, I spoke about it with Nick more than a year ago, and he was interested.
There are a few ECS source files in FS (Forgotten Shore, a to-be-released-when-it-will-be-finished game based on FtD code), so I suppose Nick tried the ECS a bit.
But yes, in fact for any block-building game an ECS is perfectly suited, not only for performances, but also for the ease of adding/removing/modifying blocks features.

It's calmed down a bit these last 3-4 months, but the good news is that I'm pretty sure to have finally fixed the very last leaking-explosion bug!!!
I must admit that I have been working frenetically on my own game for the last 2-3 months and I haven't even finished the steam engine update (shame on me... but I will work on it again beginning next monday and update the boilers!).
With the tons of new models, the new missiles system and the new steam-attached-propeller system (to be finished ), I guess things have been going at a good pace last year.
And I'm sure that when Nick will have finished overhauling the UI and the way data are saved in blocks and exchanged over the network things will go very fast again.
And this year we have great hope that some of the AI will be (finally!) updated.

Thanks for the info, I will have to test the performances of double with and without burst.
But anyway, I don't really have a choice because I'm pretty sure the floating origin won't work well to describe entire orbital trajectories in order to find out the closest points of approach so that the AI can try to maximize/minimize engagement duration or distances or anything it will need to do.
Meaning that I will probably have a lot of trajectories to try and a lot of math with Double.
Maybe I can try to do some of these calculation by sectors when I find a point of interest such as distance under X between 2 trajectories.


And I am very excited by the fact that Unity is now taking optimization very seriously, I'm sure that it will provide us with great opportunities to have deeper and more complex games.

 

Ah right, Forgotten Shore meant to be a thing. Sims is have been delayed by many months now, for FTD updates.
I wonder, how diffucult for FTD/FS would be, to implement ECS. At least jobs could be feasable. But Did he port eventually to 2017? I know he did for Unity 5, if I am not wrong. Which was pain already.

That is definitely major benefit for blocky games indeed.

Thx for sharing. Oh but boilers are not meant to be thing year or even 2 years back?

Just checked their forum, but not sure what is involved. Any idea how what AI system Nick is using / planning to use?
I assume these wont be BT. Utility AI, or other?

You can always have scaled representation, to fit all you need. You may loose some precision if scaling down, but you should get pretty accurate trajectory anyway.

Yep, this is why I moved completely form my older prototypes, based on OOP, to ECS. Literally redesigning all. But sure benefits of previously gained important experience.

 

FtD is now using Unity 2017.3.1, I constantly advise Nick to update very regularly to avoid having too large update to do.
Unfortunately, FtD is still using some old particles, which means some amount of work to update to the latest Unity, but as the next Unity version will have a new garbage collection, the reward may be great enough to update the particles.

Speaking about that, I think it is important to update the version of Unity at least every 2 versions, in order to always be enough up to date to avoid having a major blocking problem.
I lagged a lot to go to .NET 4.x because it broke my garbage-free hack for the 'StringBuilder.ToString()' method, the result has nearly been catastrophic.
When I went for it last month, it simply crashed Unity without any crash log or any useful information in the log. It just crashed.
I had to use dichotomy on the repository in order to find out the commit which added the wrong thing, it took some time to go back more than one year before and observe that Unity crashes if you allocate too many small chunks of memory in .NET 4.x and then change scene.

So, update often, use the latest of Unity as much as you can, even if it means having to refactor something quite heavily.
That's the advice I give to anybody (and to myself!!!).

Yes, but they aren't balanced at all, so now that I have added a system to connect pistons to a shaft which is connected to reducers and ultimately to a very large propeller, I now must update the boilers to make them more balanced and more interesting.
That should result in a full engine-to-propulsion system which should be able to compete with the others (with its own drawbacks/advantages).

If I understand it well, FtD is using some sort of Utility AI equivalent, for choosing the target for example, you configure the weight on some things, and the target with the highest score will be targeted.
I have no real details about the new AI, but I do not think it will use any 'AI' algorithm, it will mostly be a very configurable piece of code, as Nick want it to be easily moddable so that people can come up with their own AI.

That must have been an epic work!
That's also the thing I fear a bit about the ECS. While it's relatively easy to find out how the concept work, I still cannot wrap my mind around how to implement it in a real-life project.
I mean, how to choose the components? How to build the entities? What systems implement?
ECS is a completely different way of thinking, and I must admit that, for now at least, my mind doesn't seem to accept to cooperate as I would in order to produce ECS-adaptable ideas.

Fortunately, I am still at a stage where I am implementing low-level and medium-level tools and I delay the time when I will have to implement the actual gameplay by finishing all the graphics part, networking, saving, etc. first.
I hope that by the time I am ready to write the gameplay there will be enough example on how to design an ECS (as opposed as 'how to code a basic example in an ECS') so that my mind will finally understand how to work with it.

 

Yep. It wasn't without struggle. Specially, starting with ECS, when very little tutorial were available, and ECS itself is dynamically evolving and changing. Means need follow and keep updating constantly, every 1-few months.

I mean, you have own ECS, so that should not be that difficult for you, I presume? Same way using Data Oriented Design, struts etc. In contrast, I had to learn from scratch.

Personally it didn't took me too long to switch. Now I can work both in object and data oriented paradigms. So I can bet, once you step up there, you will be good in no time.

This is something which evolves along with your project. As long you keep Single Responsibility Principle, you should be fine.

Starting from ECS examples, is good beginning. Then trying make something out of it, and you will be fluid in no time.

This is a bit of down side. Current Unity OOP and assist with editor, give massive productivity bust. But if trying looking at maximum performance, going lower level can be quite time consuming. Something I could cobble in Unity Editor in few days, it took me weeks in ECS at current state. But I had a clear goal.

On that side, yes, it may be worth to wait a bit. Unless you want to get you head starting at least, to grasp the concept. But there are thing, which ECS devs know, will change and will need adapt to these. So need to be aware of it. In the end officially ECS is not for Production Ready yet.

I just recall some FTD sliders for Naval, Air AIs cards. Could set distance, type of enemies etc. Which makes me think yes, Utility AI probably was used.

Here also I want allow certain degree of flexibility, in terms of AI design. Got general concept, but need more desing details, as you will be aware by now. I think generally is good direction.

Ah balancing ... Yeah this is probably one of most challenging aspects, for multi components based games. I could compare as if with Utility AI. Tweaking here, tweaking there, testing, and breaking all other stuff
Allowing mods makes this task no much easier. But I suppose, this will have to be dealt by community/mod devs.

Oh that nice. Massive improvement since Unity 4 I remember big discussions, and postponing transition between 4 and 5. Following breaking completely wheels colliders and more. Btw, FTD wheels should work now more decently? Some time ago, I remember mentioning, there was only support of 20, or so wheels at the time. Weather was correct information. Which was a bit upsetting me and didn't want try back to modding.

I can imagine a pain ..

I do try keep relatively frequent updates too.
But do understand, some devs don't want, or can not afford for frequent switches. May be not worth for short living projects.

 

Well, I've create an optimized ECS framework, but it doesn't mean I know how to use it...
Same thing for people who build race cars, they can't drive them as fast as real pilots.
I guess I will have to learn how to drive now

Well, I hope you're right. In fact I know you're probably right, but I can't help it, I always fear things I never did before.
But until now I have done a few dozen things I never did before, and I've done all of them better than anticipated, so I let's hope that this one won't be any different.

I know what you mean.
In order to make things fully moddable, I do not use the Unity scene system.
I deconstruct the Unity scene in the editor in order to create a lot of configuration files, and at runtime I load these configuration files and rebuild the scene one GameObject at a time from scratch.
The result is that it's now completely moddable and it's also a lot faster to 'load' a scene (I don't know what Unity does when it loads a scene, but mine are now instantly loaded).

The downside has been that while loading a Unity scene has virtually no dev cost, it took me months to deconstruct it and rebuild it...
But in the end, I have the Unity editor to build my scenes efficiently, the modders have full access to anything in the scene so they can do anything they want, and in addition scenes have 0-time loading, so I'm happy to have spent these months.

Nick implemented a new Wheel system in order to get rid of Unity's one.
It can now go over 20 wheels and it's a lot more stable.
It still isn't perfect, but you know how wheels are, it's always very hard to make them behave in any possible situation.

I think it's a mistake not to update, even for short-living projects.
Of course, when the project is 'finished' there's no need to update anymore, but during the life of the project it's usually not so long to go for a new Unity version.
They usually do a good job with that, and the steps are usually small enough so that they are easily manageable.
Also, it's often easier to integrate 3rd party assets if you are up-to-date (at least, for well maintained assets, but I avoid the ones which aren't regularly updated).

Not updating may prevent access to some really important features which would improve about any existing Unity project.
The next Unity update will integrate a new (an more efficient) garbage collection system, which will be the first step to another one which will be even more efficient.
Think about it, FtD may not stutter as much as before... Ok, it will lose some FPS overall, there's no magic in all that, but it would probably be a better experience for the players.
And I'm pretty sure that a lot of games would benefit of such a feature.

Also, if Nick hadn't updated to Unity 5, he wouldn't have been able to use the PBR shaders and most of the new assets he included in order to completely overhaul the graphics of FtD.

I am convinced that as long as a project is still updated, its engine must also be updated.
Except maybe in some industrial/military/medical/nuclear environments where you need to have some certification on the tools you use. But last time I checked, in games there's no such requirements

 

I guess is just a barrier, similar to mine with Scriptable Objects

Oh that is interesting. I din't know you can rid off scene almost totally. Do I understand right? I assume this approach is outside Unity editor, after game build?

While results seams nice, I don't want to go that route. I mean time wise.

This is nice to hear. Finally wheels, after so long waiting

You know, depending on project of course, every update may not necessary be beneficial.

Agree. Generally I try to limit number of assets, which may potentially break, to minimum. That is assets, where scripts are involved. I often decompose such, ensuring i got max compatibility with project.

I agree with that. However, we knew that FtD was long term project, even after release.
And we know, many devs stops at very first release. With maybe some small updates occasionally.
2017 upgrade definitely gave a massive performance bust.

That must pleased many players

 

In fact, I decompose the scene by writing one configuration text file for each GameObject.
These configuration files contains the position/rotation, the mesh, the material, and some more information about the GameObject (along with its attached MonoBehaviours, etc.).
That is done in the editor (or can be done manually if you just want to do a simple modification over existing files, such as to change the mesh used for one GameObject).

Then, at runtime, when the player press the 'Start' button, I go to an empty scene, and from there I load all the configuration files and reconstruct all the GameObjects using the dta stored in the configuration files.

So, I do not completely get rid of the scene, I still use an empty scene, mostly because Unity does a few clean-up when there's a scene change (like getting rid of the unused assets a nd calling automatically the garbage collector).

The main problem I have for now is that it means I cannot use Global Illumination, as it must be baked in the editor.
I would need to be able to bake GI at runtime, just after having re-created the scene, unfortunately there's no way to do it with Unity, and there's no asset doing exactly that (either it's not at runtime, either it's not creating a lightmap and using the vertex-color array instead).
Anyway, I still have plenty of time until the release, so I hope that either Unity or someone on the AssetStore will finally come up with a way to bake GI at runtime.

I understand, in addition to that my solution has many limitations, as I do not export everything, so I do not have access to all the Unity features (when I need something, I add it to the export/import, which takes about an hour or so).

You will need a way for modders to replace/add textures and meshes.
If you place a limitation to modders to only mod blocks, and not the 'world', then it's quite easy, at least the way FtD does it is quite easy.
My game is a simulation, not a build game, so I needed to provide modders with a way to modify/create a full scene (the interior of the ship) in addition to the ships/missiles/planets/etc.

So I have chosen a very global solution, which has cost me a bit more than 6 months of work, so yes, if you do not need it, I absolutely not recommend you try something like that

I do the same, I currently have 29 projects in my solution.
Each asset containing scripts has 2 projects, one for the runtime and one for the editor.

I have even split my own code in several Dlls:
- one for the C# standard tools
- 2 for the Unity standard tools (runtime & editor)
- 2 for a standard game which contains a generic main menu (runtime & editor)
- 2 for my specific current game (runtime & editor)
The idea is to be able to reuse a lot of code for the next game.
And I am planning of adding more dlls, in fact internal mods, in order to organize the game itself a bit better.

Strangely enough no.
Players have been very reluctant to use the new graphics despite the fact that it's really better (especially the new water with reflection and some cool features).
There are 2 reasons for that:
- it's way better, but not polished enough (for example bloom is too strong)
- it's a lot slower if you do not have a good graphic card
A lot of FtD players are using a laptop with an integrated video card, meaning that the game went from 50fps to 5fps, which is not good...
So, it's now possible to have the old graphics or the new ones. And now everybody is happy.

That's a thing to remember for your game: the targeted hardware.
It's very important to make it clear to the player what is the target.
For example, my game will not be playable on an integrated video card, it's a choice I have made very early as I want to have a minimum standing for the graphics.
So I have a computer which represent what I want to be the lowest hardware my game will work on, and I will use that computer to check that the game runs fast enough with minimum details.
And I have another computer to check the game in normal details.

I think it is important to be clear on the targeted hardware soon enough in the project so that you make the important choices when necessary.
Which is why, like you, I began by experimenting with some algorithms to check the performances.
But that won't work with the video card, there you need to have real models with a complexity close to what you will end with, same for the textures sizes.
For example, in my game I already have about 1/5 of the models/textures and I already consume about 1Go of VRAM, so I know that I must be careful about that because when you exceed the VRAM of a video card, it starts using the RAM, which result in a massive and unacceptable (at least for me) slow-down.

For information, there's a very good place to know what is the hardware of the players:
https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam
It also shows the resolution used by the players, which is very important for the UI layout.
It also indicates the language, which may be useful if you're planning to localize your game.
And it's updated very often, which means you can try to predict what will be the average player's computer at the time you're planning to release your game.

 

That makes a bit more sense. While I do not decompose scene as such, similarly I also generate scene from data, rather having pre-built things. With few exceptions, where Game Object is must.

But I expect a bit longer startup as we discuss, since I read files, and garbage may be generate at initialization time, or when scripts are modified in game. But that should be it for GC. And I hope, at runtime with ECS, should be GC free.

Not sure how feasible can be, but I think ECS brings opportunity, for baking at initialization / runtime at demand. Maybe even someone works already. But that are just my speculations atm. Sure quite plenty time.

I leave graphics for very late part. Is not mine main agenda. At least not any time soon. As you said, some new cool stuff my come up along.

I got prototypes, which load both meshes and textures at runtime. But is not integrated with my current project.
I am planning to utilize it, and apply with scripting mechanism. Which means, I will allow change meshes and textures at run time, even event based. But that is for bit later.

I do expect in my case, quite fast startup, up to menu. Then load relevant assets at demand as well. Loading constructs or full scenes may be more demanding, since I hope complex structures, with some fancy blocks mechanics. But after than, my goal is to keep smooth. Whose not?
Hence periodically I run stress test.
I remember Nick mentioning at Unity 4, regarding profiler. That wasn't an option for me back days.
But I do imagine now, how much more complex was to debug game performance before.

So my understanding is, in your project you will have full models rather than constructed models. Then modding / scripting "on top of them"?

I was considering converting some own assets to dlls. But updating and modifying them frequently. So that would be a hurdle for me. However, with ECS again, is much easier now, for modularity. That is very handy.

I was trying using in past definition assembly files. But the loading performance and productivity result was afoul.
Apparently much improved with later Unity. But I haven't tested. So I may try come back to them once more.

This is interesting. But even myself, I percived FtD as more important in terms of gameplay, rather graphics. Even I was attempting writing mods, to improve graphics. Not avail back then
While graphics can be nice, I see, that targeted audience, or I would say ability to target wider audience is surely beneficial.

I would love be able provide gameplay for intergrated hardware. But I have seen recent ECS topics, where some dev was struggling with performance. I don't know specific circumstances, other than few screenshots, from samples. But that may be potentially a limitation, to reach older hardware.

However, I do work on now 5 years old rig, which in back days was high end machine. If I can ensure smooth ride on it, by the end of first project release. Probably major players will have such computers as standards. If not already.

Here LoD can be fery useful. I understand, in your case may be harder to achieve, do to nature. With blocks, should be easier, as there are algorithms, to reduce poly counts, for many neighbor blocks. Not to mention ECS, which has superb support, for over 100k blocks, providing LoD is executed accordingly. Latest Unite 2018 LA, shows off the massive city demo.

Spoiler

I want to utilize it eventually.
Worth to watch full talk btw

Yep I have seen it before. But good to have it for reference. Thx. It is indeed good feedback information.

 

No problem for the garbage generated at startup or while loading a scene.
You just force the collection when it's done, and after that you won't stutter if you don't produce garbage, it just add 0.5s to the loading sequence, it's not a problem.

But with a block game using FtD technique to have a lot of blocks, you will probably generate garbage.
FtD is concatenating the blocks using the same texture in one very large mesh. When a block is destroyed, you just change the alpha of its vertices inside the large mesh and it just 'disappear', and it's easy to make it appear again.
Unfortunately, in order to do that you must provide the entire color array to the mesh, and last time I tried that you had no choice but to use the Unity setter which generate a copy of your array, generating garbage.
I think that now Unity has added some better way to do that in order to avoid having to recreate an array of the correct size and I think it can avoid garbage, but I'm not sure.
Take a look at that:
https://docs.unity3d.com/ScriptReference/Mesh.SetColors.html

There may be other places where Unity force you to generate garbage, the UI for example. But as you use HTML5 I don't know if you'll be having that problem.

As I am not very good at graphics, I have decided to start with it. Also, I expect that it will be the bottleneck for my game, so I prefer to start with that in order to see what I can, and can't, do.

Unity's profiler is quite good, but it has several problems, including:
- for FtD it requires more than 16Go of RAM, and it's so slow that if you go deep in the call hierarchy it may take several dozens of seconds just to open the next item... at some point, you just give up
- on my game, it crashes when running for too long (about 10mn). It's a new problem, it came with Unity 2018, I had no problems before
Other than that, it's very helpful to track garbage, just be careful that sometimes the Unity Editor generate garbage which won't be there in release, so you have to profile the release build from times to times.

Yes, modders will be able to add/replace meshes/materials/shader/animations/sounds/etc. for full models.
It is not a building game, it is a simulation, so they will be able to add whole new ships, factions, etc.
The only possible configuration will be for some additional equipment on the ships, such as the turrets, guns, decals, radars, etc.

It is not yet very efficient.
It works better, they got rid of most of the bugs now, but it's still a bit slow due to the connection Visual <--> Unity I think.
When I add/rename/remove an assembly, I close Visual Studio, I do the job in Unity, and I open Visual again. Without that Visual hangs for a very long time (more than a minute sometimes, when it doesn't crash...).
Closing Visual when modifying the solution in Unity is very efficient and I had no problems since I began doing that.

LOD will be a thing that's sure. When a ship is far away there's no need to render all 12k triangles and full texture.
But if I want high quality when close to a ship I need very large textures, and these are to be in the memory.
In fact, LOD for textures (MipMap) consume more memory as it requires you to have several sizes of the same texture in memory.
It can be offloaded from the VRAM while not used, but it takes some time, so it's always better to be able to stuff everything in the VRAM, which shouldn't be a problem considering the radical increase of VRAM on the video cards these days, I have no doubt that in 3-5 years everybody will have at least 3Go, if not 4.

For the meshes, LOD will probably be very useful, I haven't yet done that so I don't yet know about any drawback to it, but I suspect that memory won't be one as meshes aren't very VRAM-hungry compared to a 8k texture.


I saw that city demo about a month ago, and my jaw is still wide open.
They said that the project will be provided at some point, I can't wait to put my hands on it and learn all their secrets


About ScriptableObjects, I recommend that you to watch this whole video (1 hour).
I used it when I was deciding how to store the data used to configure the items in the game.
After having seen it I decided to scratch what I did previously and to use ScriptableObjects for everything, I created a class named 'BaseData' which inherit from a ScriptableObject and has some cool features, and as its name indicate, I use it for all the data in my game.

I know that you're not there yet, but at some point you'll need it, this one is good to help optimize rendering (1h, beware, this one is tough, I still struggle on most of it...):

This video shows how to work better with the assets, it's interesting if you want to know a bit about custom editor (a bit of SO, but mostly custom inspectors and metadata), starts at 13mn and finish at 30mn (the other parts are also interesting, depending on what you know/need):

 

I remember, when FtD was using Instantiating and Destroying blocks, before color changing was implemented. Oh that was many years back, when nick announced this changes. It may have been even before Steam release, but I wont give my head for it. This stuck in my mind so much, even before I was really into Unity.

ECS forum has nice multiple discussion about mesh vertices managing. But I think, I will go even different way, not to worry about vertices manipulations, but simply disabling and resizing blocks accordingly. This may leave me with bit more blocks, tan if I just use greedy meshing, or other merging method. But I don't think it will be any major issue, in terms of ECS handling. Plus mesh culling. For which I remember, FtD didn't had it for long time.
Either way, if I stay on ECS side, I should be garbage free. However, I may use some colliders, which are GameObjects. Still resizing will be method to managing them. Here is potential GC if any.

Also, ECS handles colors / mesh swapping a bit diferently. Well for now is not ideal. But there was some example, implementing shaders. But since I got colliders (OOP side for now) and mesh (ECS side) separately, I can just move mesh outside the view, and disable collider. Hence, I wont need specifically making block transparent, using colors.

I think I may have some here, even running on separate thread. But I am looking for best way, to reduce it. Is not because of HTML5. But because of intercommunication of HTML5 with Unity side.
Yet, I will need to run more tests.

I think you shouldn't worry about graphics at the start. Problem with such approach is, if you focus on graphic now, in few years time, it may be looking already old. So you need refactor. Nothing unusual with that. Other than you may be doing work twice. Imagine 2 years time raytracing become so common and cheap (dreaming). Many techniques are simply be obsolete = wasted time.

But of course, you may have other good reasons for that.

I can imagine that. And wouldn't be surprised. But ECS seams to handle it well. It has a bit different way of profiling, specifically for systems and thread, along with default one of course. But true excessive will come up, when start testing and really playing.

Will you allow my blocky constructs hyper jump to your realm? Joking
How you finding topic, knowing there is multiple similar project running every year?

For me personally ECS side, is major game changer on many fronts.

Exact issues I was experiencing. When I could, I was renaming stuff in VS, or just in folder, from OS. But there is mix responses from people. Which is quite interesting. But for me, quite frustrating.

Yep, expecting sometime early 2019. Should be good timing for my project.

Yeah texturing. I may just use Minecraft one lol
I haven't been thinking too much about them. I just wait and see what will happen.
But yeah, if wanting 4k or even more, that become RAM hungry. I let you enjoy such fun.

Meshes should be fine, if you don't overkill them.
But there is always GPU instancing, which helps a lot.

I always look at this front thumbnail image of video and reminds me some cheap family film
But I did skimmed through it before. Sure, I need look more deep into it. I just need make sure, current SO <-> ECS proposed solutions are stable to use.

Second and third video I haven't seen yet. So this may be on my future agenda. I may watch when doing nothing, or in parts

Good sharing.

 

Your reasons are very valid, but I like to begin with what is scaring me the most, in this case the graphics.
Also, I'm pretty sure it will be the bottleneck for my game, and I want to know as soon as possible what I can do, how many ships I can handle, how many lasers, how many missiles, etc.

The next thing which is frightening me the most is the multiplayer feature.
I should be able to start on that one by the end of the year, but it doesn't frighten me as much as graphics, as it's code and I usually do quite well with code.

So yes, I will probably have a few things to modify/scrap in a few years (I am planning at least 4 years of work before reaching early access state, and 4 years after that to reach the end of the beta, knowing that I am working on the project for about 3-4 years now).
But in the end, I will have acquired some skills and some knowledge, so it's not really time wasted, it's time invested in order to find out a good way of dealing with graphics (or anything else).

Well, it has crossed my mind to have a block-construction system compatible with my game.
My game is not about building, but there is one part where it would be useful.
In fact my game is more a bridge simulation than a ship simulation. The players take the place of one crew out of a few hundred for the smallest ships, and a few thousands on the large ones.
One of the station the player can be is engineering, and in this one the player can interact with the internals of the ship, including different types of pipes and wiring.

When there are damages, or the need for more power/plasma, the player can re-route pipes/wires in the ship. That is based on 'blocks', as pipes and wires are in fact in an invisible 3D grid.
In order to create the ships internals with all the wires and pipes I will probably require an editor. And I think that a bock construction editor would work quite fine
In fact it's FtD which has given me that idea (I never played a block-game before).

So, in the end I may have to use a block construction system, but not all the physics which is around it, which should make it a lot simpler than what you're doing.
The only physics I intend to have will be in deep-space, so no ground to bother me, whereas you will have a lot of ground to manage, which is quite a lot harder than vacuum on the physics algorithms...

Purely by luck.
I wanted to post a bug I found in Unity (a nasty one, problem with allocating memory blocks in .NET 4.x) and went to the Unity forum, where I saw your name while looking for which thread to post in.
I was intrigued to find a fellow FtD modder here, so I looked a bit at your posts and was very interested/impressed in your work with Burst and the ECS, and here I am

I am pretty sure it will help me a lot also, but for now I have some trouble grasping how I can use it.
Mainly, how to proceed with the interaction of 2 different types of entities? For example, a missile hitting a ship.
ECS is very good at processing entities individually, but I donb't really see how to process 2 entities in the same system, it's like breaking the whole ECS concept.

Also, I'd like to have several times the same component for an entity (for example several turrets on a ship), but that also break the ECS principle.
I guess that for that I have to define the turrets as entities and add several references to other entities (turrets in this case) on the ship's entity.

I like to test in real conditions, so I wanted to have textures as close as the ones which will be in the final game in order to check the VRAM usage, the time it takes to load/render them, etc.
I do not want to have bad surprises with performances.

It also helps to find out nasty bugs which have to do with high numbers.
For example, I found out a bug in Unity with .NET 4.x which lead to a crash without any log when changing scene if you have allocated too many memory blocks of the same size, and that works very well with .NET 3.5.
I wouldn't have seen that one so early if I weren't using numbers (events in this case) close to what I except to have in the end in my game.

That's an old habit I have from my previous work, always have a dev-environment as close as the final product as you can.
For example, I also build and test a bit a standalone version about twice a week in order to ensure that it's behaving the same as in the Editor, and I find some differences about twice a month...

 

@Gladyon do you have own thread wit a project?
I did spy a little your forum participation, but I don't see you got one.
So in such case, are you planning to create one?

Multiplier is also something I would like to implement in some far future. But I remember challenges Nick had in FtD, with syncing etc. Hoping new upcoming multiplier system, will help address such issues, in more pleasant manner. But for now, I am not touching it. However, ensuring game system is compatible, with hooking int network system. This also adds extra layer of work. Including determinism.

Yep. I am pretty much on same wagon. Building, rebuilding ad scrapping prototypes is my day norm
I hope 2 years time, to get at least into some playable state. Depends on circumstances of course.

Gosh, this is me, spending too much time on forum, instead in coding
But I am glad, you came visit this thread, disusing range of dev aspects.

Processing multiple entities in same system is much easier now, than earlier in the year.
There is GetComponentDataFromEntity, which allows you to grab desired component, from any entity. This is apparently recommended way of doing it, at giving time.

Also, there is BufferArray, which acts like a list of components in entity.
It resizes at demand. But once is grown, its capacity does not shrink. Only length changes.
I didn't know that detail initially, and had created specifically my own length index. But this gives me one an extra advantage. Anyway, you can allocate for example n number of entities in such buffer.
Let say you have ship entity. This ship has BufferArray turretsEntities.
Once you got that store, now you simply iterate, or get relevant index, to grab turret entity(ies) and call GetComponentDataFromEntity, to get desired data. And that is all possible in one system.

For lasers, raycast is best, and you could do nice batch raycast if needed in ECS.
Missiles also can be triggered by raycast. Optionally by box, or sphere collider. Which may be bit more challenging at the moment.

Sure, looks like good habit. I think can be a bit anoying, when editor shows different result than built version. However, performance wise, built should always results in much improvement.

So far, I have't experienced, or discovered yet hidden issues in my case, with .NET 4.x. I hope it will stay this way.
Even tho, I do profiling on every critical change.


On side note, this week, and maybe next one, will be a bit slower in terms of dev. Got things to attend.

 

I haven't my own thread yet.
I work very slowly because I polish everything I do, and I also do not want to start a task if I haven't finished all the previous task.

To show you the level of small detail I am trying to reach, the progress bar I use at startup is auto-adaptative, it will monitor how much time it takes to load each part of the starting sequence, and it then will use the average of the last 5 launches to provide the most accurate progression possible even if you add mods or change hardware.

There are no 'simple' task when you polish. Everything takes time.
In the end, I hope to release a game with a high-level of Quality of Life even while it will be a alpha in early access.
As a player QoL is very important to me. So I try to provide it to the players.


So, to answer your question, I do intend to create a project thread, but I fear that it's too early now.
- First, I won't have anything to show before the end of this summer.
- Second, I'm pretty sure I won't be able to release the game before about 4 years (my planning indicates 3 years, but you know how plannings are...), and I fear that it may be a bad idea to start the thread 3.5 years before the release.

That said I am lurking in this part of the forum to check out how people do, when they create their thread, when they release their game, etc.
I guess that if creating the thread early is the standard I will probably create it a lot sooner that what I have in mind.

I prefer to implement it as soon as I can, because it may have a huge impact on the way data are stored/exchanged.
As soon as I have an interface allowing me to send/receive orders and to display the ships moving I'll implement networking, that way I will be able to define an architecture which works in multiplayer for all the players/AI actions.

That is the missing feature in my own ECS.
I know how I can go around it, but then it will be very hard for the algorithm to track dependencies for multi-threads.
I suppose that Unity found a way to be thread-safe when you access an entity of type B in a system processing entities of type A. I'll take a look at how they did it, it may give me a few ideas.

I'm planning to 'cheat' for collision tests. The player will be on the bridge, looking a his console, so it's not important if a laser hit a ship even if the graphic laser didn't.
There will be an external view for the show, but I hope that things will be confused enough (enough missiles, lasers flying all over at high speed) so that it won't matter if it's not totally precise.

 

I do understand what is on your mind. I initially also was a bit unsure, weather start it, or not, for such long term project. But if I don't, we would never had such discussion

I am also aiming, polishing each feature as possible, before going to next stage. I am aware, things can propagate badly through a project.

I don't think it is a bad idea. You always open door, for feedback, and knowledge mining.

Probably another good reason, to get into Unity ECS, rather than what I would call, reinventing wheel. Your approach would be viable few years back. But now, I would be concern in your place, that you may spent unnecessary time, developing ECS, rather your game.

Yeah, cheating is completely fine in game dev. as long do not break mechanics

 

You got a point there

Well, not exactly re-inventing the wheel. It's more like changing a bit the blueprint and creating my own wheel, especially since I've created my own ECS before Unity announced they were creating their own.
Anyway, I don't have to use the ECS before the end of the year, when I'll do the multiplayer.

And I haven't lost my time, it's been useful to have a very precise idea about how an ECS works, why it is efficient, and what are its flaws, at least some of them.
I also happened to improve a lot my skills in IL code generation in the use of generics, both already have been useful, and both will even more useful in the future.

 

I didn't meant that you lost time. Sure I bet you gain highly valuable skills and knowledge. And that is most high likely, it put you in far better understanding position, of low level programming, than myself. I am just barely scratching IL.

But what I try to say is, if you decide to use your system, which you need to learn as well, as you mentioned before, plus you would like to expand its features, as per Unity ECS, I think you would be better of with Unity ECS, than further pursuing your own ECS path dev.

But of course, you will decide what is most suitable for your project. I can only suggest things
Mind, Unity ECS team is more than one person, and they probably near full time on the job. Hence soon, we should see some further improvements, even still in dev too.

Oh btw., you may be interested at least with Unity Jobs system?
It is simpler than ECS, and allows you to set good foot stand toward ECS, if you desire in future.
Not sure, if it would benefit your own ECS tho, in terms of compatibility, since I don't know your system.
But what I could probably expect, it would help you structure your code, for either ECS route you decide.

 

That's the reason I am hesitating. My framework will never be able to compete with their own.

Well, what if I told you that I already have a similar system?
I've called it 'Tasks', but it's quite similar to their 'jobs', I even have the 'Parallel' execution.
Same as for the ECS, my version has a lot less boilerplate, but it's probably lacking a few features (I'm probably a lot closer to their 'jobs' than their ECS), I do not have the same level of thread-safety than them, and I do not have Burst.

But my version has something which is absolutely crucial to me, I can schedule the execution of a task at the exact time I want.
I need that in order to simulate the time it takes to transfer orders from one station to the other, or to make the AI act at a 'standard time' (not doing all its actions simultaneously), because in the end my AI is there to simulate a member of the crew.
Also, I think it will be very handy for networking, if I can sync the clocks I will be able to execute actions simultaneously on the computers just by delaying them a bit.


In the end, I'm pretty sure I will not use the Jobs system, or at least it will be only for a few very specific things.
But for the ECS, except if Unity's version has a 'flaw' that I really don't like, chances are good that I will use theirs. Perhaps keeping mine for a few specific parts such as the AI (as there's less boilerplate, it's easier to read and closer to reading English, which may be handy for the AI).

 

Ha, Interesting

Not sure if I mentioned already or not, but it is possible in Unity ECS, to to schedule job at desired time.
Is not most elegant as of yet, but is feasible. There are few discussions on the forum on that topic, if ever interested. Probably in year time, we will have something more appropriate / friendly.

But yes, if you run "default" job, then it runs as fast and as soon, as it can.
Timing is something which I also look at, along with orders, since these are critical, to ensure game core is ready, for networking integration (sometime in a dream ).

However, if you are after boiler plate reduction, this will be always triggering you
Well, I do sometimes think, "Gosh I need write it again ...". But in such case, I typically use template, or copy of other system, and just replace names +- relevant features.

I mean, if I would have to write many systems very often, that would be quite annoying for me. But most type I tend spend on algorithms in the system. Granted, still boiler plate often need be modified accordingly. Which is a bit of hassle.
But for me personally, is good trade off, for performance (threading / burst).

Btw, have you seen latest LA Unite presentations on networking, integrated on FPS samlple game?
Lots of interesting concepts there, which gives me full bunch of ideas already.

https://forum.unity.com/threads/networking-feedback-and-questions.555070/#post-4070665
https://www.youtube.com/user/Unity3D

 

I'll take a look.
For now I was looking at that asset:
https://assetstore.unity.com/packages/tools/network/mirror-129321

Again, because of the boilerplate I'd like to have something completely transparent. For example, if I add the attribute [Networking] to a property, then each time this property is modified it is sent to the server which then send the modified value to everybody, and only then it is really set to its new value.
Same thing for methods.

The idea is to be able to write my code without even thinking about multiplayer, just having to add a few attributes here and there.
I'm pretty sure that I will have to do it by hand.


And yes, I really do hate boilerplate
It slow down code-reading, and code-reading is how we find/fix bugs. So, it slow down the whole project.
And in addition, it's ugly. Maybe if they add some colors to it in Visual Studio I could accept boilerplate more easily


To be more serious, the main idea behind me creating new wheels for a lot of things is to simplify things.
Unity is forced to do things very open, with a lot of possibilities for configuration, because they have to be ready for any use we can make of their tools.
I like to keep things simple. So I usually add an interface between my main code and the tools I use.
For example, when I was using DirectX directly, I've created a 'DisplayImage(string ImageName, int X, int Y)' method using their own method which had about a dozen parameters.

That way, my code is easier to read.
The downside is that I don't have all the features, but honestly, who use all the features? And when something is missing, then I either add it, or then I use directly the interface of the tool if it's only to be done once or twice.

 

Yeah, I hear about mirror for a while. But haven't really chance put hands on it, as of yet. Too early for me. I mean my project is not in appropriate state yet

Sure I would love such mechanism in current Unity ECS.
Not sure however, what is future plan in terms of potential simplification.

While I generally agree, this is not my really concern of an issue, as I spend on such only small fraction of time. Even if I need update it. Rest of work as I mentioned before, stays inside jobs. These are really just acting like hooks.

And since ECS offers diffent ways to work systems and their jobs, I see, there may not be as easy way, to resolve the matter.

Yet not so long ago, IJobComponentDataFromEntity (if spelled right) has been introduced.
Which further simplifies system by little bit. You can just attach required components inline.

True that.

I was applying this method not so long ago, (providing I understood you correctly), which worked nicely. It kind of makes me think about ROS (Robotic Operation System) and its mechanics, which I was working while ago with.

Sure, that works perfectly for bespoke implementation, where no flexibility is required.

 

If you haven't already seen it, you should take a look at that:
https://assetstore.unity.com/packages/tools/animation/crowd-skinner-129524

Also, I have taken a very quick look at the 'imposter' methods (there are a few assets about that), and it may be interesting in some situation.
If you plan to have some buildings, trees, or other static things in your world it may help to reduce drastically the GPU load.

 

Thx, haven't seen this one in particular.

However, I am familiar generally with impostor approach.

Only thing is, none of current assets (at least I am not aware of any yet), works with ECS. Which makes not very useful for my project. Also, don't know, how challenging / feasible is converting impostor asset to ECS compatibility.

But for example, I can simulate already in ECS, 100k objects itself on the scene, with however, lowered frame rate, while no LOD, or culling. If adding LOD / culling, this is where great power comes from (See earlier discussed demo Mega City #35). And I am running this probably, on average potato PC (this days). Was high end 5 years ago.

Then when applying mesh merging, into bigger meshes, then that rendering count is dramatically reduced by default.

Saying that, it would be interesting seeing comparison, between native ECS solution, and OOP with asset impostors, or relevant. I am leaning toward ECS, that is going win by far.

 

From what I understand of the imposter, it's just displaying a 3D object on a billboard, which is in fact a 3D square.
So I don't see why it would be different to display 3D objects using an ECS from displaying billboards using an ECS.


I'd like to share that video with you, it explains very precisely how ScriptableObjects work, and it also provides some very concrete examples of how they can be used in a game:

The first 20 minutes are describing what is a ScriptableObject internally in Unity, and the last 10 minutes provide a few game use-case or them.
I do realize that they probably won't fit in an ECS, but you will have to configure your data and behaviours, and they may help for everything which is not purely performance intensive.