A new programming language is needed

I wonder if the people at Unity has ever considered this possibility, but I think is time to put it on the table. Let's face it C# is and always will an OOP language, so we are basically using C# to do something it wasn't designed for, something that in many ways goes against it most basic principals. I don't know if there's anyone out there sharing the same feeling, but I feel like in the old days of C when OOP was a new concept and everybody was using C to implement OOP functionality, it was hard because C wasn't designed for that, it wasn't until C++ came to light that people really started to see OOP in action, and it wasn't until C# that it was properly implemented. I think we are now a similar situation and the best way to exploit the full potential of ECS is through a new programming language specifically designed for the task.

 

There was already discussion somwhere regarding ECS vs C++. Not sure if that wasn't even on some conference talks.
If I remember correctly (anyone to confirm?), there was and indication that Unity ECS can work as fast, or even faster than equivalent C++.
C# has been designed for faster implementation than C++ can, hence here it is its massive benefits, as you save a lot of time.

ECS also aims to be compatible with shaders language. At least in significant part.

Why developing something new when you got resources and tools already on the table, when no guarantee of performance, or even feasibility.

So to the above, there is no reason going backward.

What you propose as "new language"?

 

Does Unreal have similar to ECS ?

 

That's exactly my point!

I'm not talking about going backward but the exact opposite!

Simplification!, readabilty! usability!

imagine that instead of this:

Code (CSharp):

1. [BurstCompile]
2.    public class SysResetPosition : JobComponentSystem {
3.  
4.       public struct ResetJob : IJobProcessComponentData<WorldPosition,TrackPosition,ResetPositionTag> {
5.  
6.          [ReadOnly]
7.          public ComponentDataArray<WayPoint> points;
8.  
9.          public void Execute(ref WorldPosition world,ref TrackPosition track,ref ResetPositionTag tag) {
10.             // your code here  
11.          }
12.  
13.       }
14.  
15.       protected override JobHandle OnUpdate(JobHandle inputDeps) {
16.        
17.          ResetJob job = new ResetJob();
18.          return job.Schedule(this,inputDeps);
19.       }
20.  
21.       protected override void OnCreateManager(int capacity) {
22.          // your code here
23.       }
24.    }

you could write this:

Code (CSharp):

1. System SysResetPosition(WorldPosition world,TrackPosition track,ResetTag tag) {
2.  
3.    Import WayPoint points
4.  
5.   OnUpdate{
6.         // your code here
7.   }
8.  
9.    OnCreateManager{
10.       // your code here
11.     }
12.  }

which one looks cleaner, simpler and more readable?

 

I never used Unreal Engine, but according to quick reading I did, UE 4 doesn't have have ECS integration. Is heavily OOP oriented. They however have something call Actors Components but still based on OOP, from what I gather.

There are however some works, to make resemblance of ECS in UE. Apparently inspired by Unity ECS. But still worked by community, rather than official UE and based on C++ OOP architecture. So won't be as fast as Unity ECS.

From EU4 forum

Implementing ECS architecture in UE4. Giant space battle.
https://forums.unrealengine.com/dev...ng-ecs-architecture-in-ue4-giant-space-battle

Spoiler

Spoiler

And other quick soruce

Entity-Component-System (ECS) for UE4
https://forums.unrealengine.com/com...s/1423789-entity-component-system-ecs-for-ue4

 

Maybe Unity team has some simplification on mind for later. For now, feasibility and reliability is more important, before adding another boxing and wrappers (if I got terms right). This is not really new language tho. Mind, ECS for Unity is at least since Unity 5.4 on drawing boards. And I don't know, how much earlier before then. So obviously, is not something simple, what can be implemented on existing engine within a year, or so.

We know Unity team wants add at some point more to inspector, and eventually be more user friendly, as current GameObjects in editor. So we need give devs. a time, to implement things step by step.

 

It does not need a new language.

1. That was tried long ago.
   • Boo wound up being not ideal in many scenarios. And ultimately dropped.
2. Usability becomes vastly more difficult for entry.
   • Currently, any dev who has used a C like language or Java can easily wrap their heads around the usage of C#. I wouldn't just blanketly say "C# Is an OOP language". That just in itself doesn't make sense. OOP is not language dependent nor vice versa, but rather a standard way of usage. You can use C# very efficiently for Data driven design.
3. Readability is really an "eye of the beholder" argument. For instance, I really appreciate the current IJobProcess layout for an entire system. It is very clear of what it is doing from the get go.

Could a new language be neat? Sure! Is it needed? I would say no. It would just become confusing and potentially not any more performant as the end result will be.

From what I remember, the entire engine is shifting over to more C# rather than the current 50/50 split C# /C++ which will make developers' lives easier in all honesty.

 

Agree, I just pointing out to the fact that we're doing a lot of extra typing just because we're using a language that's not designed for the job. There's a real need for a new language, and someone has to take it into account for the future, whether short or long therm, whether inside Unity or outside.

 

Yes please.
It doesn't need a new language, C# is fine, it just needs some unity magic to wrap all that crazy boilerplate into something simple.
JoAnte said that's the plan after all is implemented. How far will they go? Hard to say. Let's hope the few elitists in the ECS team won't oppose simplification.

 

What about boilerplate code generating tools. Have you ever considered that? Take for example ReSharper. Yes it costs money and some time investment to set up the templates but once you‘re all set you‘ll be flying through creating ECS code rather quickly.

I can see the need for some sort of tool or meta-language that can simplify ECS coding constructs, ensure correctness and then convert the results preprocessor-style into actual C#. Perhaps a subset of C# with a more powerful preprocessor would be a good compromise.

 

The whole idea looks like a macro that maps 1-1 to a specific C# phrases and not an OOP problem e.g.

- Don't want to type

protected override

, it is always there in Unity ECS. (Real problem : abstract pattern in C# is verbose, not that because C# is OOP based. We just want a method with exact same name of superclass to implies protected override + same argument automatically. But I think that would cause problem for code analyzers, they now have to look at superclass to know what's available to use in the scope)

- 50%+ of fields in the job is

[ReadOnly] public ComponentDataArray<PascalCase> camelCaseWithS;

, why can't I type something shorter than this. (Real problem : it happens to be the common pattern of Unity ECS, which has to be implemented on top of more general purpose C#)

- IJobProcessComponentData pattern requires repeating the component name on the generic and again on the Execute. Because that's how C# can take a type as an argument plus a custom logic at the same time. I agree that C# struct-as-a-job requires more layer of reasoning "why" I must do this than most Unity ECS API. (because we have to ensure self contain-ness) but at the same time seems like the only way if we are sticking on C#

So we are essentially making a new language that hard wire to a subset of C#?
Also what about inteoperability with other C# code in MonoBehaviour and Intellisense?
What if we want a private field once in a while in the job? (btw I have quite a lot now)

I am still with retaining the flexibility of C# + code snippets to fight with verbosity. The pain point is that code snippet is one-way. When editing them later, preprocessors will have an edge over code snippets. (e.g. I made a Job with 6 [ReadOnly] public, then I realized I don't need one of them. I now have to remove 2 places, that is inside the job and at the job scheduling where I copy the values to the job.) But when the code is working structurally (arguments number are correct, but logic still bugged), I find C# easy enough to read, just not easy enough to type & mass-edit where Ominisharp failed to do the job.

Also VSCode is able to auto complete the abstract methods, but they pop up a bit slow so I am also using a code snippet for that. (e.g. oncredes -> OnCreateManager OnDestroyManager with empty space to type code)

 

Rust is really good candidate for new data-oriented language. It looks like that it encourages ecs-like design, also it is enough low level for game performance optimization, and like golang designed for easy and reliable multithreading stuff.

But anyway it is very extreme experiment for Unity, especially remembering not so good experience with another languages in the past, and Rust is completely outside of .Net/Mono ecosystem.

 

To clarify:

1) It is an undeniable fact that implementing ECS on C# requieres a lot of extra typing.

2) I agree C# is ok and I'm not talking about replacing it but about building on the top of it a new set of instructions that would make the job of writing ECS code easier and faster.

3) New doesn't mean incompatible, for example, you can input any C code to a C++ compiler and blindly trust you'll get exactly the same results you would if you input the same code to a native C compiler. A similar approach can be taken in this case and no one would need to rewrite what they already have.

4) What I'm proposing could be implemented in a first stage as a "pre-processor" and then evolve to full blown compiler.

That's what I'm talking about, there's need to replace what you already have.

Paid solutions are ok for specific problems, but if Unity is about democratization, then you shouldn't have to pay for a solution to a problem that affects the whole community.

The problems is not the OOP itself, but the complicated grammar required to implement a different paradigm. The current grammar forces you to "re-word" what you already have expressed in a previous sentence, it forces you to be explicit about things that could be easily implied, some errors that currently can only be catched a run-time could be catched at compile time. So the real problems a syntactical and gramatical, that's why I'm proposing a new language.

 

A new language is totally unnecessary. The issue with writing a lot of boilerplate code can be handled by tightening up the ECS/Burst/Jobs integration. It just seems cumbersome today because it's in it's infancy and there is a lot up in there air about how it actually works and should be integrated.

 

@zulfajuniadi made a code gen https://github.com/zulfajuniadi/unity-ecs-system-debugger
It helps during initial coding but read through is the same bowl of spaghetti.

 

You could write an extension for Unity that translates some custom ECS language to C# like CoffeScript => JavaScript.
But to make it useful you would need IDE that supports this language.

 

The real problem lies only on the grammar, all the rest is fine. So you do need a new language but not a whole full language, just one that makes easier to write the ECS definitions, all the rest can be done painlessly on C#. It's a well know fact that VS supports multiple languages, I wonder if it is possible to write an extension that defines a costume language that could then compile to the same intermediate language C# does and link and interface seamlessly to C# code.

 

I support this. Considering we are using a subset of C# as HPC#, why not make a more concise language out of it? Then we wouldn't need any "magic" syntax.

 

What you are thinking of is called a domain specific language, and in almost all cases it's simply a limited set of expressions using the existing language. But the domain in question here is not different enough to warrant that. All it needs is just more refinement of the abstractions. Don't over complicate it.

 

We don't need to be reinventing the wheel.

• C# has practically-infinite resources behind it including MS
• All IDE's support it
• Current C# is not going away. Projects can use both ECS and EC*
• Being restricted to a subset of language features is not new.

So being restricted to a subset of language features does not mean things will be improved if everyone has to learn a new language.

I could never advocate a new language for Unity. Seems the complaint is "ECS is hard with lots of boilerplate". Well to fix that, simply keep posting examples in new threads within the ECS forum. Unity needs these examples to remove the pain points. ECS is still early in development.

*EC is Entity Component, the current Monobehaviour way.

 

Is not really new language, but feels you refereeing to some form of higher level wrapper.
On other hand, there is a reason why Unity JS has been depreciated, and other languages were removed, or are not supported form the start. Too much unnecessary overhead for Unity team, while now, Unity can really focus on performance benefits.

Personally I was removing / replacing Linqu, if existed in any of downloaded assets.

 

Yeah it's just really, really slow and spews out garbage like a bad illegal download of WALL-E. It is useful for people but not so hot for performance.

 

OMG, so you want to hunt for a sparrow, so you start to shoot it with a regiment of howitzers...

Solution 1: wait till Unity sort this out

Solution 2: set up your macros in your IDE, you can do that so you just push 2-3 keys and you get the entire template you want...

Even if it wouldn't be fixed it wouldn't be okay to start a new language from scratch to just save a couple of keystrokes. That's just crazy.

 

I agree with @georgeq ECS is in essence functional programming with good multi-threading.

A lot of the boilerplate code is pushing this paradigm onto an OOP language structure.

What if Unity adopted F# it appears to be a more apt language than C# for data-driven development.

How to program pong in F# https://medium.com/@joshmiles/pong-in-f-940ac1c9c24d
github F# pong repo: https://github.com/JoshuaMiles/SimplePong

It even supports the Actor programming model which sounds ideal for games https://en.wikipedia.org/wiki/Actor_model

It's also part of the MS .Net language 'stable' (?) and therefore could be easily added to the build process for Unity (in theory).

 

Yeah just... I'm happy with C#. I don't want to learn F# or anything else. And I don't need to. ECS is fine. The boilerplate will be handled by Unity. Just give them examples of pain points instead of theory.

 

Sounds like I good idea, but I'm not sure about that. The syntax is completely different to C# and most people would feel sacred just because of that. There's a vast code base written in C# which will also generate a lot of rejection. What's need is a language which's syntax is fully compatible with C#, so people can embrace it easily. C# is indeed a great language, but if ECS becomes a major trend then it needs to evolve to something that makes programmer's lives easier, and that implies the ability to compile existing code. In the old days people gladly embraced C++ because it could compile existing C code without any changes, and the only way people will accept a new language will be if it can compile existing C# code without any changes.

 

Yes, this is the key. But you're talking about a new language instead of evolving the "problematic" feature.
And besides that, you're talking about a feature which is in preview stage. Not even beta. Yes, Joachim and the team said multiple times that they're aware of the problem and they will address it when the most important features are in place and everyone can see and try what it is about and how it is done.


And you're talking about developing a new language too easy, I suspect you have no idea how much work is that, how much time and what kind of connotations you can have, especially in an environment like Unity when the target platforms are abundant and diverse.

Again: IDE macro is your friend. 100% compatible and 99% compressed.

It's like: "I don't like the tiles in the kitchen, let's build another house".

 

Here is a good dive into the strengths and weaknesses of C# as a functional programming language https://hackernoon.com/is-c-7-starting-to-look-like-a-functional-language-d4326b427aaa

For instance, all of those [ReadOnly] tags are fighting the fact that everything in C# is mutable which is bad for functional programming.

 

Fixing the problems you perceive the language to have can't be done while maintaining full compatibility. Attempting to do that will just leave you with a mess very similar to the one with C and C++. Under the hood many compilers used hacks to implement C++.

https://en.wikipedia.org/wiki/Name_mangling#C++

 

Why not? Is a gramatical problem not a functional problem.

This won't be case, the change from C to C++ was both gramatical and functional, that's why it became a mess. Here I'm talking about gramatical additions only (not changes), the compiler could handle those additions without affecting functionality at all. Any existing compiler could be modified to handle the new gramar while keeping the rest intact.

 

I'm not proposing a "whole" new language, it just requieres to add some minor features to current C#

Yes, I do have an idea, I have some experience writing compilers, but again, I not talking about building whole new thing again, I just proposing a few of additions (not changes, just additions) that could be integrated without too much hassle. Basically adding a few new keywords, for example adding "component" and "system" to the current set of data definitions, so in addition to struct, class and interface you could declare component and system, maybe a few more keywords need to be added, but I confident the number of additional keywords won't be greater that 10 or 15 at the most.

"New" does not mean "completely different", go outside an find any car model 2017 you like, then compare it with the same brand but model 2018 and tell me how many differences you find.

 

C# 8's Records look like an ideal fit for ECS https://www.dotnetcurry.com/csharp/1440/csharp-8-new-features

Code (CSharp):

1.  
2. public class Sword(int Damage, int Durability);
3.  

This single line would result in a fully functional class:

Code (CSharp):

1.  
2. public class Sword : IEquatable<Sword>
3. {
4.     public int Damage { get; }
5.     public int Durability { get; }
6.     public Sword(int Damage, int Durability)
7.     {
8.         this.Damage = Damage;
9.         this.Durability = Durability;
10.     }
11.     public bool Equals(Sword other)
12.     {
13.         return Equals(Damage, other.Damage) && Equals(Durability, other.Durability);
14.     }
15.     public override bool Equals(object other)
16.     {
17.         return (other as Sword)?.Equals(this) == true;
18.     }
19.     public override int GetHashCode()
20.     {
21.         return (Damage.GetHashCode() * 17 + Durability.GetHashCode());
22.     }
23.     public void Deconstruct(out int Damage, out int Durability)
24.     {
25.         Damage = this.Damage;
26.         Durability = this.Durability;
27.     }
28.     public Sword With(int Damage = this.Damage, int Durability = this.Durability) =>
29.         new Sword(Damage, Durability);
30. }
31.  

 

Actually, if you look at F#'s Agent based system...

https://fsharpforfunandprofit.com/posts/concurrency-actor-model/

Code (CSharp):

1. let printerAgent = MailboxProcessor.Start(fun inbox->
2.  
3.     // the message processing function
4.     let rec messageLoop() = async{
5.      
6.         // read a message
7.         let! msg = inbox.Receive()
8.      
9.         // process a message
10.         printfn "message is: %s" msg
11.  
12.         // loop to top
13.         return! messageLoop()
14.         }
15.  
16.     // start the loop
17.     messageLoop()
18.     )

F# you have a MailboxProcessor that responds to Messages.
ECS you have a System that works on DataComponents.

Note F# agent tested and handled about 30 million messages in a second https://theburningmonk.com/2012/03/f-how-many-messages-can-you-post-to-a-f-agent-in-one-second/

Could F#'s agent-based system be used to build ECS systems in Unity, only with a fraction of the boilerplate code needed?

 

C# is capable of a heck of a lot of syntax sugar. Code weaving, control over compiler etc are all things Unity has though. It's really a case of trying to design a new car before the first one got past the prototype stage... If it was a case of chicken and egg, it would give birth to a mutant instead.

 

C# is designed to be OOP, ECS is really functional programming. C# is slowly gaining functional programming features but it's legacy OOP syntax and design look as though they are just getting in the way of a clean easy to write ECS syntax, IMHO.

Check out F# have you seen it's so concise and powerful, I'll be amazed if Unity has not had a play with it as an alternative to C# especially for functional data-driven programming system like ECS.

https://fsharpforfunandprofit.com/posts/why-use-fsharp-intro/

 

C# OOP is 100% optional and not required. I do not have any inheritance in my own code. I have not had need for it. Unity's own stuff uses it.

I always preferred a flat design where if I wanted to add functionality to something, I simply create an entirely separate class or structure, and use that. My class can simply add these building blocks as needed. Inheritance, and OOP in general is not how I like to work.

So I can comfortably assure you that OOP isn't a requirement of C#.

https://fsharpforfunandprofit.com/posts/fvsc-sum-of-squares/
C# is more readable to anyone coming from any other language (basic, C, C++, Javascript) and therefore a better choice IMHO.

Anyone remember fiddling about with AMAL for AMOS?

 

Asynchronous F# gives up to ~x2.5 speedups compared to C# TPL in our extreme concurrent computation tests with/without stackalloc (to utilize CPU cache if possible). The reason why asynchronous F# faster is that it has lower per-operation overhead internally. To achieve the same speed with TPL in .NET Core, we made the following things: minimized overhead under the hood of TPL, implemented an optimized tasks scheduler, in-depth IL analysis/adjustments. So, we achieved what we aim for, without migration to another programming language and ecosystem. If you aim for high-performance in C#, programming paradigm and its design doesn't matter since it's a high-level programming language. The things that matter the most is an architecture, concurrent computation model, low-level implementation and only then well-optimized code that sits on top/works in harmony with stuff.

If you care about productivity and you want to boost workflow with a particular technology, then you have to write or use existing tools to close the specific gaps.

C# is a programming language of great possibilities. Everything is in your hands.

 

Not quite, ECS is a data oriented programming pattern. Functional programming and data oriented programming have some similarities, the main one being is that they both decouple data and behavior (unlike OOP, see chart below). But they are different in that functional programming has greater emphasis on the behavior (functions are first class citizens). Data oriented programming on the other hand has greater emphasis on the data, and behavior is less important. And because of that, it can leverage the power of data locality which gives us the main performance gains for using ECS. Not something that's possible with functional programming.

To drive the point home a little more: the very popular front end library, React.js, is a functional programming library that uses the component pattern. Components in React.js are functions. Where as components in ECS are data. And of course, components in Unity's current OOP architecture (MonoBehavior) are objects.

Programming at it's simplest boils down to data and behavior, so for fun here is how the 3 paradigms differ from one another. (bold is what the paradigm has greater emphasis on)

OOP: Data + Behavior
FP : Data | Behavior
DOP: Data | Behavior

I find it very interesting how DOP is the complete opposite of OOP.

 

The thing is Unity could add F# to the mix as another option, then if the ECS system is easier to use via F# it's up to the programmers to choose the one they like best.

F# is a .net language just like C# and Visual Basic both of which could be added to languages supported by Unity as they both compile via .net and Unity are adopting the Rosyln compiler so that might even allow them to tailor C# or F# to be really easy to use with Unity.

 

Locked because it's not relevant to the existing ECS implementation. Unity aren't adding F# support. Beta is about discussing what Unity is providing, not new features.