DOTSNET - High Performance Unity ECS Networking from the creator of Mirror!

This looks really great. I have one question.
I am using unity for rendering on server side. So basically I am streaming frames from my server. Does DOTSNET supports connection from client outside unity? Say a webpage running with kcp?

 

DOTSNET supports different transports.
For a web browser you need websockets, kcp is UDP which doesn't work for web browsers.

 

Thanks! So in this case if I want to use it with web browser I will need a middleware server to transfer from UDP/TCP to websocket? Sorry I am not vary familiar with different transports.

 

You just need a websocket transport.
We have one for Mirror. It's relatively easy to convert that to DOTSNET if you want to try.
Websocket transport is kinda slow though. Not really fast enough to keep up with DOTSNET, that's why I didn't add it yet.

 

That will be great ! I can try it. The reason we want to use Dots is because we can increase our concurrency on server and save some cost. Latency is not our first priority so websocket should be acceptable. Thanks

 

Websocket transport is slow, as in can't handle that much data.
It's a starting point, but for DOTSNET we'll need a way better optimized one later.

 

Hey guys, just a heads up.
There seems to be a 'no available transport found' bug in builds ever since upgrading to Unity 2020.3 LTS.
Something is different in the bootstrap process, not sure why yet.

Testing 2020.1 again today and will downgrade if it's stable.

 

V1.28 pending review!
Important: downgrade to Unity 2020.1.16 again for now!

 

Quick progress update:
- NetworkComponents almost finished
- Documentation moved to Gitbook

DOTSNET is still 50% off during the Spring Sale btw.

 

V1.29 pending review: NetworkComponents

 

Is my project feasible with DOTSNET?

500CCU, around 40k entities(enemies, nature objects, projectiles, potentially moving and colliding with each other), RPG style mmo(items, inventory, health, player stats, special abilities(area effects), anti cheat, trading, chat and loot).

Thanks in advance.

 

Sounds like an MMO, I have the same goal. That's what I made DOTSNET for.
40k entities is no problem whatsoever. Millions of them are possible.
500 CCU test is planned for some time this year. So it's not proven yet, but ECS is surely fast enough.

Performance isn't a limitation in ECS world. It's only ease of use right now.
For example, NetworkComponents were added just a few days ago.
NetworkTime, Rpcs, CMDs etc. are still missing from DOTSNET. Working on it

 

V1.30 pending review!
kcp can now handle 10k+ entities: https://gyazo.com/7c7c4a647d3c03c0c2cb913e173980a4

 

Hi I have a question regarding scalability, I see the interest manager but are there plans for you to make the server scalable in a way it divides the world up into pieces so that a few machines running dotsnet server can share the load?

 

DOTS is really fast. Ideally that would not be necessary.
Splitting your world into multiple machines comes with a complexity & maintenance overhead, which I would personally rather avoid as an indie dev.

 

Yes, it isn't a real concern for indie devs, but it still would be nice to have the option to increase compute capabilities of the server by horizontal scaling (i.e. throwing more machines at the problem instead of better hardware for a single server) in case one trasitions from indie to a larger project. Without this option, one would have to rewrite the whole project to accomodate higher CCU.

I envision something like a distributed ECS, where data is automatically distributed and replicated to the server nodes where it is manipulated and read respectively, so that one does not have to take care of load balancing manually (for example via separating the world into regions for each server node). Either that or a distributed actor model like Akka would also be a good fit.

 

Hi.

Our team is using DOTSNET for several months, and it looks like we'll have to add the following features sometimes in near future:
- support DynamicBuffer synchronization. We could work around this, but it feels that these workarounds are too messy. Right now we use dynamic buffers for various buffs/physical effects/etc applied to Entity. We can have multiple effects of the same type (but with different parameters) applied to same Entity, so we cant use IComponentData directly in this case
- synchronization for slowly-changing components when component changes detected. For example, we could have several hundreds of characters, each of them having huge component CharacterMovementSettingsComponent with for example 50 floats inside. Lets say this component can change but we dont expect these changes to occur very often - once every 30 seconds for example. Sending CharacterMovementSettingsComponent every network tick would be waste of traffic. Keeping in mind that every time I change this component in some new system, I have to send network message for this component is not very reliable from developvent point of view. Right now we just ignore traffic waste.
- synchronizatoin of slowly changing buffers - just combination of previous 2 cases

So the questions are:
- if there been any plans for such features support?
- or if somebody can share good practices working around described cases using existing DOTSNET features?

 

Yes.
DynamicBuffer sync sounds useful.
Reducing bandwidth for slowly changing objects as well.

Currently spending 100% of my time on Mirror to get some major features done, which will also be backported to DOTSNET.

It's a bit difficult to work on two networking libraries at the same time, especially with DOTS future being so uncertain.
But we'll get there

 

Progress Update
Still no DOTS status update from Unity.
Still banned from asking questions.
DOTS forum is nearly dead with only a handful of posts every day.

DOTSNET development is on hold until Unity shows believable commitment to DOTS again.
Bug fixes only for now.

In the meantime, I'll focus on Mirror to finish the roadmap this year.
Most of the techniques can be backported to DOTSNET once we know more.

 

V1.31 pending review:

 

Thank you!

Your talents and perseverance greatly appreciated.

"What we see now is like a dim image in a mirror; ..." ~ Ghost in the Shell

 

Eh? This is a very immature response.

Unity3D are saying (in a really fumbled way admittedly) they have essentially hit a fork in the road with their development of DOTS and its likely that fork results in breaking change between old and new iterations. Therefore anyone in the old stay on a stable release until they can figure out a way to bring that work forward. Should one ever eventuate.

However those in the new iterations, play on.

 

This is your interpretation and speculatively positive.

They've actually said that DOTS won't be into newer versions until at least the end of the year. Given Unity rarely hits absolute targets, when they use the words "at least until... " it might be safer and more accurate to just add another year, right when you read "at least" and "year" in the same sentence.

Maybe add two years, if SRP and URP being "production ready" are anything to go by.

And they have a far higher incidence of just dropping things than ever fixing and shipping well performing things, and this is leaking out into many aspects of things added in the last few years.

So if you're going to interpret things, it's probably best to consider all inputs, not just an otherwise unsupported forum post from someone somewhat down from the top of the decision making and corporate responsibility trees.

The continued silence is one of the inputs to consider, especially when they claim to be reading along and intending to answer questions, but don't.

 

I take what was said in that quote as emotional based.

They aren't being specific because likely they are lining up launch dates, conference announcements etc. When you manage a product you say things like this to provide scope or horizon on whats coming soon, but being vague on specifics gives you room to work out strategy on how you talk about it.

I used this tactic a lot when managing Microsoft .NET, I'd give seasons or quarters as a likely landing date for an announcement but wouldn't be specific because i still have conferences too book and work out with marketing/pr what our story is going to look like.

Has nothing at times to do with "is the technical effort required likely to land"

Right, so if i go File-New-Project right now you're saying its a waste of time to use anything DOTS to ship a product with as-is tech stream - or could it be folks are busy chasing the "new" tech stream updates because they are new. What are the holding points or blockers from where they are now to where they can be? thats the part that is getting lost in this discussion. What changed so dramatically that prev ver compared to current versions i preventing you from shipping.

Not really. I'm still not seeing the problem beyond emotional tantrums. Silence is they have nothing new to add, why linger in a forum charged with a lot jaded responses to just fan the flames more. The first post explained as clear as day without saying "breaking changes sorry".

Anything new to add? what are the breaking changes and when can you fix them. Answer "not sure, and if were sure i'll hold off until <event here> to discuss in a way that i can control the narrative without more emotional tantrum"

 

You can build as many strawmen as you like, the facts are clear.

Unity has a shipping problem with the "performance by default" systems and performance problems in existing systems, which are atrophying rather than having any efforts put into improving them to hold us (the users) over.

Further more, there's an obsolescence issue with the older/builtin systems, that's getting worse with every passing day, as older systems are not being updated to comply with new gaming platform OS updates, nor are they being updated to take advantage of new hardware.

So there's a pincer event occurring, in slow motion, over the course of years (the time it takes to finish a production).

Rely on old/existing Unity systems while the world moves on (new gaming platform OS updates, new gaming hardware releases) and you'll eventually have a lesser market and not be able to access the latest and greatest, the markets where game players are most likely to be most happy to fork out cash for your endeavours etc.

Aspire to use the newer stuff that's (apparently) being updated and going to be made compatible and compliant with new OS and hardware changes and you risk Unity not getting to where they claim they'll be.

Across several years, Unity have made significant claims, and strong suggestions about which versions we should use. So far as I'm aware, none of their claims have panned out well.

 

To what degree is this integrable with uMMORPG? Or is that running on Mirror?

 

Not at all.
uMMORPG uses Mirror & MonoBehaviour/GameObjects.
DOTSNET is pure ECS.

 

does it support fizzyfacepunch transport?

 

You can add your own transports.
The ones from mirror can be converted easily. It's just two components instead of one.

 

Is it possible to use DOTSNET in hybrid projects where some parts are synced as mono and other as ECS?

 

I try to use as pure ECS as possible for DOTSNET.
Since hybrid is going away eventually, don't want to do the work & convert projects twice

 

V1.34 pending review. Smaller fixes & improvements.

 

V1.35 should be on the asset store in a few minutes.
lots of minor improvements in preparation for delta snapshots.

 

Impressive work! I wonder if it would make a project mine possible?

I would like to have 500-5000 clients on Android and iOS devices to send position data to the server. The server just visualizes the player positions. Would the platforms or client count be a problem?

 

Thanks.
Working on the core architecture for now.
CCU test coming soon, until then I don't want to give any promises.

But obviously if we can do 300-500 CCU in Mirror, this should handle more

 

Thanks for the quick reply. Do you think having the client on Android would be a problem? I read somewhere that it isn't handling ECS that well.

 

Haven't tried on android.
Might be better to ask in the DOTS forum

 

will do

 

Hey. Nice wór so far.
I got question, but if been answered already, please point me to the page of the thread.

How DOTSNET, would handle an RTS game, with multiplier and possibly some lockstep features?
How does solution handles the lag?
Imagine intercontinental Gameplay, with some latency.
Also does the solution covers syncing/desyncing?
I.e. clients run simulation, as well as host does, but host send authoritative messages to players.
How many players can be handled, with 10k units?
Does the solution implements any predictions?
Is it UDP, TCP, or otherwise?
What solution do you use for compression data, before sending over the cable?
How big and how often packages are sent, between host and client?

 

Thanks.
- DOTSNET does not use lockstep. Check out the age of empires netcode article, lockstep is a lot of work because you need to get everything deterministic. You are essentially trading productivity for bandwidth. Without lockstep this would be way easier, but more bandwidth.
- DOTSNET doesn't need to worry about desync. It's meant as server authoritative simulation for large virtual worlds.
- How many players: will run a CCU test in the next few months when the core features are finished. We can do 300-400 with Mirror & GameObjects already. This should be more. I don't want to give estimates before the actual CCU test though
- No prediction. Still working on delta snapshots and later eventual consistency first. I don't want to make FPS games so I am not that interested in prediction for now.
- DOTSNET supports different transports. You can use Telepathy (TCP) or KCP (reliable UDP) or any other
- Working on Data Compression this week. It will use delta compression. Almost finished.
- How often / big depends on your game.

 

V1.36 should be out in an hour or so.
For delta compression, make sure that all Serialize() functions write the same amount of data.
For example, use FixedList<T> for inventories and always write the full thing.

 

V1.37 with quick bug fix. If you are on V1.36 already, then please update to V1.37 immediately.
Should be on the Asset Store in an hour or so.

 

V1.38 should be on the store today.
Fixed several bugs.

 

bug fix release!

 

Actually, any reason why you screenshoting revisions, rather just pasting in text? Would be much easier to search for keywords. Otherwise is impossible.

 

Is there currently a way to support multiple client worlds? Would be great to be able to test multiple clients at once in the same unity editor instance.

 

Not at the moment, sorry.
Maybe in the future, but other stuff has higher priority atm.

 

Bought this asset a year ago to support the author. Kudos for such a great work!

Skimmed through patch notes and it seems to still be not quite 'production ready', is it?

Specifically:
- full burst
- support for complex structs with DynamicBuffers, shared components and blob references
- generic structs support
- fast reconnect with only delta sync from last known state
- time sync

 

Thanks. Had some decent progress this year. For example, server broadcast is now fully burst compiled.
Plenty of work to do still. And waiting for DOTS 1.0 to see what’s different.