Search Unity

The day that unity become thread safe, a man's dream

Discussion in 'General Discussion' started by Jiraiyah, Feb 12, 2015.

  1. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    Did you ever tried to use threads in your code for unity? I have done, these days you can't think of Advanced AI systems without thinking about putting parts of it on threads, why? think about it, an agent needs to do path finding, and at the same time decision making, well, you would want decision making on higher priority maybe, but that can take half a second for each agent, so you would prefer to handle path finding first and then start decision making, now you may say that we can use coroutines but the thing is that it is still on the main thread, meaning that much of your computer's resources may not being used at all.
    another example, I will speak about a pet project of mine, currently developing an AI sand box, started with node based grid system, restoring every type of data u need for a game in serialization is out of question, it will cause like gigabytes of data on hard drive (more can be found on my weblog : unityai.net). so you would like to keep the minimum data serialized with scene levels and the rest being calculated on the fly, one of the most important thing is line of sight and line of fire for each node, that is if you want to be able to do tactical decision making. Now, both of them are using line casts, and you would never want to do them on the fly time and time again when you are doing decision making, the sane approach would be to have a data lookup table so that when you need to test a node, you could simply look inside the table and get the result. how this effect us? think about it, you are starting a game, loading a level and as soon as the level is being loaded you would like to have your data there or at least starting the collection of data. You can still use coroutines but as i said it's waist of resources on machine, so you would like to start a thread that is looping through every node out there and do the ray casts for you, and that is only LOS and LOF, how about finding covers? another sets of line casts, at least 8 on mid height for crouching covers and another 8 for full standing covers, if we have 4000 nodes (pretty low for these day's games) that would mean 18 line casts per node, now we still would be able to do them on the fly for when an ai need to make decision and mark the node being processed but that would cause possible freeze on the agent till it is being done. So you would want to have to do those line casts on a thread that is adding data to a static class and later on agents can read these data but wait.....
    Line cast is part of unity's engine and api, and .....
    yah :( you can;t use them on threads GAH

    I think the day that unity announce that these type of things can be done on threads i will have a heart attack !
     
    0tacun likes this.
  2. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    2,830
    Where's NPSF3000/skull xray dude when you need him to harp on about such?
     
    shaderop and Aurore like this.
  3. JamesLeeNZ

    JamesLeeNZ

    Joined:
    Nov 15, 2011
    Posts:
    5,618
    Wall of text is wall'y
     
    shaderop and Meltdown like this.
  4. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    seriously, i wouldn't consider that much line a wall of text but if it bothered you, i apologize
     
  5. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    The problem is more the punctuation, grammar and paragraphing, as opposed to the straight out quantity of text.

    Back to the OP, I'm not entirety sure making the engine thread safe would be in every ones best interest. Multi threaded environments can create all sorts of difficulties. Unity prides itself on being an engine that anyone can make any game in.

    Making the engine thread safe would come at a performance cost. That cost would be mitigated by the benefits of multi threading in some situations, like the one you described. However there are still platforms Unity targets with only one core. So the cost of being thread safe there would come with no gains.

    You can still farm out anything that doesn't touch the unity API to different threads. I'm not sure how many games would actually benefit from other threads touching the API.
     
  6. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    Not to mention that internally there's plenty of threading, and you can do your own threading. You just have to push data or make calls via the main scripting thread. It's a limitation, yes, but that's not the same as threading being impossible.
     
    Dustin-Horne likes this.
  7. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    first of all, sorry for my English problems, I'm not a native English man, but i agree with you that some of the platforms are single core, but should it restrict us for somethings to be possible? I mean, if someone is targeting mobile, he already knows that he should not use something like line cast on threading, but if he is targeting pc, why not being able to choose between using it on thread or not? As much as I know, the most important things for games like FPS TPS, is line of sight and fire for making proper tactical decisions and they only need line cast and ray cast stuff, provided, those two may need to make the whole physics engine thread safe, but I assure you, the day will come that Unity developers may need to consider this just so that they don't fall on the development speed of other engines, question is, why wouldn't they start doing something about it now instead of when they are forced to by the nature of industries?
     
  8. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    From reading more of the text above, why do you need to do all of your calculations based on the data directly in the scene? A couple of considerations.

    First of all, the representation in the scene isn't necessarily the best representation for your AI making decisions or other potentially heavy workloads. For my heavy AI I make specific data structures optimised for the data they need and the decisions they're making with it. Once I've got that data structure making it safely accessible to another thread is pretty easy (as it's probably read-only for the most part), and I typically want to assign any new commands at the start or end of an update cycle anyway so going via the main thread isn't much of a downer.

    That brings me to the next consideration, which is that being able to access things in the scene willy nilly from other threads complicates both your work and Unity's - you have to not accidentally trash shared data, and Unity doesn't have to work around your access to said shared data (which can be a major) performance issue if it's not handled well).
     
    GarBenjamin and Kiwasi like this.
  9. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    ok, first of all, we are on same boat about collecting data on needed structure and making it available for other systems in game, but i think you didn't get the point, first take a look at this article anout alien swarm :

    http://aigamedev.com/open/review/alienswarm-node-graph/

    How many nodes are there? I bet more than 8000, if you put node positions, their connections, AND the visibility into a serialized file, even with protobuf that is making nice serialization sized you would end up with hundreds of megabyte of data for single game level !

    so one way or another, you would want to collect these data at run time, with near 8000 nodes it may take up to 30 seconds for COLLECTING those data, the data structure and how you present them to other systems in not the question here, the collecting is. All you are arguing about is that it can be done in main thread and I know that, as I said in first post I'm not ignoring the ability to use coroutines, but using main thread mean that :

    1- You are ignoring more cores that are available for those calculations
    2- You are dividing the main core resources that maybe very much needed for stuff like rendering etc
    3- You are limiting the whole system until those co routines finish their jobs

    so, as I said many times by now, the question is not about IF there is no core, IF you can do something this or that way, the question is about NOT being able to do more stuff WHEN you can there is enough hardware resourced to do it. The AI situation that I presented was only and example of many other things that is getting on the way. Any part of unity api will not run and through errors telling that this or that should be done in main thread, that means that you are restricted lots of times from using processors available, specially in heavy parts of the game making that NEED unity API
     
  10. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    No, I'm just not assuming that more cores will make things magically faster. I get where you're coming from, I just don't imagine the performance benefits will be as big as you clearly do.

    Edit: I'm also not seeing the issue with accessing 8000 nodes that have to already be stored in the level data anyway. Why do you see making them separately available to your AI system chewing up hundreds of megs of extra data somehow?
     
  11. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    very simple, because i'm speaking from experience and already tried it, and don't even think I didn't optimize it on serialization, when you sit and think, each point has a vector 3 as coordinated means 3 * 32 bits,

    now, for each node we will consider 30m radius a rough distance to keep track of visible nodes, for 2m node gap = 15 node in each radius means that you will have a look up table of 225 nodes

    multiply it by 96 bit per node position roughly = 21 kb / node and multiply it for 8000 node roughly = 167 MB just for visibility table sir , simple math there, roughly same amount for the line of fire makes it near 335 MB ! and that is only for position, that's if we ignore the fact that we need to convert back that position to node it self to get more data reading.

    if you instead use guid as visible node ID in the table, you get 16 byte / node because guid is 16 byte data structure type, in other words, not only you didn't reduce the size but you couldn't solve the look up for the node finding in future too. So, instead of blindly arguing with me, sit back and if you don't want to bother trying, at least accept a man's words that HAD already tried it in many different ways. Hell I even thought about representing the node visibility and line of fire in single chromatic textures and even int structures but even that wouldn't help as much as you think it would. Long story short, these type of things can't be done in a short cut fashion more than what we all know possible. And as I said, instead of clinging to my AI example, think about many other possible times you can use api on multi threading.
    Also, I never said using multi threading would magically speed up every thing but, is there anyone out there that can argue about multi threading is not being useful in game programming and design these days?
    If you want to still continue arguing about that AI example, I won't comment any more simply because I know the facts and worked on them and there I don't see any more point in arguing on that.
     
    Last edited: Feb 13, 2015
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    7,180
    You can use multithreading for all your logic however you want. The limitation is that you have to serialize back the results if you want to pass them to Unity APIs.
     
  13. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    Why is a node 21kb? It's a position and a list of other nodes. I get 12 bytes (position) + 60 bytes (4 byte reference * 15). Even with overheads that's nowhere near tens of kilobytes. What am I missing there?
     
    calmcarrots likes this.
  14. sootie8

    sootie8

    Joined:
    Mar 25, 2014
    Posts:
    233
    Just a wild stab in the dark @Jiraiyah; Have a dictionary(dictionary A) of all the nodes (8000 Vector3's (value)), with a key of type ushort(0-7999) call it the instanceID. Then have another dictionary(dictionary B) that contains the same 8000 instanceID's(as key), with the value being 225 ushort values(as an array).

    Then, once you determine your starting node and you know its InstanceID, you lookup its corresponding nodes from the dictionary B. Then for each of those nodes you can cycle through and do a dictionary lookup on dictionary A for each of them, and so on.

    All you need to do is store and serialise this beforehand, JSON.net or something.

    EDIT: It should reduce the size of your initial data storage.
    112KB for dictionary A (8000 * (12 bytes + 2bytes))
    3.616MB for dictionary B (8000 * 225 * 2bytes + 16000(for ushort))
     
    Last edited: Feb 13, 2015
    angrypenguin likes this.
  15. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    3,807
    honestly, they should make multithreading way harder so fewer people think they need it.

    ie: you need to have multiple threads running raycasts in order to run ai code.

    sorry to be a grumpy old man, but of all the developers i've seen try to write multithreaded code - only maybe 1 in 10 actually knows what he's doing. the rest are just F***ing disasters, and would generally be much faster single threaded due to terrible lock strategies: ie: lock everything all the time because we have no idea whats touching the data when or where.

    Or actual real deadlocks under simple small scale conditions!
     
  16. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,562
    This, or make sure you're using a good data storage strategy to where you can you are using your other worker threads to update the data and do your calculations and store them off, then just accessing the storage from your main thread using a component. Then no serialization would be necessary as you're essentially using your "storage" object to maintain state between your other thread and your main game thread.

    This is so very true, and multithreading gets abused. One good example is Microsoft's new(er) Parallels library for doing PLINQ. You see people doing parallel foreach loops when the actual operations on each element are literally like nanoseconds and far more time is wasted on thread management by the CLR than if they had just either done their work in a single thread, or batched it across multiple threads.

    Locking strategies and locking in general are also a very important point. Obtaining a thread lock is not a cheap operation. In order to make the entire API / Engine thread safe, there would have to be a lot of locking taking place to account for potential concurrency issues. This would slow down the performance of the entire engine, so much so that for the very few bits any individual developer would be using async operations, the costs of obtaining locks elsewhere would likely mitigate most of the gains. Not to mention, at that point they could only guarantee that the base API was thread safe... the second you add a third party component of any kind from the asset store, you lose any guarantee of concurrency.

    While I love multithreading where appropriate... I think it would be extremely unwise to try making the entire engine thread safe. Why do you think the entirety of the .NET framework is not thread safe by default? Some things are... but many things are not. You have to implement and handle thread safety on your own.

    On that note.. it would be nice if it were possible to access the API from separate threads as a "do so at your own risk" type of scenario... but I wouldn't want it to be thread safe by default.
     
    angrypenguin and Ryiah like this.
  17. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    3,807
    this. In all honesty, I would say that if it seems like accessing unity api from other threads is a problem, then you probably shouldn't be trying to write multithreaded code in the first place.

    OTOH, we all had to start somewhere, and anyone who's written a multi threaded system has probably written terrible multi threading code (mine was ungodly bad when i was first learning). It's part of learning.
     
    Dustin-Horne likes this.
  18. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    that is exactly what I am doing right now, actually the things that I'm keeping in serialization is a bit more that just the position of the node, it's a guid for node zone, the position, three lists for possible covers close by, a list of guid for neighbour nodes, well, the neighbour list and cover nodes can be optimized a little based visibility in neighbour case and covers on top of each other (crawling, crouching and full stand cover in same x,z coord don't make sense you would only keep full stand one and remove the other two.)
    but even with all these data still the size is totally acceptable, what I was speaking about was when you would like to serialize the LOS, LOF on the disk. That would cause huge amount.
    also, two notes for you
    1- Protobuf is much better than json for multiple reasons :
    a ) it's binary format so won't use unnecessary size on disk for string formats.
    b ) because it's binary players won't be able to hack to it easily
    c ) it supports Unity by default
    d ) it's free
    e ) it serialize / deserialize much faster than json
    2- A proper look up table for node's things would be like Dictionary <Guid, List<Node>> for each neighbour, in range, LOS and LOF tables, this way you won't waist any extra time for finding nodes from ID or position of them.

    Finally an answer that would make me double think the situation, knowledgeable answer, thanks sir, now I know why it would be impossible to think about the whole engine as a thread safe, that makes totally sense now.

    That was exactly my point on first post sir, but your first part of the post makes me double think the possibility of it, but at least I would like to see little parts of physX being available from threads.
     
  19. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,306
    Won't Unity 5 have to be thread safe with it's new Multi-threaded Task System?
     
  20. Eric5h5

    Eric5h5

    Volunteer Moderator Moderator

    Joined:
    Jul 19, 2006
    Posts:
    32,225
    No, that's for certain internal things such as culling, not user code.

    --Eric
     
  21. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,306
    Pity.

    I thought as they were making the core a multi-threaded task based system then the API would naturally be more easily converted to be thread safe.
     
    shkar-noori likes this.
  22. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    Why does saving it on disk have to take any more space than storing it in RAM? It doesn't need to be portable, just write the data out exactly as it is in RAM (or, as close to as .NET/Mono will let you - close enough).
     
    sootie8 likes this.
  23. imaginaryhuman

    imaginaryhuman

    Joined:
    Mar 21, 2010
    Posts:
    5,695
    The graphics api's are single-threaded, and the way they interact with the operating system I think it's sort of less possible to treat them multithreaded? Like you couldn't submit multiple graphics rendering requests from multiple threads at once.
     
  24. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,306
    No, surely it must be multi-threaded now or how else will they get the most out of Metal, Mantle oops sorry DirectX 12 and Next GL???
     
  25. Velo222

    Velo222

    Joined:
    Apr 29, 2012
    Posts:
    1,374
    I still don't know how to multi-thread anything in Unity. Even with Unity 5 now.

    Everything you guys are saying both scares me, and excites me at the same time. The very concept of multi-threading is amazing to me in terms of possibilities.

    Also, could someone please explain to me what Serialization means practically in terms of code?

    What does "serialize back the results" mean?

    Sorry, in some ways, I'm still a complete noob when it comes to some programming concepts, because I'm largely self-taught.
     
    shkar-noori likes this.
  26. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    Serialisation is the concept of turning a class into data, then turning it back into a class again. Its what enables saving, network play, the unity inspector and a bunch of other stuff. It can be as simple as saving the raw memory bits that make up the class, or more sophisticated, like storing all of the variables in a human readable xml.

    At its simplest form, multi threading runs something like this. (This is a massive simplification. Don't jump on me for the details, I'm trying to hit a beginner level here).

    Unity thread
    - Sets up the data for the new thread
    - Starts a new thread for complex data manipulation
    - Checks from time to time to see if the manipulation is done
    - Implements manipulated data into game world

    Other thread
    - Accesses data
    - Modifies data

    The big power of threading is that both threads can execute simultaneously. And not just the fakery we are used to, a multi core CPU can actually process multiple threads at the same time, by using different cores to do it. This leads to the biggest challenge for multi threading, often referred to as "thread safety". In essence this occurs because two threads can manipulate the same data at the same time. Data can be lost, corrupted and cause any number of disastrous results if this happens.

    Thread safety is achieved in a variety of manners. You can lock data, preventing any access to it from other threads. This can lead to dead locks. You can use a queue to store instructions from each thread to act on later. You can use flags that indicate when your thread is finished. Or you can take the Unity approach and say "Only one thread can touch me".

    Each of these approaches has its performance costs. At the heart of this thread is the question "Is the performance cost of making the Unity Engine thread safe worth the performance benefits of having multiple threads?" As you can see there are opinions on both sides. The answer really comes down to "What kind of game are you making?" There are also development concerns, good multi threaded code is much harder to write then good single threaded code.
     
  27. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,440
    Need to thread this thread.
     
    shkar-noori and Kiwasi like this.
  28. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    I would suggest, learn C++ and create your own game from scratch.
    That way you can use multithreading, use strong, fast compiled language, not scripted one.
    Have FULL control over the game, yes you have to create everything from scratch but you have the control.

    And last big PLUS, you will learn A lot from this process. And not have to rely on other engines, what will they support, buying 1500$ Unity pro version etc.
     
  29. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    Really? Thank you, but....umm, why are you here then?
     
  30. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    15,310
    Or you can use Unreal 4. :p

    The biggest catch with this is you won't necessarily do a better job. It is entirely feasible for someone who thinks they know multi-threading to produce a program that is slower than a single threaded application.

    It isn't like you won't have to convert down to single threaded at some point. Unless you only want to target DirectX 12.
     
    Last edited: Feb 15, 2015
  31. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    The right tool for the right job. If a particular project doesn't need the added power/flexibility/control of writing your own stuff from scratch then why incur the significant costs of doing so?
     
  32. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    Well, I would say that somethings like physic's ray casting should be easy to handle on multi threading, simply because almost 80% of time you are not passing it's result to other methods or other classes, worst cast you are using it to collect some data and make a decision. But as you said, it seems that unity developers tend to use the least tedious approach and say, you know what, we will handle all in one thread.

    I had seen some people starting to port or make wrappers for bullet, simply because it could be used on multi threads, but these people make sections that they need and then leave the project. Honestly, I'm not that familiar with physic engines my self to make any comment on difference and which is better PhysicX or Bullet, but at least, those tries should show something to developers, if you are not going to make the engine thread safe (that I can totally understand why after all discussions), then at least make little parts of physic engine specially ray casting thread safe. You may ask why?

    well, the thing is that unlike RPG games that are more dependent on game mechanics and stats, shooter games are more dependent on two aspects a ) good graphics b ) good AI simply because there is no time for telling stories like what is possible in RPG, and for having good AI, the amount of times industrial level games perform ray casts is crazy. But the thing that make them be able to do it, is most of the time being able to put these kind of decision makings to threads, and remember, part of each of these queries is to check line of sight and line of visibility and find a possible good cover, and in all three cases it uses ray casting. Hell, even left 4 dead is using multiple ray casting for their AI climb animation decision making just to find a good anchor point for their hand and choose proper animation to blend to ! If you want to do all these ray casting on same thread, you come to the place that one of the most shooter games spoke in an AI interview, "We could not have more than 10 AI at each time. And that is when we keep all the AI on the server and the scene presentation on the server is as simple as boxes and characters are moving around with poses and not real animations!"

    10 AI at each time? I remember Rainbow six vegas 2 had maps with more than 25 AI and those AI where some what smart (not top notch but good enough to make you double check your back instead of rushing in.)
     
    Last edited: Feb 15, 2015
  33. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    Do you really think unity devs are not doing stuff on multi threading internally? main question is, why don't they trust developers to expose parts to us, even little by little or at least the minimal possible. Hell any thing more than debug or vector math would yell at you that it should be handled in main thread !
     
  34. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    No, see my earlier posts. I think that they are doing it internally and restricting our API access to the main scripting thread is a part of their strategy in managing that. They can control what they're doing on the inside of the engine. They can't control what our scripts do or when they run (with regard to the fact that we could be spawning our own threads), but they can control when scripts can access the API.

    What you do with the returned data is far less important than what you're making the computer (CPU/RAM) do by calling the method.

    Consider how a raycast works, particularly with regard to memory access. Raycasting involves a few "expensive" calculations interspersed with a bunch of random reads spread out over a potentially large dataset. Multi-threading increases computational power, but all threads have to share memory access bandwidth. Mileage will vary depending on many factors, of course, but I wouldn't just assume that more threads will make raycasting it go faster because I wouldn't assume that the bottleneck is necessarily computational speed.

    Of course, the job queue stuff could porentially allow us to take advantage of internal threading in the future and give us the best of both worlds.
     
    Ryiah likes this.
  35. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    Wow. I really hope this is tongue in cheek. If you can make an engine that does everything Unity does in less then 300 man hours I applaud you. You should probably have several PhD's by now, and a Nobel prize or two. Or you could be secretly building SkyNet in your basement, and I've just singled myself out as your first target. Yeah, I'm just going to stop talking on this one... :)

    Think this through. Raycasting requires the position of every rigid body to be known and constant. You can't raycast during the internal physics tick, as the position of every ridgidbody is not known/constant. This happens on average 40 times a second (depending on your fixed time step). Note I said on average, its not at a set rate of 40 times a second, the engine plays catch up depending on your frame rate.

    There are a bunch of systems you could use to overcome this.
    • Lock the physics data while you do each raycast. I'll let you think through why this is a bad idea.
    • A double buffering system. Copy all the physics data into a buffer, then raycast on that. May work while you are doing multi threading. Terribly expensive if you don't ever use the buffer, or if you have to maintain it when you are not using it
    • Queuing the raycasting back to the main thread. This is not really multi threading. But its the way the system is designed to be used
    In games its all about faking things. We don't create real intelligence, we create artificial intelligence. We create something that looks like its really smart, but really isn't. Here are some general thoughts.
    • Bake constant data into the scene itself. Do something similar to the NavMesh concept
    • Raycast less frequently. Do you really need to check line of sight with every enemy every physics step? This is far faster then a human player could ever react.
    • Line of sight doesn't change that often. Cache the results for a second or so.
    • Only collect data when there is a decision to be made. Found a target? Keep firing at the same target until it makes sense to change.
     
  36. 0tacun

    0tacun

    Joined:
    Jun 23, 2013
    Posts:
    245
    I could be wrong, but I read somewhere that UT plans to implement/expose a job system to make it easier to do multithreading stuff.

    And I think in 2015 it should be already possible to do so, since we aren't effectively using all cpu cores. Currently Unity games have a "bad" reputation of performance pared with not so stunning visuals.

    Modern games are heavily using it, personally I see the Uncharted series as a prime example (they used 500 asynchronous raycast per frame back in 2009 on a ps3).

    And i would not propose to hide this mighty weapon because "some guys will hurt themselves" as such thinking is not very forward orientated.
     
    Kiwasi likes this.
  37. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    Thinking about it exposing a couple of Async methods couldn't hurt. Something like RayCastAsync. So rather then allowing unrestricted access to the API unity tags a few specific methods as accessible by other threads. You'd still need a task based system on the other thread to wait for the result. I just can't see how the current physics engine can let a user do a raycast at an arbitrary point on another thread.
     
    0tacun likes this.
  38. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,471
    Yeah, that's exactly what I was thinking re: best of both worlds.
     
  39. Jiraiyah

    Jiraiyah

    Joined:
    Mar 4, 2013
    Posts:
    175
    and that is exactly what I mean by letting us having a way to do things parallel, because no matter how good you design your behavior tree, no matter how good you cast data in scene, when it comes to today's very dynamic game plays, you would struggle with your ai hiccup on decision making times unless there is a good way of putting those decision makings on a parallel process an let your ai continue what he was doing for few frames till the decision making result gets available.

    That was why I would like to see something like what we are all saying in place, I know about the process of ray casting enough from forums to know that it can be tricky to make it thread safe, but the idea of async ray cast and job manager is something that really needs to be visited by devs.
     
    0tacun likes this.
  40. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    I do have to finish my Unity project first don`t I ? :D

    The "engine/game" from scratch isn`t meant to build new engine like Unity with editor, dragging components etc. It is meant to just make game without any existing engine.
    So being able to import texture with C++ and display it on screen, move it can be called 'game engine'.

    But these days people think of Unity or cryengine when word engine is mentioned. These are one of the most complicated softwares in computer science to this day so....

    Yes, but then for the subscription I would rather use cryengine :D
    And again, you will learn more how engines and games work in a background when you try to do something from scratch.
    Also when you want to apply for dev position, it will be better to have strong C++ knowledge, then have strong Unity3D + C# knowledge, since almost no companies use Unity (At least I know of)
     
    Last edited: Feb 16, 2015
  41. Eric5h5

    Eric5h5

    Volunteer Moderator Moderator

    Joined:
    Jul 19, 2006
    Posts:
    32,225
    Lots of companies use Unity. EA, Disney, Blizzard, etc.

    --Eric
     
  42. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    Yes but I doubt I will get job as heartstone unity developer :D
    Not everyone lives in US where there are big AAA companies. So I have to look for job opportunities that suite my continent/region(EU)

    But no, no, I would go C++ 80% because of huge learning experience. To learn how game engine works, what is main game loop, buffer objects, sending VBOs to GPU, learning about memory allocation, doing entity component system architecture etc. It would be pain and a lot of struggle :) And even if word is: "Make games not engines", I would rather make an engine first to gain the skill and knowledge, and then make a game xD

    But that is just my opinion.
     
    Last edited: Feb 16, 2015
  43. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    15,310
    It's a good way to learn, but I think you'll find yourself never doing it again. At least that's how it has ended up for me.
     
    Last edited: Feb 16, 2015
  44. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    So you was in Unity, moved to C++/OpenGL, and then back to Unity? :D
     
  45. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    15,310
    I picked up C++/OpenGL before Unity existed. At the time there was little choice.
     
  46. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    How did it go? What was your experience with it? And why did you give up?
     
  47. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    15,310
    I didn't give up so much as reach the conclusion I couldn't get very far without a considerable amount of effort that simply felt wasted. I couldn't afford any APIs that would have made it easier and there weren't many, if any in some cases, free alternatives.

    It was a similar situation with compilers. I had to use GCC as practically every other compiler was very expensive. I believe Visual C++ sat around a few hundred dollars.

    I primarily stuck with SDL, as it handled practically everything including threading, aside from OpenGL which handled rendering. If I ever needed a custom engine for any reason, I would still go that route. The tutorials have definitely gotten better for it.
     
    Last edited: Feb 16, 2015
    angrypenguin likes this.
  48. Wrymnn

    Wrymnn

    Joined:
    Sep 24, 2014
    Posts:
    299
    There is also Visual Studio 2013, which is free.
    You could you SDL, which is very good (it`s from OpenGL).
    But yes, now modern developers couldn`t imagine making game without the editor. No dragging components on object, no having object on scene, positioning your modeled tree to where it should be etc....

    But the people who can work with normal C++ code and make games from scratch, I admire them and have my utter respect.
     
  49. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    3,807
    I think this kind of thing is exactly what many people misunderstand. Writing high performance is not about throwing in 500 threads or some other silver bullet. The first step to writing good, fast code is just doing less work or at least, shifting around when work is done.

    BoredMormon lists some really good suggestions to making the code 'faster'. Notice that they're not about a 'programming trick' they're about just thinking through the problem and asking yourself what you really need when. Performance, especially in higher level systems (ai), is often more about redefining the problem than anything else.
     
    angrypenguin, Kiwasi and Ryiah like this.
  50. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    3,807
    I don't really see why this is needed though. Just make a queue of "raycast request" have the main thread dequeue the request, and hand off the result to an output queue that a worker thread processes. Or am I missing something?
     
    Kiwasi likes this.
unityunity