Search Unity

Job is struct, no pointers allowed, so how does a job return a value?

Discussion in 'Data Oriented Technology Stack' started by UnLogick, Mar 24, 2018.

  1. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    It seems excessive and wrong that I have to allocate a NativeArray<int>(1, Allocator.Temp) to return an int value?

    It could be that the job was to analyze some data or it could simply be that you allocate a too large data buffer and then want to return a count of how many entries you wrote.
     
  2. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    598
    Current solution is a NativeArray<T> of size 1.
     
  3. RootCauseProductions

    RootCauseProductions

    Joined:
    Feb 1, 2018
    Posts:
    31
    Remember the two words fast and safe when you find a limitation or something that seems "excessive and wrong" in the Jobs or ECS systems. NativeArray (and other Native collections) are designed to be very fast, but allows Unity to check for and prevent problems you will run into with multi-threaded code.
     
    recursive likes this.
  4. mike_acton

    mike_acton

    Unity Technologies

    Joined:
    Nov 21, 2017
    Posts:
    109
    Are you calling Complete on the job handle? A return value implies that you’re waiting for the job to return. Ideally systems do not wait on their jobs so they can be scheduled overlapping with other systems (which is safe because we can guarantee things with data permissions.)
     
    laurentlavigne and recursive like this.
  5. RootCauseProductions

    RootCauseProductions

    Joined:
    Feb 1, 2018
    Posts:
    31
    It is working as documented. I.E. only NativeArray and friends are returned from a Job, not a raw int variable. @UnLogick wanted to return just an int instead of a NativeArray<int>[1].
     
  6. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    @mike_acton this is not a bug report but more a feature request. Sometimes the output of a job is not a large array of data, but merely a summary that ought to be a class/struct. As I wrote in the original post I could create an array of size 1, but that's a hack not a proper solution.
     
  7. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,397
    But most good concurrent designs have a way to pass messages/events to the next thing in the pipeline/flow when they are done. Actor model, networking pipelines, even ring buffer based stuff like disruptor. The only obvious thing with jobs is to complete and get a return value.
     
  8. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    Well that's part two... once I've analyzed the data I know how big buffers to allocate for the next step... however with the current design I only have two choices... allocate too much or wait till the first job is completed. Neither option is particularly optimal, would be nice if you could enqueue a callback once your job was complete. A toggle could decide if it's enqueued on main thread or simply invoked on the worker thread immediately following the optimized method.
     
  9. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
  10. RootCauseProductions

    RootCauseProductions

    Joined:
    Feb 1, 2018
    Posts:
    31
    All of the jobs that I have done so far are just to move processing into another thread. However, it seems like you would just use the same Native* in both jobs where one writes to it and the other reads from it and setup the dependencies correctly.
     
  11. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    //Sounds potentially promising, where can I get this NativeList? (It's not in my 2018.1.0b12)
    Edit: Any particular reason why NativeList is part of ECS? Lazy merging?
     
    Last edited: Mar 24, 2018
  12. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    Yes, that's handled by Job dependencies, you can schedule a job to run after another job. My problem was that my first job needed to output something dynamic that the second job then read. I'm still not sure I can use the IJobParallelFor since my second job takes the first dynamic output and makes yet another dynamic output. Luckily I need to run many of these batches so each of them doesn't have to be threading.
     
  13. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    It’s part of the unity.collections and unity.jobs package. They are seperate, ecs depends on them. But if you like you can pull them seperately
     
  14. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    I'm afraid this is part of that missing documentation, atleast I've found no instructions on this. The way I got ECS was to copy a magic manifest file, I have no clue on what packages are available and what to write in a manifest to get it. Several people refer to getting experimental packages, but I found no ui that will let me browse them and download them. Googling for stuff like Unity Experimental Packages it comes up short too.
     
  15. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    Experimental packages are currently not listed anywhere, in future releases we want to allow for listing preview packages as well in the package manager window.
     
    FROS7 likes this.
  16. UnLogick

    UnLogick

    Joined:
    Jun 11, 2011
    Posts:
    1,594
    The problem isn't so much they're not listed, without a blog/forum post about what it contains it would just be noise. The real problem is that even when I learned about the experimental packages I could not find a single article on how to get it.

    So for all those out there who are still wondering, you download experimental packages into unity by placing a manifest in the Packages directory, next to Assets and Project Settings in your project.

    The manifest file I used came from the ECS samples, if my json reading works this gives me entities, collections and jobs.
    https://github.com/Unity-Technologi...es/blob/master/Samples/Packages/manifest.json
     
    vestigial likes this.