Search Unity

  1. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  2. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  3. Want more efficiency in your development work? Sign up to receive weekly tech and creative know-how from Unity experts.
    Dismiss Notice
  4. Participate with students all over the world and build projects to teach people. Join now!
    Dismiss Notice
  5. Build games and experiences that can load instantly and without install. Explore the Project Tiny Preview today!
    Dismiss Notice
  6. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Why? (Rant alert)

Discussion in 'UI Systems Previews' started by pbrito_unity, Nov 28, 2018.

  1. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    Why did Unity choose the "semantic web" approach?
    I started recently using Unity, because I wanted to escape the establish "web way" of programming and now it's here also.

    I use to listen to Casey Muratori (and people alike) laughing about how stupid "web development" was, and how difficult it was to a basic thing like center a element.

    No one writes html, html usually is generated by "web frameworks" and every year there's a new one that will solve all the web programmers problems. Now there's even people saying the future is web assembly -throw away the html stuff.

    XML is just a hierarchical structure, a tree structure, represented in a textual way.
    This makes difficult to visualize the structure, trying to find the wright closing tag can be a difficult task( html browsers don't reject incorrect html). And contrasts against the Unity tools way of doing things.

    Also means that you have to use a external tool, a xml editor. What "web programmers" do is open a browser and go to the "developer tools" and compare the html that is shown by the browser(that was generated by the framework of choice) and try to figure out what the browser is doing, not pretty.

    CSS is also a bad idea, historically CSS was not created for programmers. Was a way for the user to chose the way he liked to see the document.The document had the information and meta information.For example a document had a title(meta) and the CSS described how to show the title.
    As the web evolved, CSS started to be used in different ways, for animations and other things.
    Now you have three "languages" (html, javascript, css) to produce a single web page, and they have to be congruent in a synchronized way. The separation of concerns, if you are doing something more than the basic, never happens there is always some kind of inter-dependency.

    XML could easily be replaced by a Hierarchy window like the one for unity scenes, or even merge the two.
    XML element would be equivalent to a game object, XML classes to prefab variants and you could add components to elements (event triggers, button component, image, etc ).

    Some more arguments:
    Java also tried the XML for UI without great success.
    Swift doesn't uses xml for UI,
    QT doesn't uses xml for UI,


    I hope Unity makes the best choice.
    Good luck.
     
  2. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    23,776
    Unity appears to be making decisions about tech where less or no training is required for people to use Unity. So if you know a bit about photography, the physical camera makes sense. I imagine it's similar thinking throughout.

    Should they design something that works like nothing else does, or design something that works like other things do?
     
  3. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    Apple has this for non coders:


    And Qt has this:




    Even Google is pushing Flutter now
    They ditch xml and css:
    Code (JavaScript):
    1.  
    2.  
    3. <div class="greybox">
    4.   Lorem ipsum
    5. </div>
    6.  
    7. .greybox {
    8.   background-color: #e0e0e0; /* grey 300 */
    9.   width: 320px;
    10.   height: 240px;
    11.   font: 900 24px Roboto;
    12.   display: flex;
    13.   align-items: center;
    14.   justify-content: center;
    15. }
    for:
    Code (csharp):
    1.  
    2. var container = Container( // grey box
    3.   child: Center(
    4.    child: Text(
    5.      "Lorem ipsum",
    6.      style: bold24Roboto,
    7.    ),
    8.   ),
    9.   width: 320.0,
    10.   height: 240.0,
    11.   color: Colors.grey[300],
    12. );
    13.  
     
  4. LurkingNinjaDev

    LurkingNinjaDev

    Joined:
    Jan 20, 2015
    Posts:
    1,698
    Honestly, I don't understand what your problem is. Unity already told us that they're planning to deliver WYSWYG editor for UI, so what's your problem exactly?
    XML and CSS are good for UI and good for the underlying engine (it's a mature technology) so I don't see what's your problem.
    Oh, and Unity UI isn't just for mobile so you really need to calculate with ALL of the supported platforms. This means it is irrelevant what Apple or Google do or don't do.
     
  5. Meatloaf4

    Meatloaf4

    Joined:
    Jul 30, 2013
    Posts:
    21
    @pbrito_unity

    Under the hood though they are essentially building out xml and css like files. I think the thought is that once you have a good foundation in place (the xml / css approach) you can create a visual editor to supplement the UI creation of these files.

    Having created UI's for both Unity and the web I am firmly in the camp of web. The current UI implementation in Unity is easy to get into but fails for long term maintainability in my opinion.
     
  6. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    12
    Hello,

    I am the lead of the development team of UIElements and I've been working on this since its inception a couple of years ago.

    Your comments revolve around the choice of using XML and CSS as an inspiration for UIElements resources formats. I will try to address your concerns the best way I can.

    First I'd like to add that UXML and USS are independent and completely optional to work with UIElements:

    • you can build hierarchies of elements in C#
    • you can set styles on elements directly using element.style.XXX =
    • you can hookup event handlers to change the style interactively
    So far the Unity Editor UI has been built on top of IMGUI which among other limitations, really encourages a mix of layout, visual appearance, UI logic and sometimes business logic.

    A very big lesson of web development is to adopt a data driven approach where the XML hierarchy describes the contents of a document annotated with IDs/classes.

    This promotes a separation of concern that enables non-programmers change style definitions at large and allows the presentation of an application/page to adapt to many situations.

    I agree this is not a perfect technology but it is a fundamental aspect of web development that allowed to thrive and basically become universal.

    This aligns perfectly with what we are trying to achieve at Unity, starting with the Unity Editor itself. What I mean here is to enable contribution with UI/UX designers to our UIs and allow the editor to target more platforms.

    Another thing to consider is the familiarity. Since we want to make Unity as approachable as possible, building on known patterns lowers the barrier of entry for new users who may already have a background in HTML/CSS.

    Surely doing layout in CSS may cause some frustration but it is a very powerful tool and as such takes some time to learn. Besides, CSS has come a long way since its early years and new layout features were added which hopefully makes it easier now.

    For example centering an element is as simple as changing the justification property when using Flexbox: http://jsfiddle.net/jm830xw9/

    XML is an appropriate data format to describe UI assets:
    • Its main purpose is to describe hierarchical data (slightly better than JSON for example)
    • It has standard mechanisms to write schema validators
    • It is easy to merge
    • It is supported by most text editors / IDEs (that should help with finding these missing closing tags :)
    However you're right that offering a visual workflow for hierarchy and properties is very important, that's why we have a visual UI builder in the works.
    Our early prototype taught us that we can both provide direct control over element layout and properties in the builder while maintaining perfectly valid UXML/USS files. (We'll be in touch in this forum section to share early versions of this tool). So right now it seems like those file formats are a good fit both for source code and data format.

    In the Editor there is also a UI debugger which can help you inspect live hierarchies and understand which properties are applied to elements.
    The hot reload of USS assets for editor windows also allows previewing of styling changes, which might not possible in other pipelines where styling is baked into the code.

    I hope that those clarifications will make you less skeptical about our decisions.

    Feel free to raise any other concerns you may have or ask clarification about my answers.
     
    Gor-Sky, Kirsche, devion-user and 5 others like this.
  7. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    1,730
    the upcoming visual UI builder sounds great, I hope it also works together with the upcoming visual scripting, so non programmers can make an editor UI with the help of those tools, as wel as an in game UI
     
  8. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    12
    Yes, definitely something we have in mind :)
     
  9. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    My objection is precisely that XML and CSS are not good. Why use it?
    It's not flexible tech, not scalable is not even readable.

    If it's a mature technology why don't big corporations use it?
    big corporations love mature tech, maybe they think that XML sucks. I don't think they are stupid, so it is not a bad idea to check what they are doing.

    As for your last point,Unity is compiled to the platform you choose, that means you don't have problems like the need of compatibility and legacy. Just take in account resolution ,height and width, other things are common to the rest of Unity (perfomance, etc). One of the selling points of Flutter is precisely that is multi-platform.

    Just expressing my opinion,
    I first use XML a long time ago(2000) with java, picture that! XML can be nice for small and static tasks like svg, not for dynamic and interactive UI.

    Edit: I wrote this before reading antoine-unity response. Reading now.
     
    Last edited: Nov 29, 2018
  10. LurkingNinjaDev

    LurkingNinjaDev

    Joined:
    Jan 20, 2015
    Posts:
    1,698
    Apparently you know very little about big corporations. They actually use these. I pretty much hate XML as data storage, but they are excellent as template files.

    Wut?
    You know that people using Unity frequently developing their application to multiple platforms, right? Does Flutter run on XBox One? Ps4? Okay, maybe Nintendo Switch.

    BTW, this entire Flutter reminds me to GWT... It's scary. I don't like defining the view portion in code. At all. It blocks non-coders (designers and all) to tweak. They understand XML and CSS (since they are just markup languages) but they don't understand Dart or JS or Java or C#.

    I last used XML with Java about 10 minutes ago. What's your point? Do you know what XML is good at? Describing structured elements -> structured UI is exactly that.
     
  11. Vacummus

    Vacummus

    Joined:
    Dec 18, 2013
    Posts:
    49
    Being a web developer for 6 years, I am excited about the work you all are doing, and I am all for the move towards more web based UI development. Though I do have some concerns. It seems the new UI system is inspired by JQuery, which is becoming out dated tech in the web development industry. The industry has been moving more towards functional programming and the component pattern over the past 5 years (thanks to React.js). Have you all looked into React.js and JSX? I think that would be a much better direction than the JQuery/Html approach.

    Also, have you all considered ECS? Data Oriented Design and ECS would work really well for you UI development. And though I love React and functional programming, I think a data oriented approach to UI development would be better as I believe that it would be more performant, more scalable, and less complex then React. It also would be more familiar to Unity users using ECS, and along the lines of the direction Unity is pushing the product.
     
  12. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    Thanks for your reply,I appreciate the tone of your response.
    But this is a topic that I have strong feelings.


    Basic UI with a menu and buttons is easy to do, you can even use XML,shure.
    Now when things get tough like :
    responsive design,
    complex user interaction
    and animations.
    XML/CSS is almost useless (mainly because of the declarative nature and because is not a programming language).

    Example:
    You want to put a layer on top like a moving fog image with transparency , underneath you put some buttons.
    If you do not control the event system how do you click the buttons? In Unity where the event system doesn't relies on the hierarchy is easily done. Why didn't you take advantage of unity colliders concept.
    One consequence of this model is that end up with a complicate structure(divs inside of divs) just to overcome the limitations what make the layout more complex and error prone.
    In my last project I end up with an almost flat hierarchy and managed the layout myself with React, then I could move around divs and manage the user clicks.

    I'll take airbnb company as an example of current web development.
    Their main focus is to build a workflow to work with higher concepts, so they use or try to build tools for that effect.
    This is a leading designer talk:


    They use sketchap, and build tools on top(looks like Sketchapp opened their API). Sketchapp is very popular at the moment
    https://airbnb.design/the-evolution-of-tools/

    A piece about empowering designers :
    https://airbnb.design/prototyping-for-hosts/

    What I see is that web development is past the XML, now is more a obstacle, and peolple are working around that.

    Edit:
    this is the CSS the ugly reality:


    Edit 2: And I forgot to mention there is navigation also, that is tough to get it right.
     
    Last edited: Dec 3, 2018
  13. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    2,408
    Ia am not fan of xml, but CSS

    Surly google would be first in line, to imply new tech, if better and reliable solution would be there.
    No need much, to see web pages using CSS.

    upload_2018-12-3_2-9-30.png

    Familiarity and simplicity is major PLUS.
     
  14. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    Let me clarify,
    CSS is worse than XML.
    XML is just verbose but flexible way to store and pass data around. XML can be useful.

    CSS is just a bad DOM API (mix with meta-data stuff),
    In Unity you can select a game object in the scene window and make modifications in the inspector.

    Do you think a browser dev-tool compares to the inspector?
    It's even possible to control events here, instead of the bubble up DOM/CSS idea. Try that in the browser.

    There's no word processor that uses CSS, or photoshop , or maya or any other consumer(easy to use) application that uses CSS. Also the mobile space that is new is not using CSS - XCode,Flutter....

    Its simple for example for Unity to create a Ad hoc CSS thing on top of the framework. For basic stuff, just to make CSS people happy.
    I think the team at Unity is taking XML/CSS as base(inspiration) and the consequences will be bad. They could take inspiration in what Unity does well.
     
    Last edited: Dec 3, 2018
  15. antoine-unity

    antoine-unity

    Unity Technologies

    Joined:
    Sep 10, 2015
    Posts:
    12
    I would like to clarify that the new system isn't directly inspired from jQuery. jQuery made it popular to use CSS like selectors in JavaScript to manipulate DOM data, but the core functionality of it mostly comes from native DOM APIs like querySelector().
    Although React/JSX is a drastic change from writing vanilla JavaScript/HTML, it still runs on top of the existing HTML tech stack and therefore couldn't really live without it.

    As such the vision is not replicate jQuery or React, but rather make it possible to write similar framework and APIs on top. This layering has the advantage such of letting developers decide what works best for them.

    Yes we are considering ECS and data oriented design as well high performance C#, as many other development teams working at Unity. I don't have details to share right now but we are definitely exploring opportunities in terms of performance improvements as well as being de facto compatibility with ECS when it's ready for prime time.
     
  16. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    336
    Awesome. Btw another problem of current UI system is the performance problem when using Animator. Is there any plan to create a special high performance UI Animator specifically for new upcoming UI system? I would like to have the power of Animator or even better than Animator to create high performance UI animation.
     
  17. benoitd_unity

    benoitd_unity

    Unity Technologies

    Joined:
    Jan 2, 2018
    Posts:
    57
    Performance and animation are the top priorities for runtime UI. To your point, we don't believe supporting the Animator should be the way to go in order to create simple UI animations, but something like a tweening system and CSS animations.

    Did you have something similar in mind?
     
  18. LurkingNinjaDev

    LurkingNinjaDev

    Joined:
    Jan 20, 2015
    Posts:
    1,698
    Slightly on topic (animation, performance, chromium, DOD):

    Please, try to overlook the click-bait title, it's not the point. Also, I'm sure Unity Staff does know it, probably even this presentation, it's mainly for the others. :)
     
    Vacummus likes this.
  19. Vacummus

    Vacummus

    Joined:
    Dec 18, 2013
    Posts:
    49
    What an incredible talk. Thank you for sharing this.
     
  20. pbrito_unity

    pbrito_unity

    Joined:
    Mar 30, 2018
    Posts:
    11
    But why?????

    CSS Animations are so limited, that is why (web)people came out with alternative like : CreateJS, GreenSock, D3.js, react.js

    You cant synchronize animations, etc ....
    Even W3C had to come out with Web Animations to deal with CSS animations limitations.

    Instead, Unity could create a visual editor for animation.
    And that editor would automatically create the Animator states, maybe with default states for press, drag, etc.
    With the plus that the underneath tech could change without the user notice anything different with the editor.
    Changing the CSS .....

    But CSS Animations??
    Who uses CSS animations?, how do you create a CSS animation?
    Unity already has tools (like the Animator) that can be explored for UI.

    Unity is not helping designers with CSS,
    and is making a bad decision with this XML/CSS route.

    Just my two cent.
    Best of luck guys.
     
    Last edited: Dec 6, 2018 at 11:22 AM
    Lars-Steenhoff likes this.
  21. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    1,730
    I like keyframe animations,
    tweeing libraries are good too.

    so the animator has defenitly its place, and states could be triggered via visual scripting.

    I have been doing this in unity since like 7 years ago, with the help of playmaker and a tweeing library, and animations in the old animation system before mechanim made everything more difficult.
     
  22. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    3,465
    The Animator doesn't do stuff at edit time, so I could see the value of having a unified system for UI animation when you want to support UI Elements in the player as well. Tweening/CSS animations are decent alternatives for that.
    That being said; the Animation window in particular is a very powerful tool, and if you're serious about letting non-programmers edit the look and feel of editor UI, asking them to code up css animations or apply tweening libraries is a much worse choice than letting them use a tool they know.

    Ideally, we would be able to target divs (by ID) or classes, and modify their values through the animation window, with support for scrubbing and all the fancy curves. In short, Unity's systems should support each other.


    Then again (again), an official tweening system would be interesting. Has there been any attempts to make a proper one in-house, or do you consider the wide array of open-source Unity tweening solutions as good enough that it wasn't needed? Are you considering one now? Would it be a bespoke UI Elements tweening thing, or a generalized one?
     
    Guerro323 and Lars-Steenhoff like this.
  23. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    1,730
    Agree

    Unity needs a native tweening library that goes beyond lerp.
    Unity systems need to be able to work with each other.
     
  24. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    336
    I notice @antoine-unity mentions about visual UI builder. Not sure it's for designed for Unity Editor UI only or real build UI but I hope I can have a unify visual UI builder that will support both Unity Editor UI and real build UI as well as UI debugger.

    For CSS animations, I didn't do any CSS before. Currently I'm using DOTween but it's still not really nice. It lacks a visual editor like Animator to preview and edit the whole UI panel animation and individual UI panel button easily. Every time you want to test UI animation, you need to go to play mode which is really bad. Another problem is if you want to do complex UI tweening, u will end up adding a lot of Do Tween Animation components to the game object that make it very hard to tweak it as you have to scroll it to find the component you want to tweak and you have no way to preview only one component of animations without disable all the remaining components. Then, I also have the problem with interactive UI such as Button that has different transition animation state (Normal, Highlighted, Pressed, Disabled). Now I recall that Timeline is the great example you can reference. You can bring most of the features of Timeline to the visual UI builder as a base and improve it.

    Now you have visual UI builder that not only can design the UI but also preview and edit the UI animation. Then you bring in the tweening feature. It should provide widely used tween animations and also able to create custom tween animation saves as file that u can keep reusing and potentially improve performance. The visual UI builder should able to do the following UI easily with high performance. One thing come in my mind is when doing this kind of UI Panel, the visual UI builder should inform the user when user trying to delete one of the element of the UI panel. If user choose to delete it, it should delete from the list of the visual UI builder. This is also one of the problem that Animator does not solve yet that will only show you in yellow text to tell u the game object is missing.


    Another hard problem is multi resolution support. I think the current UI Canvas does not solve it really well and u cannot solve it without creating multiple set of same UIs for different resolution. I hope you can improve this area too.

    Since now you have Burst compiler, I guess you can generate the most efficient UI code at final real build. So regardless of you are using visual UI builder or CSS animation, your final real build will have the perfect data layout and best possible UI performace.
     
  25. benoitd_unity

    benoitd_unity

    Unity Technologies

    Joined:
    Jan 2, 2018
    Posts:
    57
    Yes we've been working on an in-house tweening solution, I will verify but it should be usable outside of UI Elements.

    We're considering supporting tweening and CSS for the very simple and common use cases when animating UI: scale in/out, bounce effects, slide in/out, etc… and support current Unity animation curves for the more complex scenarios.

    We're not excluding any option, we just want proper solutions in respect of the 80/20 rule so you can do simple things easily and have more controls over advanced ones.
     
  26. benoitd_unity

    benoitd_unity

    Unity Technologies

    Joined:
    Jan 2, 2018
    Posts:
    57
    Yes this also something we want to improve. Having the ability to have different properties and assets, adaptive layouts and eventually vector graphics.
     
  27. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    1,730
    +1 for vector graphics.
    and it would be a good start if the SVG importer did not make sprites but created a mesh with dynamic tessellation when zooming in and out
     
  28. orb

    orb

    Joined:
    Nov 24, 2010
    Posts:
    2,944
    Yes. Vector all the things.
     
  29. Korindian

    Korindian

    Joined:
    Jun 25, 2013
    Posts:
    390
    Yes and antialiasing for UI vector graphics for deferred pipelines please!
     
    Lars-Steenhoff likes this.