Unity Unity MARS Companion app - Open Beta announcement

Unity MARS Companion app for mobile is now available in beta






The Unity MARS Companion app is available in beta for iOS and Android devices, and we’re looking for beta testers. Be one of the first to try out the new mobile app before it’s released.​

What is the Unity MARS Companion app?

The Unity MARS Companion app is the newest component of Unity MARS and is now available on iOS and Android devices.


With the app, you can capture real-world data directly on your device and bring it into the Unity Editor to quickly create and iterate on your AR experience to significantly decrease iteration time and deliver an AR experience that accurately runs in its target environment.


The Unity MARS Companion gives users the power to perform two primary tasks: data capture and in-situ authoring.

• Data capture: Using the companion app, you can capture room scans, take pictures, and record video with AR data. Once saved to the cloud, this data will sync directly to the Unity Editor, where you can open it using the Unity MARS authoring environment to create a simulated environment that mirrors where your AR experience is intended to run. Now, you’ll have a more true to reality development environment to author your AR content against.
  
  
• In-situ authoring: The companion app also has authoring functionality, that allows you to create content and layout assets directly on your device. And, for example, if a bug occurs because of lighting conditions or a particular room setup, it’s useful to have the actual device data to reproduce and fix the issue.

Both of these tasks involve sending and receiving data to and from the Editor, which is done via cloud storage. This means that users all over the world can work on the same project together, and that the data persists between sessions and is shared between users.


The Unity MARS Companion app is available to Unity MARS users at no additional cost to their existing subscription and includes 10GB of cloud storage per seat. Non-Unity MARS users can also take advantage of the app with limited functionality (see below).


Please note, the app is currently in beta. While this means that content captured or edited with the beta may not be compatible between versions during the beta or with the final release, we’re very interested in hearing your feedback to further refine and improve the product so it fits your needs.


What can I do with the Unity MARS Companion app?


Create and Layout Objects in a Scene



Scan surfaces and author or preview proxies.


Create Environment



Map out an environment by placing corners.


Record Data

Record videos, surface data, and camera paths.


Create Markers



Capture a marker and add hotspots.


Unity MARS user vs. Non-Unity MARS user

Based on your subscription type, you might have different levels of functionality within the app. In the chart below, we explain the differences in these functionalities.



*All changes that have not synced to the cloud will be stored locally and can be re-synced once the user authenticates with a Unity MARS entitlement. If the user logs in with a different username/Unity ID, they will be able to make local changes, but will not be able to sync those changes with projects they do not have access to.

**Unity MARS projects can be synced from the cloud via inputting Project keys or scanning a QR code.


Join the beta today and share your experience with us.


To try out the beta, open this link for iOS or Android* on your mobile device. The steps outlined in our documentation will help familiarize you with the workflows.
*By opting to download Android or iOS versions of the Unity MARS Companion App, you agree to our Privacy Statement


To use the Unity MARS Companion app alongside the Unity MARS authoring environment, we recommend you use Unity MARS 1.2, as well as Unity version 2019.3.0f6 or newer.


As you try out the Unity MARS Companion app, we’d really like to hear about your experience. For product support, troubleshooting problems, sharing projects and feedback, and general discussion about the app, comment in this forum thread or share your feedback in this survey.

 

> Installed it. Used it. Love it.

Thanks!

> the planes are waaaaaay too messy to be useful

Is this on iOS or Android? We are (for now) just exposing all of the planes the platform is providing, but I agree that it can be difficult to pick between planes that are overlapping. We will be working on future improvements to filter out overlapping planes and/or help you pick between them when they overlap. If you have any ideas or suggestions, please let us know!

 

> Probably a more helpful suggestion might be to allow the user to mark corners and then base / refine the detected planes from those?

Interesting suggestion. For the Environment and Data Recording flows, the intent is to let you capture exactly what the device is going to give you, so in a sense the "Lovecraftian overlappyness" is something that we actually want to capture in order to replicate in the Editor. That way, you can make sure your proxies and C# code can handle bad tracking gracefully. One way or another, if the platform is giving you bad results in the companion app, you can expect your users to encounter the same bad results "in the wild" when they run your app.

To be honest, it sounds like your empty apartment is creating a difficult environment for AR Core to give you consistent camera tracking and high-quality surface extraction. Furniture and clutter actually help with camera tracking, because the SLAM algorithm needs fiducial markers (recognizable details) to work properly. Do you get the same results in that space from a basic MARS or AR Foundation app? There could be some bug or performance issue in the MARS Companion App that we can fix to improve the results you are seeing, so it's good to know how other AR apps behave as a baseline. You may get better results on ARKit, if that's available to you. We also plan to improve the floor plan feature by letting you control where the base of the corner post sits on the screen. That should give you a bit more control over where it lands while also keeping more of the room in view.

For the Proxy Scan Flow, we do actually want to sanitize things a bit more so that you can do your work, even if you're not getting ideal results from the platform. However, we don't want to lead you into a situation where you are authoring against data that won't exist later on when your users run the experience. If we let you tag a surface in the Proxy Scan Flow, but there's no way to do that in a build of your app, you'll end up creating experiences that only work in the companion app, which isn't what we want.

With all of that said, you may be working on a location-based experience, or we may one day get more rich semantic information from the platforms (this already exists to some extent today) which means that you can make assumptions about the space beyond the raw data you get from plane scanning. We will be working on ways to leverage this type of information in future versions of this feature. Currently, you can edit the `Trait` property of a `SemanticTagCondition` which may match up to some semantic tag for data provided by Synthetic Objects or some future data provider that supports semantic tags.

A simple step toward what you are describing would be to just let you hide planes you don't want to interact with. But even that gets tricky, because we still need to allow your content to match to those planes, otherwise we're letting you "cheat" during authoring in a way that will still be an issue when users run your app. Then it might be confusing if you start editing the Plane Size condition on your proxy and it jumps over to a plane that doesn't exist.

We're actively exploring these kinds of improvements, and the platforms are always improving their algorithms. Stay tuned, and thanks again for the feedback!

 

> Can I assume that this is something undesirable that you folks are working to eliminate?

Yup! PlaneSizeCondition and ElevationCondition are a good start at coming up with "a good plane" but we want to go deeper. For the situation you're describing, we really need a "non-overlapping condition" but that's a trickier problem to solve efficiently on a mobile device. Putting content which you want to prevent from overlapping in a DistanceRelation (part of a Proxy Group) is a good way to prevent overlapping planes from turning into overlapping content. You might also see if Proxy Forces are a good solution to dealing with planes that don't behave as well as you'd hoped.

As always, we want to prioritize problems that our users are encountering in the real world, so thanks for the feedback!

 

I'm sorry to hear that you're having trouble. What device and Android OS version are you using? Where do you see the loading screen? Is it in Google Play? Is it the "Made with Unity" splash screen?

 

Hi there @glitchbeam! Sorry you're having trouble. Are you able to upload other types of resources? Is the project that you linked attached to an organization with at least one MARS license? You will not be able to upload to cloud storage if the project is not associated with a MARS entitlement.

In the short term, you may be able to work around this issue by unlinking the project, recording some videos, then re-linking it and trying to upload from the resource view. If you need to create a new project ID, you can always link the existing Companion App project to the new ID.

 

Thanks for the info. We are investigating the issue and will get back to you.

 

Hi there, @mariachiaramonti!

Sorry that you're having trouble with the app. How long are the videos that you are recording? Do you see a file size in the save dialog? If so, could you share the exact size of the video you are trying to upload? We have seen issues sometimes with extremely large video files. The issue that @glitchbeam was seeing had to do with our license check, which was resolved on the backend yesterday. If you are able to upload anything at all, then the license check is not the issue.

The app currently doesn't have the ability to resume or retry a failed upload, but something that you could try is to delete the recording in the Companion Resource Manager in the Editor, and re-uploading it from the device. If it fails _every time_ that you try to upload, this sounds like a bug that we need to fix. I think the only workaround is to record shorter videos. Please let us know if this is the case, and we will try to replicate the issue on our end and come up with a solution.

Thanks for your interest in the MARS Companion Apps and for getting in touch about this. Hopefully we can work out a solution for you!

 

Hi there! I think you are talking about viewing a data recording? If you have made a data recording in the app, you can download it in the Editor and view it in the simulation view. You need to change the mode to `Recorded` and then select the name of the recording in the dropdown. You can read more about this in the documentation.

 

Just so that anyone reading this thread is having similar issues with video upload, it seems like folks in Europe are hitting this issue due to their distance from the U.S. West Coast data center. We are working to resolve this issue. Deleting the resource in the editor and retrying the upload seems to work in some cases. More info here.