How do several people work inside the same scene and resolve merging conflicts under Git

Heya everyone!

The title basically says it all. I have a large team of developers (5-10 people) who have to work on the same scene/scenes. However, when several people accomplish their tasks, we often have merge conflicts resulting in loosing some of the work when we try to merge them correctly. We know about text serialization of the scenes and we properly configured the project to integrate it in our workflow. However, merging scenes is really a nightmare. Even with such plugins as UniMerge.

I am aware of the fact that you actually can create as many prefabs as you want and lately construct the scene out of them (maybe dynamically), however it doesn't solve our problems completely, since sometimes you have to change the instances of the prefab based on the functionality in concrete scene and pass few object references around. Additionally we would like to have our scene active so that artists can construct the level right in the scene file and store it there.

So yeah, I would like to hear from people how do they manage large projects in Unity, assets merging, prefabs/scenes merging etc.

Btw, we use Git if this info is relevant for answering my question.

Thanks in advance.

 

What about Advanced Additive Scenes? It's not exactly what you're asking about, but it's the best solution I've found so far.

 

Thank you for the tip. Can I use this tool with some kind of evaluation period? My managers won't buy anything that is not going to solve our problems completely for the whole team. Also, if I understand correctly, this tool basically divides the scene into several scenes. Do they still retain connections between them? Some object references etc.

Also, I would really like to hear from experienced project managers with large teams and projects how do they actually organize their Unity workflow so that everyone can work in parallel. I couldn't find any decent information across the web. I would really appreciate if you could point me in some directions.

 

Never hurts to ask. Here's the contact page for the developer.

Valentin Simonov write a blog post on this. I only mention it because I came across it while searching futilely for Schell Games' excellent Unite 2011 presentation on Scalable Game Development. (Does anyone know why it was apparently scoured from the web after 2012?) Their presentation had some great tips for large Unity projects. You can download the slides here at least: http://unite.schellgames.com/

 

I would like to know how others handle this too, we use Git, and right now levels have to be owned by one person. Sort of like 'checking out' a level. Anytime we've worked on the same level its a nightmare and the merge fails.

One strategy we started using is a basic Game setup scene, with main character in place, and the Game Manager. This scene is owned by the programmer and starts up the game. It then loads additively a Level Design scene, which is owned by the artist and level designer. That way the artist can work on level design, while the programmer tweaks the interactions.

 

Yeah it's a hassle.

You might want to look at Asset Server if you need to merge scenes, from what I understand it has good support for it.

 

We just don't use scenes. Everything is handled in separate projects, and then just bundled for load at runtime. Since the everything is just code (on the eng side), merging is simple.

 

Sure, with no scenes, merges would be easy! But, how do you design levels? We need to test and tweak the placement of objects in the scene to get the right level of difficulty. And, visually, we are constantly making adjustments.

 

You can just use separate projects or scenes to build your content and add as a prefab or as an assetbundle. Or optimally, store you "scene" (level) in a data structure, then you have your staging project/scene that can run through the scene and store the info needed to reconstruct it. An addition benefit of this is that since you are parsing the scene, you can also optimize on fly as needed. (batching meshing/materials, etc).

We have around 7 projects, models/fx/environments/ui/etc... and one main project. The main project has virtually no content at all, just one scene with a single script called "engine". Everything else is loaded as/when needed. The other projects are where content is assembled, the content is then exported as assetbundles. Our team is about 20-30, and we can all work without running into (asset) conflicts. If we are working in the same area, or on something that will span more than a few hours, we create a local branch in GIT. For level building, the designers have a tool that lets them set up a scene and tune however they want, then the whole thing is stored as .json data.

There are other ways to do it as well. But basically scenes get very clunky and problematic as the project or team grows. They can be pretty fragile and difficult to repair if you rely on connecting things via editor/public vars. (and have you discovered, have multiple hands on it).

 

Thanks for the link. I already saw it before posting on this forum. Unfortunately I haven't found the answers for my questions. Asset Server maybe the good alternative, but we are working with free Unity version, so this is not an option for us right now.

I consulted with some of my friends who happen to work on Unity games and they told me that they actually don't have any scenes too. They store everything in prefabs and later they load everything from those prefabs and place it on the scene according to configuration files, created by tools provided to level designers.

Thanks for pointing this out. I think we should try to employ this method. I hate public object references, since it is just destroying the purpose of the prefabs, because you cannot change them freely. However, this approach is harder than simply working with Unity and saving scenes. You basically have to create your own scene serializer, don't you? With all of the pitfalls such as properly serialized collections of components and so on. At least in our case, since we don't have Pro version of Unity, so we cannot separate projects and build them into Asset Bundles.

 

Yup, it is more work for sure. Unity is like Flash in that respect. For simple/small stuff you knock stuff together fairly quickly and with little effort. But as you scale up (project size or staffing), the quick and simple stuff doesn't work anymore. Even with asset bundles, there are challenges, as they cannot include any code, you still have to serialize some elements.

 

Proper structuring of project and good communication IMO. That way you can isolate areas people work on while others work on other ones so there's no conflicts and talk with others when you may be messing with an area someone else is working on. You should know who's working on the AI, who's handling the user discovery and who's handling the proxy servers message forwarders etc.

You don't always get the luxury of the former(adopted projects) but communication is a life saver in every situation. Optimally, a project would have numerous folders with subfolders intuitively named that also help split responsibilities(Models/Structures is separate to Prefabs/Structures). Same with code, you don't dump everything into subfolders in your project, you want separate .dlls which also has the benefit of enforcing modular and reusable code.

I get problems with merging conflicts, I hate them with a passion and it's usually settled within a couple minutes by talking with the person responsible. You can't have everyone chaotically changing everything, you need to learn to work together or not at all.



I haven't combined projects like people here have, but I have had a project where I coded a custom serializer for levels and the builds were "stubs" that contained nothing more than the .dlls required and an entry point level. We'd export scenes from other projects with the serializer and these would be loaded in at runtime and most unique code was handled with Playmaker.

 

Here is the article, summarizing my thoughts about this pov http://www.intelligent-artifice.com/2013/05/unity3d-for-teams-of-6-people-or-more.html
You can't yell out load "I'm locking this!" every time you want to change the scene, component or prefab. And you actually want to work on those in parallel anyway, because there is no ideal isolated modules, belonging to the developer, which other developers can't touch or shouldn't rely on. What if someone is developing some kind of system you rely on, often changing its interfaces and public object references? Yelling every time you use this system and want to modify it to do your job is not productive at all. Also, when you try to merge YAML scenes, it doesn't really matter who made the changes, you can't do the merge properly when different scenes have different references, ids, whatever after they have been converted to text. Some plugins do that, but not particularly good (maybe you can point me into the right direction on how to do that?). You can even loose some references to images after you think you completed the merge successfuly, which is quite sad situation. Resolving scene merge conflicts often results in loosing the work of your coworker or self.

 

I'm dealing with this stuff as we speak and we don't have these issues because we split responsibilities. When our responsibilities collide in a single class, we usually settle it by communication. I don't see how this is complicated, all you have to do is be organized and have a proper structure to everything. The only time it's hard is at the beginning stages of development when a ton of code will be made and changes will be everywhere, scenes are split up into prefabs and typically two people won't be working on the same one at the same time..if they do, then the other guy goes out for a coffee or works on something else.

There's no "I'm looking at this!" every time we are changing something. People know what they're responsible for. On a very large team this would be harder, but on a very large team you usually have someone dedicated to managing everyone(and changing things willy nilly isn't appropriate). 6 or so people, not a problem in my experience.

It's not acceptable to change whatever you want and merge later. Communication is important and you need to take time out of your day to do it. If you're not willing to do it, you will have problems. It's that simple.

Ideally it would all be brought in at run time from prefabs on separate scenes dedicated to them. Right now, the project I'm on, the only thing that's there is the terrain and a few other objects(which I'm trying to get rid of), all new work is in separate scene files that contain things like the Inventory UI or the Chat UI etc. No one should be touching the same scene or code and like I've said, if in the bad situation we have to merge(which hardly happens except for the beginning stages of a project) then we do it. Scene merges are rare and should never occur, they typically involve someone working on something else for a bit or taking a coffee break.

This is just the way it is, there's no perfect solution to it right now sadly. The best solution is to be organized. If you can't do that or think that isn't an option, you may be not suited for working with others.

Does it work with scenes too? If so, that would be awesome. How much will it be?

 

Me too, because there's only two approaches I can think of that'd work for scaling up: those based on splitting up the work as per zombiegorilla's description, and those based on genuine multi-user environments. Either is going to incur a significant cost somewhere, and that cost is why large games are so much more complex (and expensive) to make compared to small ones. 10 times more content doesn't mean 10 times more work. It probably means closer to 20 times more work, because you not only have to manage the work but also the systems that manage the work...

 

Unity had a blog post where they demo additive scenes, which might end up in a future 5.x version. It seems to be modeled after UE4's sublevel system, which is a great way to subdivide scene work among people.

In the meantime, we use Advanced Additive Scenes as well which provides close to the same functionality except for the cross-scene linkages.

For the OP, there are no direct linkages between subscenes (one scene can be made up of multiple subscenes in this system), but we have a system for fixing up linkages at runtime.

Much more convenient than our old system of treating prefabs as levels, though it does have its quirks when running under Unity 5.

 

We just abandoned the idea of prefabs as level designs because it incurred a huge cost in scene loading. It appeared that Unity had to assign all prefabs unique IDs, including all the scene elements even if the prefab was not instantiayed.

I supspected a large studio might have a custom scene serializer that sets up the levels via code. I assume that means all the prefab placements, linkages and properties are stored in a data file. Then, they must have a tool to visualize this and test it. That sounds almost exactly like, say, a 'scene'! Seems like this is something the standard scene should already be able to do.

 

I don't have any. Our previous project was made with Flash and we merged everything together pretty much fine, because there were no "internal id's" system in the first place. Everything were explicit, thus clear to everyone. However not only it was a little bit easier than our current project, we don't have a lot of Unity experience in the team. That is why I am asking for help. We have already tried several approaches and it turned out they were wrong.

Maybe you are right. We are in the middle of the starting stage that is why we have so many conflicts - we still don't have defined and set in stone responsibilities for each part of the code and design. The core systems in our project are still changing a lot, that is why we have a lot of merge conflicts. However I didn't mean that we have code merge conflicts. We have to merge components, game objects, prefabs and the scene itself (this is especially hard when internal id's in two different projects are not the same).

It is true that we didn't separate everything in the scene. For example, our main scene contains everything - from giant game object with lots of managers to the level representation and GUI system. I know, this is wierd, but it was our idea about Unity-way of doing things. We are now trying to switch to more compelling strategy, that is why I am asking in the first place =) I also wonder what do you do in case of refactoring if your responsibilities are set within the specific field of the project code. Also, I really wonder if "Scene for everything" ideology works well. How do you organize links between them? How do you load the scene with GUI into everything else? How do you tweak the same components in parallel?

 

Nope, we didn't have any. All of the assets were stored in SWF files with static assets divided into "asset bundles" for dynamic loading during gameplay progress. And of course we didn't store our level in those SWF files. The levels were stored in separate JSON-format files, loaded during the runtime from the server. This data was then processed by the client and we would spawn the objects, according to the level files. Since SWF consisted only of reusable data (images and animations, really), we didn't have any conflicts.

That is why when we first encounted Unity, its workflow was very uncomfortable for us with all those public references hardcoded and flying around.

 

Exactly how we handle it (many of us have a flash background). Throwing on the stage and hard-wiring is awesome for prototyping and quick testing, but like flash, doesn't scale. But also like flash, having the option to develop effectively multiple ways is fantastic.

 

Thanks guys for all of your tips. I think I'll try to incorporate our old Flash workflow into our current workflow. The amount of community support on this forum is amazing =)

 

I got a question. So whenever I fetch and pull, not only does none of the other guy's stuff get added to my scene, but some materials get removed from things in my scene and they become pink. Funny thing is, they didn't even change the material in their project, so it could not have been from a conflict. I have to video chat them and try to reconstruct the changes they made instead of actually importing their scene. The assets they add appear in the project folder, but the in-scene changes they made do not transfer to my scene. How can I avoid this? Github shows no merge errors. I am using Unity 5.6.

 

Maybe it's a meta file issue?
http://answers.unity3d.com/questions/932348/visible-or-hidden-meta-files-with-git.html

 

I've had problems with meta files and GUIDs when merging before.

Basically we had an artist that would drop textures into the project without opening Unity and importing them. Then multiple people would pull the same asset down in Unity. It would import with different GUIDs.

A merge conflict in the meta files would remove one of the GUIDs. Since Unity inspector references happen by GUID, this would break every reference to that particular asset in one devs project.

We ended up with a blanket rule that an asset had to be imported before committing. It was a bit of a pain for the artist, because our original workflow plan had involved them never opening Unity.

 

Yeah, same here. On the last big Unity project I worked on, we saw the exact same issue as @jsalame36. I'd make changes and commit. Then, someone on my team would pull latest and some of the materials were pink. I had either forgotten to commit meta files or their was a meta file conflict on their end.

 

Yeah, that won't work! Two different people who open Unity pull the same asset from someone who doesn't - BAM two meta files with two GUIDs and the world falls apart.

You can fix that if you know what you're doing - git grep for the "wrong" GUID and do a text replace in those files - but that's a hassle.