Collaborative Editing in Unity3D
Over the last few day I’ve been considering different approaches to collaborative development in Unity3D. Unity is a powerful and versatile tool, but collaborative editing can be problematic. I’ve worked on several team projects using Unity, and the process has been pleasant overall, but it required us to make certain concessions. In particular we had to agree upon a way to handle scenes. The free edition of Unity limits asset serialization to binary mode, which means no merging of scenes. Unity Pro provides a preference to output to text (Edit > Project Settings > Editor Settings > Asset Serialization > Force Text), which makes merging feasible, but not easy. In a collaborative environment it necessary for multiple people to work on the same scenes. In previous projects, the solution was a combination of the following: working together on the same computer, working in a queue, or working on the same scene.
Working together on the same computer and working in a queue worked well in terms of maintaining sanity and keeping the scene files in order, but it wasn’t conducive of a collaborative workflow. Our teams managed to make this work most of the time, but at times it was necessary for multiple people to work on the same scene at the same time on different workstations. This was feasible using Unity’s Force Text mode, but any manipulations on the same object by a different user led to merge conflicts. While it’s entirely possible to merge these text based YAML files, they are often thousands of lines long and are more difficult to interpret than the standard editor. From time to time we used scene merging as a fallback when multiple people needed to edit the same scene at the same time. This worked relatively well, but required someone with enough knowledge about source control and diffing/merging (usually a programmer) to walk through and resolve the conflicts. While we were able to make due with the built-in system, it definitely hindered productivity.
At this point the goal is to create a consistent interface and controls to ensure that the developers on a team don’t encounter conflicts when working on scenes simultaneously. I’ve considered and prototyped several solutions. While there is potential for some of the solutions to work, not all of them would. The overall goal was to use/create a layering system that would ensure that different developers were all working on isolated parts of the scene, thus ensuring that no one has to deal with conflicts.
The initial ideas was to use the embedded layer system built-into Unity to subdivide the scene by different facets. Then each user could choose a layer and work in isolation from the work of others. When a user starts Unity a little dialogue (i.e.,
EditorWindow) could facilitate layer management. The user would choose a layer to work on and the remaining layers would be locked. Further, the selection of layers would be limited to those not already checked out by the source control solution (e.g., Perforce); it would be possible to create a metadata file for each layer, and then those files could be checked out by users of the VCS, thus locking the layers to other users. There are a couple of issues that make using the built-in layers less than ideal: for one, the process of automatically assigning new objects to a specific layer is quite extensive and two, the layers are often used for other purposes (e.g., occluding objects from the renderer or raycaster).
Let’s first address the process of keeping a user within the confines of a layer. By default, every time an object is created, the object’s layer property is set to Default, and would require the user to manually set the object to the correct layer. To simplify this, it would be nice if when an object is created in the hierarchy it’s automatically assigned to the current layer being used. To do this you need to know the objects that were created; unfortunately, Unity only provides one callback:
OnHierarchyChange, which only informs you of a change but not what the change consists of. This means there has to be some way to scan the hierarchy and differentiate between pre-existing and new items. This could be accomplished through different mechanisms, for example: if every object is assigned a specific layer, then it would be possible to search for any objects that are still on the Default layer. This is feasible (and I even made a working prototype of this), but it felt too heavy for something as simple as figuring out the last object created. Furthermore, there’s no simple way to control and lock layers in the built-in manager without devising a system that scans the hierarchy and looks at the layers currently assigned to the objects.
Since every GameObject in Unity can only have one tag and one layer, that means we need to conserve them for their intended purpose, like searching the scene by tag and occluding objects by layer. In addition, there’s only a limited number of custom layers that can be created. Based on that, it didn’t make sense to use the built-in layering manager as a solution to the collaborative editing problem.
After testing the built-in layering system, I begin thinking about creating a layer system that is completely isolated from the existing system. This system would use a parent GameObject for every every layer in the hierarchy. This system would consist of a dialogue for creating, loading, switching, and locking layers.
When starting Unity, a layer manager object is spawned into the scene and the dialogue is presented for choosing the layer to work in.
Any inactive layers are automatically locked using
gameObject.hideFlags |= HideFlags.NotEditable. This ensured that the user was kept to a strict contract that only allowed manipulation on the available layers. As with the built-in layering system, these layers could also be associated with metadata files, which can be checked out and locked in the VCS. As with the built-in solution, creating new objects would require us to find some form of identifier to distinguish, which object was created so that it can be added to the layer. Associating a GameObject with each layer is cleaner and allowed for more control. Based on my testing with the prototype of this solution, there didn’t seem to be any merge conflicts or issues, but if Unity’s scene file format changes, there could be potential for havoc. As of now, this seems like a reasonable solution to the problem, and it wouldn’t require much time to polish the implementation. That being said, I still seems like there’s more that could be done; so I continued to ponder other solutions, and tried prototyping a multi-scene file solution.
The ideal solution would be to isolate layers to individual data files that can be checked out and locked from a VCS. I’ve seen and looked at other engines that have this capability, and it would streamline the collaborative process in Unity. Unfortunately, it’s not as easy as just breaking the scene into multiple files and calling it a day. After searching the documentation for Unity, I found that the engine was capable of opening a scene and additively adding it to the current scene using:
OpenSceneAdditive. This is a reasonable way of consolidating the data contained in different scenes; however, it doesn’t allow the user to see the work on other layers (i.e., scenes) without importing the data for those scenes. This becomes problematic when one layer needs to reference another in order for the user to work on the layer. For example, the terrain layer is complete, but the building layer is just being started; placing buildings without the terrain to reference could be very challenging.
So I prototyped a solution to this problem which used a combination of adding and removing layer scenes from the active scene. There’s a layer manager script that would run on startup, which gathered information about what scene and what the current layer was. Once it had this information, it could determine which sister scene layers need to be additively added to the current scene. They would then be placed as a child to the layer manager GameObject, which allows for easy management and removal of external layers. The imported layers were recursively locked so the user doesn’t make modifications to scenes that are not currently checked out. Once modifications have been made, the user has three options: normal save, unload-save-load, and unload-save-commit-load. The normal save would save everything including the external layers to the current scene. The unload-save-load would unload the layers from the scene, save the scene, and then reload the layers into the scene. The unload-save-commit-load is the same as the previous, except the button can be associated with a commit in the current VCS system. The prototype is functional and demonstrates that this works, but the solution is inefficient. Layers with a significant number of objects take significantly longer to load than just having everything in a single scene. While multiple files are beneficial on the VCS side, this wouldn’t be an efficient solution.
As of now, I think a combination of using the GameObject based layer system in tandem with prefabs would be an improvement for collaboratively working on levels. The layering system would isolate the objects inside of each scene and the prefabs would isolate the objects independent of the scenes. It’s far from a perfect solution, but it alleviates some of the issues that I’ve encountered working on small to large teams using Unity3D. In the future, it might be interesting to try developing a collaborative networked solution. It would be possible to make an authoritative user that acts as the server and have a bunch of clients working that connect to the server. The users could then communicate changes to the server where they will be handled and merged into a single scene.
May 23, 2013