An Optimization Tale
Hey there, Victor here.
For the past few updates we have slowed down the works on new content and prioritised the much needed Performance Optimisation.
But what does that mean? What are we actually doing?
I'd like to share a bit of what I learned with this project and discuss some of the optimisation techniques delivered in our recent updates.
There's a myriad of optimisation techniques out there, some of them are very well known due to some developers discussing them in interviews, some of them less known and discussed only within the developer’s groups/events.
But I personally like to split them in 2 big buckets: code and art.
Code is how things behave, how the world is built, how the information is handled over the network, and it's impacted by the quality of your code, by how often you run that code and by how the logic blocks of your game are set up.
Art is how things look, how the game is rendered, and it's impacted by the size of your 2D art/polygons of your 3D models, by the rendering pipeline used and by deciding what needs to be rendered and what can be hidden (aka culling).
There's obviously some grey area there, e.g. logic setups are mostly dependent on the 3D models (to account for movement and collisions) and culling algorithms can become expensive due to the time to run them.
What we already use
Because we use Unity we have some art optimisations right out of the box:
We don't render anything behind the player (frustum culling)
We don't render some of the objects behind walls (occlusion culling)
Players/Creatures don't animate when you're not looking at them, the models are not animated, but the bones that move colliders around still do
To help out rendering the game, Boramy and Serena created different Level of Details (LODs) for all of our objects, so we swap them as you move closer or further away, meaning an object up close has a much more detailed 3D model with a more complex rendering material, while the same object, when further away, has way less details.
Some of our big objects, such as the spire behind the Town hall tower, have what is popularly known as Impostor, where the object is rendered as a 2D object when viewed in the distance, this Impostor rotates around to always face the player’s view and shows a projection of the original 3D model, at close range it’s easy to tell the difference, but far away they are almost indistinguishable.
And since our servers are just ''virtual machines running in the cloud'' (Timo will kill me if he catches me calling his K8s setup like so) they don't render anything, in reality some of the stuff you see in game doesn't happen in the server at all, such as visual effects or sounds.
On the code side of things we run our network code on a different update rate than the regular game, so objects update their position and values at a slower rate than physics as an example, on top of that objects that are not changing are not networked at all.
We only load the world where we have players playing currently, reducing the time spent both running logic and rendering the world and also the amount of RAM used by the game.
What we’ve done in recent updates?
On this update I’ve introduced a maximum distance to render small objects (100m by default), the impact can be seen on metal ores, stair steps and dagger blades for example, this is only active on Players, not on spyglasses /cameras as to not prevent you from taking a beautiful shot, Joel also updated a lot on how we handle our networking messages, making use of the different worker threads the server CPU has available.
I’ve kept going on about distant objects and removed shadows from small-ish objects that are still being rendered, this includes most of the blades and handles in the game.
For this one Joel pushed an optimisation for how we build/render the trees you can cut down, by merging the 3D models used to render it he managed to make the rendering to be done on a single request to the GPU instead of the old 30-some requests.
Making this a Nature themed update I’ve updated the way we render our grass, by using the rendering tech built for the Particles lighting improvement on the previous update, this allowed not only to make the grass look better but to render faster
Side-by-side with these optimizations I took the distant objects optimisation up a notch: I simply don’t load them at all. Our game’s objects are divided between Dynamic and Static, AKA things you can interact with and things you can’t. Previously we would load the same area for both dynamic and static objects around each player, but as I was testing locally I’ve figured out that it’s actually hard to notice these dynamic objects as far as they were loaded, so I split these loading only about a quarter of the old area around each player, but I still load the area on your local machine, allowing you to still see the buildings and terrain in the distance, but saving both on rendering for you and on physics/logic for the server.
This last update brought a long awaited change: we’ve disabled Physics Auto-Sync.
Unity uses Nvidia PhysX as a physics engine, but it keeps 2 different worlds, one is inside Unity where we have our scripts and their hierarchy of Game Objects, and the second one is inside of PhysX where they keep colliders and run all of it’s calculations. Everytime we move a Game Object around in Unity without using physics it needs to be synced to PhysX, on the older versions of Unity we started ATT on, this was done automatically every time we queried the PhysX world which happen quite often (when you grab an item, when you move around, when creatures move around, etc…).
But on the current version of Unity we have the choice to Sync only when the Physics calculations are done plus when we want to and this could improve the performance of our servers a lot! But when we tried to quickly update this, we broke teleporting and some objects could be spotted flying around so we quickly rolled back (Hotfix 0.0.35.1).
After many tests of performance gains and changes to how we handle the Physics position of objects we are disabling Auto-Sync again, this shouldn’t have a big impact on clients, but on the servers where most of the physics queries are run, this can improve FPS by up to 33%!!!
To close this update I also included some changes to how we render the terrain, but here I made a choice to both optimise and improve quality which should be a net zero performance change.
The terrain is drawn “instanced” which means it sends the same 3D model to the GPU and the GPU update it to take the desired shape, this is faster because sending/retrieving information to/from the GPU is very slow.
Previously each part of the terrain had its own model sent to the GPU like this:
Now all parts send this model and the GPU modifies it:
Bits of terrain that is further away don’t need to be rendered with our slow, but beautiful, material and can be rendering using a very simple and fast material. Can you spot the difference?
Let me highlight the area changed.
I’ve increased the definition of the terrain when it’s far from view to reduce the “popping” effect that you get when walking around and see small mounds appear out of nowhere, this is a bit more expensive, but the cost is offset by the optimizations above.
Into the future
Phew, that was a lot of stuff.
Optimisation is hard work, because we need to balance performance itself with the quality of the experience and some of the techniques/algorithms are highly prized industry secrets, so we need to spend a long time researching and testing; as a technique that works for one game might not be applicable to ATT or might even perform worse!
On top of that some of the techniques can only be fully used later on, when we have more of the world set down and we can polish it properly, for example knowing everything that can and need to be seen from an advantage point allow us to hide more things/force things to be seen with a pre-defined quality, which allow us to save a lot of frames.
We have ATT’s performance in our minds every day and will be forever looking into ways to get the game to run faster and to look / feel better for you!
If you’d like to support us on this never-ending quest of performance optimisations head to our SUPPORTERS PAGE or click the ‘Become a Supporter’ button in the launcher. Thank you so much to those who already support us!
FOR THE TOWNSHIP!