Building the Metaverse : Networking (Part 1)
“If a tree falls in a forest and no one is around to hear it, does it make a sound?”
Think of a large-scale video game world, like World of Warcraft. Would the game exist if no one played it? Well…the code behind the game would exist in a database somewhere. But it would only exist to that database, not to humans.
The same goes for the Metaverse. Let’s think of it as interconnected 3D virtual worlds rendered in real-time.
Once you create a virtual world, you want people in it. Creating the virtual world and creating the infrastructure that let people reliably access it are 2 different problems.
Here’s what we’re battling against.
A virtual world is a lot of data. And a lot of problems.
The BANDWIDTH problem.
How can we send so much data in real-time? (we need to constantly to update the environment in a real-time rendered world).
The LATENCY problem.
How can the data be sent fast enough on your device to not feel the time lag between your interactions with the environment and its update?
So, what are bandwidth and latency anyway?
Data doesn’t magically appear on your computer. It needs to travel long distances from data centers through optic fibers, copper wires and gigantic undersea cables to cross continents. Even though it’s very fast, it’s not instantaneous.
Latency is the time needed to receive data, in milliseconds (ms).
The amount of data that can be transferred at any given time through cables is not infinite either.
Bandwidth is the maximum amount of data that can be transferred per sec, generally in Mbps / sec.
The network connection between 2 data centers is like a bridge. Bits of data are like cars. If a 1000 cars can be on the bridge at any given moment, it’s the bandwidth. The latency is the time it takes a car to cross the bridge.
(Note : in practice, the max number of cars on the bridge won’t be reached. The actual number of cars on the bridge is the throughput, the REAL amount of data passing in a connection.)
BANDWIDTH
High bandwidth is necessary for 2 reasons.
- A lot of user data will be generated and will need to be shared to other participants in real-time.
To run a video game, you need to download a file which contains the game logic, textures, audio, etc. If it’s an offline game, you already have everything you need to play. Not if it’s online.
You can’t download how another player will move in advance. Whenever the other player moves, you need be sent that data to update you game and the person move. But we’re talking about 3D virtual worlds gathering a lot of people (called concurrent users). The data you will need to receive at any given moment to update this virtual world will be massive.
2. The virtual worlds will get too large to store them as files on normal devices.
The reality of today’s online games is that only a small fraction of the data is sent through servers to your device in real-time. Most of it is downloaded in advance, just like offline games.
“Fortnite’s PC and console game files are roughly 30 GB in size, but online play involves only 20–50 MB (or 0.02–0.05 GB) in downloaded data per hour.”
- Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything (p. 75)
This can’t work with big virtual 3D worlds, because consumer and even industrial devices won’t be able to store enough data. Microsoft Flight Simulator (MSFS), the most realistic simulation of the real world ever seen in a game, is around 2.5 pentabytes large, or 2,5 million gigabytes.
MSFS is only possible because most of the data is sent on a as-needed basis through servers, the same model required for the Metaverse. Then, why do most online game choose to do the opposite?
Because the internet is unreliable. Not everyone has access to top notch, high bandwidth, low latency internet. What will happen to the game if some players can’t load data quickly enough to update the game in real-time, or receive it too late? By limiting the data to be sent through servers by downloading most of it in advance, you lower the risk of a disaster, but at a cost.
LIMITATIONS
“[T]he fact that a server can only tell individual users which assets, textures, and models should be rendered means that every asset, texture, and model must be known and stored in advance. By sending rendering data on an as-needed basis, games can have a much greater diversity of virtual objects.
Microsoft Flight Simulator aspires for every town to not just differ from one another, but to exist as they do in real life. And it doesn’t want to store 100 types of clouds and then tell a device which cloud to render and with what coloring; rather, it wants to say exactly what that cloud should look like.
When a player sees their friend in Fortnite today, they can interact using only a limited set of pre-loaded animations (or “emotes”), such as a wave or a moonwalk. Many users, however, imagine a future where their live facial and body movements are re-created in a virtual world. To greet a friend, they won’t pick Wave 17 of the 20 waves pre-loaded onto their device, but will wave uniquely articulated fingers in a unique way. - Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything (p. 76)
2. Whenever content is added to the game, an update must be downloaded, which takes time and prevents the user from playing.
3. The need for downloading updates means the world isn’t “alive”. It’s just a temporary version that will be replaced by a new one on the next update, instead of one ever-evolving world.
LATENCY
Remember last time you watched a video where the lips of the person speaking weren’t in sync with their voice? It feels wrong, right? Well, what if the whole world you interacted with was like that? Reacting just a little bit too slow, each time. Not only would the experience not be immersive, it would be frustrating to many.
The human threshold for latency, the time between your action and the world reacting to it, is incredibly low in interactive experiences :
“In games such as Fortnite, Roblox, or Grand Theft Auto, avid gamers become frustrated after 50 ms of latency (most game publishers hope for 20 ms). Even casual gamers feel input delay, rather than their inexperience, are to blame at 110 ms.3 At 150 ms, games that require a quick response are simply unplayable.”
- Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything (p. 80).
The gaming industry discovered of few tricks to reduce the impact of latency.
By grouping users according to their geographical regions with their dedicated servers, you can play with people that aren’t too far from you, reducing the latency. This is why we have Europe, North American, or Southeast Asian servers.
But the internet is global. This strategy won’t work if you need to connect to people far away. You won’t be able to hang out with a family member on another continent, or explain ideas to a coworker in Amsterdam from your Seattle office.
It can take 35 ms to send data from the US Northeast to the US Southeast. It takes even longer to travel between continents. Median delivery times from the US Northeast to Northeast Asia are as much as 350 or 400 ms—and even longer from user to user (as much as 700 ms to 1 full second). Just imagine if FaceTime or Facebook didn’t work unless your friends or family were within 500 miles. Or they only worked when you were at home. If a company wants to tap into foreign or at-distance labor in the virtual world, it will need better than half-second delays. Every single additional user to a virtual world only compounds synchronization challenges.
-Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything (p. 83).
We can also use delay-based netcode to bridge the gap of latency between users. You add a delay so that the user with lower latency renders its data at the same time as the user with higher latency, levelling the playground. Rollback netcode instead predicts the inputs of the higher latency user. If the prediction is wrong, we rewind the animations and correct the results.
But this only works in predictable environments. It’s one thing to predict the choice between 4 moves in a fighting game, but complex facial expressions and human behaviour?
It means we need new solutions to reduce latency. Solutions good enough to compete with the speed of light.
Consider how difficult it is to send even a single byte from NYC to Tokyo or Mumbai at ultra low-latency levels. At a distance of 11,000–12,500 km, this commute takes light 40–45 ms. The physics of the universe only beat the target minimum for competitive video games by 10%–20%. […] in practice, we fall far short of this 40–45-ms benchmark. The average latency of a packet sent from Amazon’s northeastern US data center (which serves NYC) to its Southeast Asia Pacific (Mumbai and Tokyo) data center is 230 ms.
-Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything (p. 84).
Why is there such a big difference between the speed of light and our capabilities?
- The silica glass of optic fibers reflects light. The light travels through the fibers in a zigzag instead of a straight line, elongating the distance by 45%.
- The network of optic fibers that form the “internet backbone” is not optimized to reduce distance (and latency). It was not a carefully planned global project, but the aggregation of the smaller projects of private companies and governments. There isn’t always a straight connection between cities, creating long detours which increase distance and latency.
What can be done?
- Upgrade the cable infrastructure.
Problem : Very costly and highly regulated by governments. The more direct the connection is, the more likely it is to pass through protected property (residential, commercial, governmental or environmental), which makes it harder to be approved.
2. Upgrade wireless infrastructure to 5G.
Problem : Even though it can offer ultra-low latency, it doesn’t work on great distances, only few hundred meters. Then it goes through the usual cable infrastructure.
3. Satellite delivery (like Starlink.)
Problem : It’s low latency, but not ultra-low latency. “[It] enables more people to meet the minimum latency specifications for the Metaverse, rather than offers improvements for those who already meet it.” (p.87)
Moral of the story: our internet infrastructure currently doesn’t deliver the high bandwidth and ultra-low latency we need to turn the concept of Metaverse into a good product, and upgrading it will take time. It also means that companies like Meta that promise to deliver the Metaverse can’t do it alone, because they rely of national infrastructure.
*All references are from Matthew Ball’s book