Randomness bugs - the Wi Flag

For data engineers, randomness is poison.

We strive for certainty and predictability, ensuring that the same process produces the same result, time and time again. We learn about concepts like idempotency to add consistency and reliability to the work we do. If a data pipeline has random outputs, we can't rely on it as part of a large complex system.

When it comes to games however, randomness is a feature. It's what creates excitement and unpredictability, giving players a feeling of anticipation. If players don't know what a treasure chest will contain, it's much more fun to open it and find out.

In 1999, a multiplayer RPG was released named Asheron's Call. For years, players would flood the forums with complaints about their bad luck. They claimed to be cursed, suffering from a deadly affliction that made monsters attack them first, ignoring all other targets. They called it the "Wi Flag".

"For some players, the flag came and went. For others, it was a perpetual nightmare, present in nearly every monster experience. To live a Wi-Flagged life meant to be hunted at every turn. Perhaps other adventurers could know peace in a BSD or a Citadel, but there was no rest nor respite for one under Wi."

- Asheron's Call Letter to the Players, July 2002

The developers denied the existence of the bug for years. But eventually, one developer took a deeper dive into one the fundamental game systems of Asheron's Call - monster targeting.

When a monster needed to find a new target, a simple algorithm was used: a weighted random roll.

"Generally, a creature chooses whom to attack based on who it was last attacking, who attacked it last, or who caused it damage last. When players first enter the creature's detection radius, however, none of these things are useful yet, so the creature chooses a target randomly, weighted by distance. Players within the creature's detection sphere are weighted by how close they are to the creature -- the closer you are, the more chance you have to be selected to be attacked."

- Asheron's Call Letter to the Players, July 2002

There are a few ways to implement a weighted random roll. In Asheron's Call it worked like this.

Start with a number line, from 0 to 1. A random number generator (RNG) can produce a random number along this line, where any point is equally likely to occur.

Then, segment the number line into many parts, one part for each player. The size of each segment represents the weight - how likely each player is to become a target for the monster.

To pick a target, simply roll a random number and check which segment it lands on. In this example, the random number landed in Player B's segment, making them the monster's focus of attack.

But the developers of Asheron's Call made a mistake. They intended for each player's weight to be determined by their distance to the monster, with the closest players having the highest chance of becoming a target. However, there was a bug in the conversion from distance to weight.

The player weights didn't add up to 1, as they should have, but instead summed to N-1, where N was the total number of players within range of the monster. This meant that the sum of weights extended beyond the range of the roll produced by the RNG.

The bug wouldn't have biased the monster towards any particular player if it weren't for another detail: the order that player weights were checked. When a player's character was created, it was assigned with an instanceID, a unique character identifier. That identifier was used to sort players in order. If you were unlucky, your ID would put you first in line to have your weight compared to the random roll. That meant a much higher chance to become monster lunch.

The Wi Flag was real. And it cursed players for years before it was eventually found.

"So what does this mean? The way this random targeting algorithm is implemented right now, if you happen to have an InstanceID that hashes to an early position, you will tend to be attacked more than your fair share when the creature is using random targeting, regardless of your distance from the creature. In other words, you are Wi-Flagged."

- Asheron's Call Letter to the Players, July 2002

Data engineers often don't want or care about randomness. But there's one exception: A/B testing. A critical step in online experimentation is random assignment, a phase where each member of the experiment population is given a different treatment, or experimental variant.

I have found hundreds of bugs in A/B tests. If you've never build a game before, randomness can be a blind spot. If you work on experimentation code, there's a few key ways to prevent bugs with randomness:

• Write simulation tests. Run your randomization code thousands of times and verify that the distribution of outcomes matches your expectations, within some tolerance.
• Use a random seed. A seed controls the output of an RNG, making it deterministic. Use seeds to create reproducible tests.
• Hash with a salt. When hashing a user identifier for randomization, pick a salt that can change over time - like an experiment ID or a static salt that you rotate every quarter.
• Start with a library. Python's random.choices method lets you supply weights so you don't have to roll your own weighted random. Don't roll your own if you can avoid it.

There's a delicate balance between randomness and predictability. Between ETL and experimentation, data engineers have to find that balance. While walking that line, just make sure you haven't cursed your players along the way.