Introduction:
In this case study, I delve into what the gaming company Valve Software [1] named "The Crate Depression of 2019", which brought chaos to their "Team Fortress 2" (TF2) video game and almost destroyed its in-game economy. This incident serves as a prime example of the challenges faced by video game developers addressing bugs that inadvertently impact the players experience and the company's revenue. Let's see in detail what happened and how it could have been prevented, what measures were taken by the TF2 team to rectify the situation, and how brilliantly Valve managed to satisfy all customers.
Background:
Team Fortress 2 is a first-person shooter multiplayer video game. Since 2011, it is a free-to-play game with in-app purchases. Players can customize their characters with cosmetics which can be obtained by buying them from the in-game store; combining hats and items together; opening crates; and trading with other players. Once crates are opened, they yield different items following different probabilities (these crates are what in the gaming industry are known as loot boxes). Crates can only be opened with a key which costs real money (2.5 USD). A crate once opened has a very low probability of giving an "Unusual" (i.e. rare) hat. These items are highly demanded and can be traded with or sold to other players in online markets, and as in any other economy, prices rise for scarce and high-demanded items. In particular, they can be sold at much higher prices compared with the price of a key, reaching even thousands of dollars. It is also worth mentioning that when items are sold in the Steam Community Market, Valve receives a commission.
On the evening of July 25th 2019, an unintended bug was introduced to TF2 allowing some crates to consistently yield Unusual hats upon opening. A few hours later, players discovered this behavior and spread the word. Thousands of people rushed to the game to obtain these bugged crates. As one could have expected, these "rare" items flooded the market and their price crashed. Besides the crisis in the in-game economy, in the long term Valve was going to loose money. Not only did players need to spend less real money to obtain these "rare" items, but the items themselves became essentially worthless, impacting Valve's commissions.
Bug Fix and Trade Lock:
Almost a day later, on July 26th of 2019, the TF2 team first disabled the unboxing of the crates and all transactions while the investigations were made [2], and a few hours later marked all Unusuals from the bugged crates as non-tradable [3]. This temporary measure allowed the team to assess the situation thoroughly and develop a comprehensive plan to address the varying degrees of impact on the TF2 community.
Balancing customer satisfaction while preserving the game's economy:
A week later, on August 2nd of 2019, the TF2 team released a statement explaining all measures which were going to be taken [4]. They recognized the need to strike a balance between preserving the experience of customers who unintentionally or intentionally acquired the bugged items and safeguarding the integrity of the in-game economy. To achieve this delicate equilibrium, the following strategies were implemented:
Tradable Unusual Hats: The first Unusual hat obtained by players during the bug incident was made tradable, ensuring that those who innocently or deliberately acquired a single item could freely use, trade, or keep it.
Permanent Trade Lock: All other Unusual hats procured as a result of the bug remained permanently trade locked to prevent market saturation.
Full Refund Option: To provide a fair resolution, players were given the choice of receiving a full refund for any number of trade-locked Unusual hats they possessed. This option allowed affected players to recoup their investment and alleviate any dissatisfaction caused by the incident.
These measures were beyond what most people expected. Throughout the week that the TF2 team was preparing their plan, many players speculated on the potential actions Valve might take, with some fearing potential repercussions for those who inadvertently or deliberately benefited from the bugged crates. I think the only group that might not feel OK with such measures would be those players that didn't benefit from this bug. These people could have received something, possibly a unique cosmetic. This is because there were definitely players that deliberately didn't want to take advantage of this bug (or simply were afraid of retaliation) and decided not to get the buggy crates, and they got nothing in return. Keep in mind that many players got hats considerably cheaper (i.e. a rare cosmetic paying only what a key costs which otherwise would have cost considerably more). Nevertheless, their measures are outstanding and managed to make almost everyone happy.
Another nice gesture from Valve was the decision to also apply the above measures to those players that deleted their Unusual hats obtained from bugged crates. They recovered all their Unusual hats.
Overall, the team claimed that the impact of giving away these tradable hats to be comparable to one month of the regular Unusual drops. In fact, Steam giving away goods likely fomented spending (since there were many players wanting to trade their tradable Unusual hats) and improved the economic activity.
It should be noted that the team also provided the necessary systems and support pages to facilitate refunds, item restoration, and provided clear guidelines for affected players.
Could it have been prevented?
The issue that the TF2 team suffered could have been a risk not identified, or perhaps was known but monitoring was not appropriate. Given the time it took them to identify the issue and prepare a plan moving forward, I risk (pun intended) to assume they didn't think about this scenario beforehand. Let's outline a few measures that could have prevented this crisis.
For starters, tests are crucial before pushing code to production. They likely had some sort of tests and even they might have tested the behavior of some crates. The fact that this affected a specific list of crates and in particular old ones, makes it a bit more difficult to catch during testing since tests are limited by nature. Nevertheless, this could have been identified before releasing the update. For example, a test could have been written to open all available crates several times, get a distribution of the frequency for each cosmetic and compare with a (previously defined) reference. If statistically significant differences were to be found, this would highlight that something was wrong.
Additionally, I wonder if they had metrics (and if these metrics were continuously monitored) related with their main goals which are, among others, maximizing the number of item sales, in particular keys, and the number of transactions. The number of keys being sold and the number of transactions skyrocketed during the time elapsed between the introduction of the bug and the temporary measures (arriving almost a day later). Besides measuring metrics using their own data, they could have also monitored hashtags in Twitter and TF2 references in other social media platforms. There were definitely more people than usual talking about the game and for a game released back in 2007 with only minor and sporadic updates being released, it can only mean one thing, something was terribly wrong with the game.
One important thing to consider is that Valve's works in a particular way, developers decide in which game to work, based on their preferences. A game as old as this one doesn't seem to motivate developers to actively contribute and the result is that there is hardly anyone working on TF2.
Conclusion:
Valve's decisions following the introduction of a critical bug in their Team Fortress 2 video game exemplify effective crisis management in the gaming industry. Through their measures of tradable hats, refund options, and item restoration, they prioritized customer satisfaction while also maintaining the overall health of the game's ecosystem. The incident serves as a valuable lesson for game developers, emphasizing the importance of bug mitigation and monitoring, proactive customer support, and continuous improvement to ensure a positive gaming experience for all players.
References:
[1] Valve Software is an American video game developer also owning a gaming distribution company called Steam.
Comentarios