For game developers, A/B testing is part of the standard process for deploying new game features. With a well designed A/B test, developers can get insights about the impact of said feature, how players respond, and if the feature takes the game in the intended direction. However, practically speaking, A/B testing leaves a lot of money on the table because it fundamentally assumes that there is a “best” variant. And, if there is a “best” variant, it must consequentially be applied to the entire user base.
This industry-wide problem is particularly apparent in testing offers and promotions in games. Testing whether the “Thanksgiving Sale” at $9.99 is better than the “Black Friday Sale” for $4.99 using traditional A/B testing methods does not accomplish the goal of maximizing revenue for the developer. First, the A/B test very likely won't reach statistical significance for at least 6 weeks (making the test completely irrelevant). Second, while conducting the test, revenue is not being maximized, costing the developer thousands of dollars by offering poor converting variants. Third, the test assumes that there is a “best” offer to show all players in the game regardless of spending and playing behavior.
The Multi-Armed Bandits (MAB) framework solves these three shortcomings by not assuming that there is a best variant and reaching optimal performance faster than traditional A/B tests. Here is why Gondola migrated to MAB:
Learn vs. Earn
What is often forgotten is the opportunity cost of A/B tests. While an A/B test is running, there is a cost associated with offering less optimal variants as often as the best variants. MAB, on the other hand, continuously switches between the learning phase (used to learn how different variants perform) and earning phase where optimal variants are used to maximize performance.
More Than One Solution
Anyone that has tried to run an A/B test to compare similar (but different) offers associated with an in-game event knows that the results are not binary. Though while planning an experiment it was idealized that players would all gravitate towards one of two variants, the reality is that the difference in conversion between the variants is less than 5%. There are a hundred reasons why this is the case. The most common being that two large player segments with very different preferences for variants cancel each other out in an A/B testing scenario (in mobile games the textbook example is how Android and iOS users show different preferences for offers). The fact of the matter is, there are two good variants, and it is unreasonable to expect the game’s Product Manager (PM) to choose just one. This is where MAB comes in because it never phases out a variant completely. Contextual MAB, which considers the player’s profile, can learn to channel different variants to different players.
Instead of treating every player the same way, contextual MAB attempts to make the best decision regarding learn vs. earn and which variant to present each player based on their attributes/behaviors. Player attributes can include session count, lifetime spending, player progress in the game, previously purchased offers, country, and pretty much any analytics data the game has been recording.
The conversion funnel that occurs in the first 10 minutes of gameplay is thoroughly optimized at launch. However, games are “living creatures" that are continuously growing with new levels, additional game modes, and in-game content for purchase. Externally, the game market is also evolving with new games competing for the same players, User Acquisition (UA) sources coming and going, and the fluctuating quality of players acquired. How optimal are the results from the original A/B test a year down the road? Yes, the game might still be converting players, but chances are that the funnel is no longer optimal.
With MAB, there is never a point when the algorithm determines the absolute best variant in an optimization. This characteristic is a big difference from A/B testing, where there is a point at which “statistical significance” is reached, and the PM can choose the variant with the most favorable outcome with certainty (not a chance occurrence). While optimizing a feature or offer with MAB, there is some traffic allocated to “double check” if non-optimal variants are performing better. This process is captured by the concept of regret: the difference between the total potential reward (ex. revenue) if the optimal variant was selected and the reward accrued from the actual choices made thus far in the optimization (Figure 1). Since MAB is always trying to minimize regret, it naturally limits the amount of traffic allocated to learning.
The game industry is at the forefront of game optimization. The reason A/B testing is widespread is that it offers the ability to reduce the risk of releasing game features and offers while at the same time delivering assurance that the results are not by chance. However, A/B testing is hard to execute due to the volatile player base, low conversion rates (often increasing the minimum number of players to impractical levels), and a long time to reach statistical significance. Today, MAB benefits from the decades of research and algorithms that deliver faster times to optimal performance, do not require the assumption that there is one best variant for all players, and continuously reevaluate to ensure the right variant is used at all times.
In virtually every online industry, product managers (PMs) rely heavily on multivariate testing for purposes of product development and marketing campaigns. A/B testing, in particular, has had a long run as the go-to solution for improving live products that are concurrently used by millions of users, including free-to-play (F2P) games. The reason is that the law of large numbers (LLN) guarantees stable long-term results for the averages of some random events. The more trials performed, the more stable and reliable the test results become.
Given the above, a F2P game with millions of monthly average users (MAU) may appear to be the perfect use case for A/B testing. However, A/B testing can be problematic - detrimental even - when used to optimize revenue in F2P games. There are several reasons why.
The first and probably most important difference about F2P games is that, as opposed to pretty much any other product, only a tiny fraction of the user base pays. In other industries, customers pay a specific dollar amount on a pay-per-use (food delivery, ride services) or recurring monthly basis (Internet service, gym membership). In F2P gaming, it is entirely common for 99% of the user base to not make a single in-app purchase (IAP) or microtransaction and therefore spend zero dollars in their lifetime The remaining 1% generate all the revenue. So, in a game with one million MAU, you only have about 10,000 paying players on a monthly basis. Within that group of 10,000 spenders, there is probably a dramatic difference in monthly spending. 90% of the paying players likely spend anywhere between $1 and $10, while the remaining 10% (0.1% of player base, or 1,000 players) spend anywhere between $10 and $10,000 per month. The curve is exponential, and spenders with monthly budgets of $500-plus are very rare.
When it comes to A/B testing, this means the following: For anything that is not directly monetization-related, such as content updates, new game modes, or additional characters, the product manager can rely on a vast user base to take full advantage of LLN and can expect reliable results in reasonable amounts of time. However, the majority of tests related to sustaining or increasing revenue rely on the behavior of paying players. This factor changes the sample size profoundly. Now the pool of suitable players shrinks down to about 10,000 spenders per month, even less if only recurring spenders or whales are the subject of the test. This small segment of players is divided further to create at least two groups (A and B), a control group and (at least) one test group.
To generate statistically significant results from an A/B test with only two groups requires an extended amount of testing time in a non-changing environment (3 or more player groups require even more time/players). Modern F2P games rely heavily on live operations, events, bi-weekly tournaments, and one-off special content sales. Anywhere between 30-50% of revenue from IAPs and microtransactions come from limited time offers. Monthly A/B test cycles are therefore not suitable for a “Christmas Special,” a “Tournament Deal,” or any other offer/promotion that is available to players for 7 days or less. That means that the testable fraction of players who spend is further reduced, putting the whole concept of “micro-segmentation” in any F2P monetization context into question. The fact that, by definition, A/B tests expose at least one group to a sub-optimal experience, generates even more time pressure to conclude the test as fast as possible.
Additionally, F2P as an industry is a constantly unstable environment. Rarely ever will the gameplay experience for a specific game remain unchanged for an entire month. Moreover, the competitive landscape and other games in the same genre are subject to constant change. Player spending behavior in your game is impacted if your main competitor releases a new multiplayer mode, adds newly licensed cars to his lineup, or runs a major sales event. It becomes even more extreme if an entirely new game or sequel enters the marketplace.
In conclusion, testing of monetization features in F2P games can only lead to reliable results if the test has a) a sufficient amount of players, b) a stable environment and c) enough time to reach statistical significance. Most monetization A/B tests fail at least one of these requirements, often multiple. Still, many critical design and business decisions are made based on A/B test setups that are doomed to be unreliable from the get-go.
There is however an alternative to A/B testing. That alternative is Multi-Armed Bandit testing (MAB). At Gondola, we're turning to MAB for our In-Game Offer and Video Ad Optimization Platform. In our next article, we'll tell you why.
If you are the proud owner of a game that features a game economy (virtual economy), try to consider your players as employees who play your game as their job.
Learning on the job
Like newbies on the job in real life, they will start off with a low level of skill and corresponding low expectations regarding their monetary compensation, i.e. they will have low expectations regarding the amount of virtual currency they earn in return for their "work." As time passes, they will get better at what they do (playing your game). This doesn't only mean that they will explore your game more, including the possibility to convert themselves into paying users (via in-app purchases, or IAP). It also means that their expectations regarding their (in-game) compensation for the work they put in (playing the game) rises simultaneously.
More Experience + More Skill = More Salary
Like in a real economy, the formula "More Experience + More Skill = More Salary" should apply here as well. The more time a player spends on playing a game, the further he dives into a game, the more money he expects to earn. At the same time, if you pay players too much, their motivation will suffer. And while only very few players could ever pinpoint that their frustration with a game was caused by insufficient compensation, a good analytics tool will undoubtedly show how increased player churn "coincides" with plateauing or even decreasing organic earning capabilities. Eventually, like in a real economy, it's all about balance.
Virtual Currency Shortage = Frustration
On the one hand, a shortage of virtual currency inevitably leads to frustration. Players feel that they are wasting their time, that their efforts remain unnoticed and that their improving skills are not appreciated. They feel underpaid, possibly even exploited. Sometimes you hear that "the virtual currency shortage is intentional so users start buying IAPs." Fair enough, but this strategy (of essentially implementing a paywall) only works if you have a highly addicting game. And even if your game possesses these sticky qualities, it will be off to a pretty bad start if you start choking your players for money right from the beginning, way before they even have a chance to get addicted to your game.
Virtual Currency Abundance = Boredom
On the other hand, an abundance of virtual currency will lead to spoilt users. Overcompensation means they can afford a wide array of virtual goods way too early in their (playing) career. They will be under the impression of playing a game that's not challenging, that doesn't reward superior skills, where everything feels numb, too easy, the same. Like in real life, a spoilt user's motivation to put in extra effort to receive additional rewards will suffer, possibly to the point where any kind of "work" (playing) will be subject to rejection. People get bored, and that's the killer for any kind of game.
Whether you are dealing with a frustrated or a bored player, both will inevitably quit playing your game, and they will do so very quickly if the game they are playing was free in the first place. We all know that the cost of customer acquisition is a multiple of the cost of customer retention. Generally, mobile marketing costs are still rising. So once a player is gone, all the work that was put in on the developer's side to onboard him in the first place was futile. Yet, this kind of churn is completely unnecessary and fairly easy to correct, even in a live game. Changing the number of available coins, rings, nuts, emeralds, or whatever your virtual currency happens to be, is a whole lot easier than changing game mechanics, storyline, graphics or other content.
5 Rules on How To Pay Your Players
First and foremost, balance is the key. Always. Abundance is bad, shortage is even worse. Your game, and especially your game's economy, is a highly specialized environment. Take an honest look and ask yourself if it is balanced in the way you envisioned it to be. Are your players truly rewarded and therefore happy? Are they starving? Or are they spoiled?
2. Growing Bankroll
Always ensure an increasing bankroll. While this sounds trivial, game developers often miscalculate the earnings capacity and/or the "game maintenance cost" (ammunition, energy, fuel) the player is confronted with right away. The result is that players barely scrape by after restocking for the next mission and are therefore unable to accumulate any wealth. They feel notoriously "broke" and all those shiny swords and shields in your virtual store will seem completely out of reach.
3. More Money Per Level
Always ensure an increasing earnings potential. Once again, it seems obvious that players should be able to earn more virtual currency when playing level 21 than they were able to earn while playing level 3. Yet, this is often not the case. Carefully measure your players income while he/she progresses through your game. Ask yourself how much virtual currency the player should have a certain points in the game, for example after 2 minutes, at the end of level 3, at the end of World One, once they reach the rank of "Lieutenant" etc. Then ask yourself if the virtual currency available to the average player (or whichever group you want to target) at this point in the game matches the prices (i.e. availability) of the virtual goods you want the player to use/have access to at that time. For example, if you wanted your players to approach the end boss in level 3 with the shotgun, then make sure at least 75% of your players can organically afford that shotgun before level 3.
4. Consider Difficulty
What is also often miscalculated is how increasing difficulty or the lack thereof can affect the virtual currency earning capabilities of players. If a player gets stuck in a certain level for several attempts, his earnings over time will decrease dramatically because he never gets to earn the full amount of virtual currency the level provides (depending on the setup of the game, he might even earn nothing at all if the level is not completed).
Similarly, if a level is very easy and/or rich in virtual currency, a player might replay that level countless time, thereby causing the opposite effect. Usually it's very difficult for game developers to change the amount of virtual currency available in certain levels, mainly because the result will be that all other levels will be out of sync and will therefore need to be adjusted as well. You should therefore consider tweaking the other cornerstones of your economy.
Example: If you determine that players earn 10,000 coins during the first 5 minutes of gameplay, but your first 5 virtual goods are priced in the range of 1,000 to 5,000, you could consider doubling or even tripling the prices of the virtual goods, thereby reestablishing the virtual currency/price ratio you originally intended. Definitely make sure to still have something of great value available after 2 minutes or one round of play. Getting players to buy something in your store very early on is crucial.
5. Gimme More!
When in doubt, always throw in more virtual currency for the player to earn. Collecting 100 coins is always more fun than collecting 10. There is just something very satisfying about flying through a field of coins and listening to the sound of an increasing bankroll. If you don't know what I mean, play a round of Jetpack Joyride or ask yourself why the guys from Orca created Pot of Legend.
A couple of weeks ago, the book "Console Wars: Sega, Nintendo, and the Battle that Defined a Generation" by Blake J. Harris was released. And while the review of The New York Times was somewhat mixed, it inspired me to take a look at my very own video game history, one that actually includes multiple gaming systems from both Nintendo and Sega.
Early Beginnings: Commodore 64
Born in Germany in 1979, I am a child of the 80s, and a teenager of the 90s. My first true gaming experiences came at the age of 9 when my cousin received a Commodore 64 for his birthday. We played The Last Ninja 2, California Games, Archon, Giana Sisters, Bubble Bobble and Manic Mansion for what feels like thousands of hours. I still consider the Last Ninja 2 to be one of the best games ever made, and the music is second to none.
Sega Game Gear
My first "gaming property" was a Sega Game Gear that I received for Christmas in 1991. My favorite games were The G.G. Shinobi, Sonic the Hedgehog, and an Afterburner-clone called G-Loc, although I never completely fell in love with any of them.
I had opted for a handheld because I didn't own a television, and I had chosen Sega's Game Gear instead of the more popular and much cheaper Game Boy from Nintendo because it was newer and featured a color display. I later regretted my choice, mainly because the Game Gear was so clumsy that it essentially required a backpack to be carried around, and also because the backlit screen drained 6 AA batteries in less than 90 minutes. Compared to the Game Boy that was very portable and only required 4 AA batteries (8 hours lifetime), the Game Gear just didn't qualify as a true handheld. The biggest problem was that I only knew one other person with a Game Gear to borrow games from, as opposed to at least 10 Game Boy owners in the neighborhood.
A little more than a year later, in the spring of 1993, I sold the Game Gear via a 200-character classified in a local newspaper (pre-Internet, very old school). The money allowed me to buy a Super Nintendo Entertainment System (SNES). To this day, I can remember how scared I was taking a bus to the city with all that cash in my pocket. I still didn't have a television in my room, so I had to use my parent's television and could therefore not play as much as I wanted. Yet, the move away from the Game Gear was still totally worth it. My first games were Streetfighter 2 and Super Mario Kart. Both were absolutely brilliant, especially when played in VS Mode against friends. They probably remained my favorite games throughout my SNES career. Other great games included Jimmy Connors Pro Tennis Tour, Parodius, Super Bomberman (unparalleled multiplayer fun) and Super Mario Allstars.
I still recall reading an article in "Video Games" about the first PlayStation in early 1994. The console had the capabilities of a contemporary arcade racing machine, and the racing game Ridge Racer by Namco had been ported 1:1 - at least that's what the excited journalist reported from a gaming convention in Japan. The fact that the PS1 had a CD-Rom and memory cards to save game progress and highscores made it even more desirable. So I sold the SNES and pre-ordered a PlayStation. It was released in Germany on 9/16/1994 (I actually didn't have to look up the date when I wrote this, I still remember!). My first game had to be Ridge Racer. It was super fast and had great driving physics, especially with the special analogue NeGcon controller that I got for it. Other milestones were Tekken and Resident Evil. I got so hooked on both that I couldn't wait for the PAL versions of the sequels Tekken 2 and Resident Evil 2 to be released. So in order to get my hands on them as early as possible, I bought very expensive imported Japanese versions. Since I had a PAL PlayStation they would not run unless I used a little ballpen spring to exercise the CD-swap trick (essentially fooling the console into believing it was loading a domestic PAL game). Pretty crazy by today's standards. Twisted Metal, the original Need for Speed and Micro Machines V3 (best multiplayer game ever) completed my list of beloved PlayStation games.
Game Boy Pocket
Towards the end of it's lifetime, Nintendo started releasing derivatives of the original Game Boy. The first one was the Game Boy Pocket, a smaller, lighter, 2-AA battery version of the original Game Boy. Starting in late 1996, it was offered for a very reasonable price. So I finally got a Game Boy because there were tons of great games that I could borrow or buy from friends that had lost interest. Maybe it was a little redemption for the Game Gear experience 6 years earlier... In any event, the Game Boy Pocket was truly portable, and I predominantly used it during the late stages of highschool to entertain myself in class (Shakespeare was just not my thing back then). My favorite games were Donkey Kong and Tetris (duh).
In 1998, I managed to afford another console without selling the previous one for the first time, partly because I opted for a used Nintendo 64 that already came with a couple of games. So I kept the PlayStation and the Game Boy Pocket, and got a N64 on top of that. Nintendo had released the N64 two years after the PlayStation, and although I remember that the system was graphically superior to the PlayStation, it never quite managed to catch up in popularity. Part of the reason was that the cartridge-based games tended to be more expensive than the CDs for the PlayStation, and I remember that variety and depth of the available games were not as great. But there were some All-Star-Quality classics that I will never forget. Super Mario 64 and GoldenEye 007 (awesome postmortem here) are among the best games ever made. 1080° Snowboarding was also awesome.
Sega Dreamcast (and the end of my story)
The last console I ever bought was the Sega Dreamcast. It came out in early 1999 and was the first 128-bit or "Sixth Generation" console in the market. The audio-visual advantages it had compared to everything else that was available at the time (predominantly PlayStation and Nintendo 64) were huge, and I had never seen these kind of graphics before. Everybody knew that the PlayStation 2 would be even better (I had perceived Sega as doomed ever since the Sega Saturn was such a failure compared to the PlayStation in the mid 90s). At the same time, everybody also knew that the PS2 wouldn't be released in Europe for another 18 months, so the Dreamcast was the way to go. In many ways, using it felt like a super polished PlayStation 1 experience. The graphics were stunning, and Soul Calibur might still be my favorite fighting game of all time. The other games I truly loved were Resident Evil Code Veronica, Ready 2 Rumble, NBA 2K, NHL 2K, Tony Hawk's Pro Skater 2, Virtua Tennis 1 & 2 and Ferrari 355 Challenge. I guess the fact that my focus had shifted to sports, fighting and racing games reflects my advanced age at this point. One totally crazy thing I got was Sega Bass Fishing with a virtual fishing rod. Even today, my friends still make fun of me for that purchase.
My console career more or less ended when I started law school in the fall of 2000, and I finally "donated" my Dreamcast to my nephews in 2004. The Dreamcast was the last console I ever bought. The reason is not that I don't like gaming anymore. It's actually quite the opposite: With the right console and the right game, you will find me playing in front of a TV at 4:30 am on a random Tuesday, with no end in sight. Ever since Apple's AppStore opened in July 2008, my new passion have been mobile games. I actually decided to quit law in 2011 and focus on mobile games as a career.
So what decided the Console Wars?
What it comes down to is content. Always. Every game console's lifeline is the game content that is available for it. Aside from drastic price disparity, the winning console in every generation (handheld, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit) was always the one that had the highest density of great games. Great games attracted people to the platform which in turn allowed you to more easily find friends that you could trade games with. One needs to know that in the 90s, a SNES or PlayStation game would set you back at least $40, possibly $50 or more. Keep in mind that those are the actual prices from 20 years ago and consider how much that would be in today's money. The result was that in those pre-online days, hardly anybody owned more than 2-3 games when they first got a console, and if you were a broke teenager that number would only rise in homeopathic increments every year (birthday and Christmas). Essentially all teenagers therefore engaged in "trading" games with friends ("I borrow your Super Mario cartridge, you can have my Street Fighter cartridge for a couple of weeks"), except for maybe the very spoilt/wealthy or very non-social people. In short: If you wanted to have access to a variety of content (games), you better knew a couple of other guys that had the same console that you had. And whether other people would prefer one platform over another was predominantly due to the quality of the game content available for that platform.
In the mobile world, this is obviously pretty different. The price difference between a cheap Android and an expensive iPhone 5S is not 20%, but rather 500%. Many people don't buy a smartphone with gaming in mind whatsoever. And most content is developed for both iOS and Android, and oftentimes simultaneously released. If you consider the $40+ price tags of the old days, the financial commitment for each game today is also a fraction of what it was. The most expensive mobile game I can spontaneously think of is MineCraft at $6.99, and that seems "outrageous" in light of all the free and $0.99 games out there. But don't get me wrong: MineCraft is brilliant, and totally worth $7. So go get it!
Many game developers measure the financial success of their applications by looking at the total revenue generated on a daily basis. Total revenue, however, tells an incomplete story.
The "total revenue" trap
Consider the following situation: the overall revenue of App X is $100 on Monday. On Tuesday, that number increases to $200. Sounds like we are making a killing, doesn't it? Well, the real answer is "maybe."
In order to judge the financial efficacy of a game (or any app for that matter), the developer needs to take into account more than just total (= aggregate) revenue. Equally important is the total number of users that are responsible for generating that revenue. Going from $100 in revenue to $200 in one day is indeed great if the user base of App X stayed (roughly) the same. But what if the revenue doubled from $100 and $200, while the user base tripled? Would you still consider that a success? We might not be looking at a success-story anymore, but rather at an indication that something has gone terribly wrong. Although one might argue that increasing total revenue is the ultimate goal and hence what is described here is a success story, such growth is not *sustainable*. If the revenue per user keeps decreasing, then the game will likely cease generating any profits in the near future. Especially freemium games often fool their developers in just this way.
The solution: LTV
How can you differentiate real success from impending disaster? Don't look at revenue alone, look at user Lifetime-Value (LTV) as well. Calculating and comparing LTVs is the only way to truly measure the financial performance of your game.
The LTV of a game user is the sum of all revenues that will be collected from that user, minus the costs associated with providing the game to that user. Said revenues and costs are accounted for over the entire lifetime of the user, which starts with the game installation, and ends when the user stops playing the game. Compared to total revenue, LTV provides more accurate information in two dimensions: First, it measures the profitability of users, which helps developers identify more lucrative customer segments. Second, it takes future profits into account, and hence is more farsighted.
The reason to calculate the LTV of your existing players is to predict the net profit one can expect from the entire future relationship with a new player. Of course, different players earn revenues and generate costs in different ways, giving rise to the multitude of variants of the notion of LTV that are used by developers in practice, each involving their own set of calculations.
Once a developer has access to comprehensive analytics, calculating a user's LTV is actually not difficult at all. It is however very important to ensure that only LTVs that were calculated the same way are compared as performance "benchmarks."
5 popular LTV calculation methods:
In order to determine which formula suits what kind of game, we recently reached out to some of our fellow game developers and asked them: "how do you calculate your LTVs?" Below are our favorite answers. Note that CPI (cost-per-install) is subtracted in a consistent way across all the LTV calculation methods, and therefore not specifically mentioned every single time. Note also that none of the LTV calculation methods claim to be flawless or generally applicable. They just serve as a rough overview of how actual game developers do approach LTVs these days:
Some game developers calculate their LTVs by only taking verified in-app purchase (IAP) revenue into account, thereby ignoring any social contributions (e.g. a user is very vocal about the greatness of a game on Facebook) or advertising revenue. In short: calculate the total verified IAP revenue per active user. Done.
The "receipt" approach makes sense for games that are focused on monetizing virtual economies, because developers want to determine how much money they actually make on virtual currency IAPs (user conversion within the virtual economy). Simultaneously, they have a strong interest in not having their IAP data spoiled by ads or even piracy (corrupted/fake IAP), which is why they are relying on the IAP receipts.
"Weekly cohort" LTV
Other game developers segment their users into 52 cohorts per year, one for each week. Starting from there, they track the week during which a user first installed the game, and then calculate the LTV for each cohort (weekly group) of users separately. For example, users that joined one year ago are grouped together in the 52-week LTV cohort, while users that joined a month ago will be compared in the 4-week LTV cohort. In other words: weekly cohort LTV can be described as receipt LTV controlled for the acquisition time (i.e., the time user starts playing the game). So someone who first downloaded and played a game 4 weeks ago but monetizes today (by making an in-app purchase) will count toward the cohort from 4 weeks ago.
The "weekly cohort" approach makes a lot of sense for games with higher content depth that motivates their users to return for extended period of times (real time strategy games, adventures, puzzle games etc.).
Game developers in this group only measure LTVs for a specific (rather short) user lifetime, and then simply apply a pre-determined multiplier to the cohort (time frame) they want to examine. The multiplier LTV method thereby computes observed value (rather than life-time value) and uses it as a proxy for computing life-time value. This is essentially a variant of the "weekly cohort" approach, except that the time frame the game developer takes into consideration for the LTV is very limited, thereby ignoring all purchases that actually occur after the pre-determined cut-off time.
This approach allows developers of games with very short core gameplay loops to start comparing user LTVs very early during a user's lifetime. The high amount of repetition (tens or possibly even hundreds of rounds of gameplay per day) naturally allows them to predict LTVs much earlier than for example in story-based games that really develop (and therefore monetize) over time. Use cases predominantly include casino games where the length of each round of gameplay is not particularly dependent on the user's increasing skill, but rather on chance.
Multiplying the average revenue per daily active user (ARPDAU) with the average user retention (ARPDAU x Average Retention) leads to the "average" LTV approach. The first part of the formula (ARPDAU) is a fixed dollar amount, while the latter part is percentage of players that are still active after the first day of installing the game. Average LTV computes an average revenue metric, adjusted with the most recent information on retention. Like total revenue, this metric computes an overall figure for the entire user base.
To be fair, the average retention of all users might not necessarily be a good estimate of a particular user's retention, which is why we are somewhat skeptical regarding this calculation method.
If given a set stream of receipts from a player, every game developer will choose to monetize a player as early as possible during the player's lifetime. One reason is that only money that has actually been collected can be reinvested back into a business. The "weighted" LTV method discounts later purchases in order to emphasize the preference for "early money". Game developers follow the maxim "A bird in the hand is worth two in the bush" and modify their way of calculating the LTV by putting a multiplicative discount factor that diminishes over time in front of all receipts.
Example: Let's imagine the cohort of interest is "all players who started playing between January 17th and January 23rd." The game developer now needs to calculate the sum of discounted revenues for every player in this cohort. That is, he needs to multiply payments received by a discount factor that decreases with the the number of days between the player's acquisition and payment dates. A player who joined on January 17th and paid $1 on January 18th is therefore generally worth more than a player who joined on January 17th and paid us $1 on January 31st.
The difference is then averaged across all players in that cohort. The result is the average "weighted" LTV for that cohort.
While the "weighted" LTV method initially seems more complicated, it is the only one that applies more weight to players that monetize early during their lifetime. The true challenge is to determine the discount rate. There are many options. Some developers pick an arbitrary rule like "$1 now is worth 2% more to me than $1 one month in the future." This is really dependent on their cash flow needs. A more scientific approach would be to have an accountant calculate a capital depreciation rate. Some developers even match the discount rate to the weighted interest rate on their company's debt. Developers usually look at several different discount rates when analyzing weighted LTV.
Games with high maintenance/cash flow needs like MMOs are prime candidates for the "weighted" LTV approach. The same applies to games that seek to increase their user spending by way of aggressive (and therefore expensive) marketing campaigns.
Short of completely ignoring user LTVs, there really is no right or wrong way to approach the whole business of LTV. The correct calculation method depends on the type of game the developer wants to analyze, as well as the specific circumstances he or she is operating under. We have heard that some developers adjust their LTV calculations depending on the time of the year (July generates different results than Christmas-fueled December). Others switch the LTV calculation to a different model because they need to picture their game(s) in a slightly more favorable way to prepare for a board or investor meeting.
The cardinal rule is this: always make sure to only compare LTVs that were calculated using the same method, regardless of whether you are seeking to understand the financial development of one game, trying to compare across multiple games, or engaging in comparisons that involve aggregate industry data ("benchmarks"). Otherwise, you are truly comparing apples to oranges, and you might think that your game is doing increasingly well when the opposite is actually true.
With the incredible success of the freemium monetization model in the mobile space, Game Economies (or more accurately: Virtual Economies) have become the go-to monetization method for the entire industry. Even the conservative console market is starting to use the freemium model in some games with Sony reporting promising numbers relating to PlayStation 3 and 4.
At the same time, freemium has become a new evil, creating the so-called "paywalls" which are essentially obstacles that prevent users from progressing further into the game without paying money. These paywalls are as appreciated as the Berlin Wall once was. Players are complaining left and right about being taken advantage of, and in some circles the concept of "free-to-play" is considered to be nothing short of fraudulent. All of a sudden, people are happy to pay for apps, and the formerly hated banner advertising is "tolerable," while IAPs (in-app purchases) are the devil's children.
Is Freemium good or evil?
The truth lies somewhere in the middle. Sure, there are examples of games that try to milk players for every last cent. EA recently attracted an outrage when it released a new version of the beloved classic Dungeon Keeper. For TouchArcade to give a classic like that a 1-star review, something clearly went very wrong. Another article actually stated that "Dungeon Keeper isn't the first example of a business model completely destroying any semblance of gameplay or fairness from a mobile title. and I'm pretty certain it won't be the last." EA had tried to milk its players, and they paid dearly for it.
But there are also countless examples of carefully crafted freemium games that enrich all of our lives with awesome playing experiences that are truly free to play. And while the right design of a freemium game and especially its money-generating virtual economy largely depends on the game's genre and specifics, there are some commonalities that seem to differentiate the hated games from the loved ones.
If a game's virtual economy is almost exclusively used to make money (as opposed to generate depth and fun), chances are high that this will be noticed by its players fairly quickly. People hate the feeling of being taken advantage of, and they will object if it persists. From a game developer's perspective, it might seem unfair for players to be so unforgiving in light of the fact that they downloaded and played a great game that cost $100K (or $1M) to produce completely free of charge. But this is just an app developer's life.
The best way to ensure being trashed for "exploitation" is the excessive use of paywalls. They come in countless forms. Unsophisticated paywall games try to monetize their players right away, thereby accomplishing nothing short of driving them away before they even had a chance to get into the game. Clever paywall games, on the other hand, cash in on the same principle as a 2-hour TV movie that doesn't bother the viewer with ads for the first 30 minutes (the point in time when you are "invested" in the story) and then hammers him with commercial breaks every 8 minutes until the very end. Similarly, games manage to "lure" a user in and then try to monetize as aggressively as possible once the user has reached a certain point of engagement. They then put 95% of their content behind the paywall and choke the player to a complete standstill unless he pays repeatedly. Only really engaging games can get away with this. We all know of certain match-3 puzzles that have even made our moms pay for IAPs...
In some free-to-play games, players who are willing to pay for special powers, weapons or other features gain a significant advantage over those playing for free. This "pay-to-win" concept can be considered a paywall variation. It becomes particularly relevant in scenarios where players compete with one another directly (multiplayer games) or indirectly (leaderboards, highscores). Some argue that players are not forced to spend in order to progress, while others say that the fact that sheer game skill can be completely outweighed by financial commitments will make true fans abandon a game, thereby hurting both its reputation as well as profits in the long run.
Another variation that has recently become very popular is the so-called "energy systems." They inhibit the player by only allowing for a certain number of continuous rounds of play before requiring "recharging." Example: think of a racing game where the player's 8 gasoline units in the gas tank represent 8 rounds of gameplay. Once empty, the energy (tank) recharges itself through passage of time. Players can avoid that detour by spending money to either recharge (and oftentimes simultaneously enlarge) the tank or eliminate the restriction altogether.
Energy systems have however also been the subject of frequent complaints, mainly because they seem to be completely artificial instruments of restrictions in most games that are only geared towards monetizing the player. For that reason, some people actually call them "soft paywalls."
Selling an enhanced experience
Virtual economies can also serve different purposes, and it is equally sad and unfair that many games that do not deserve the "exploitation" label are thrown into the same category just because they are free to begin with, feature a virtual economy and offer some sort of IAP.
The biggest difference is this: games in this group predominantly view game economies as a tool of generating fun and content depth. The virtual economy aids the gameplay, rather than obstructing it. Parker's classic board game "Monopoly" is a great example for an entire game that is solely based on and around the concept of a virtual economy.
By the same token, video games employing virtual economies have been around for at least three decades. Do you remember the items Link could buy in the little store in "The Legend of Zelda" on the NES in the 80s? Yep, 100% virtual economy, even back then! At the time, there were no App Stores, no IAPs and not even the Internet. Neither Nintendo nor anybody else could capitalize on their virtual economies. So why did game developers implement virtual economies in the first place?
The answer is simple: to give the user a better experience, to make their games longer lasting and more engaging, or simply put, to make their games better! We have all played games where we couldn't wait to get the new gun to finally take on that endboss, where we played all night to make enough coins so that we could afford the new engine, or the armor or the magic potion the next day. Virtual economies motivate, they give us things to aspire to. They are therefore an awesome tool for making better games.
So how do we create great freemium games that keep players happy? By allowing them to organically make sufficient virtual money by playing the game to sustain themselves in the virtual economy. And how can we monetize them simultaneously? By offering them the various opportunities to either enhance their playing experience or accelerate their game process by virtue of spending actual money. Buying an upgraded weapon, a pack of time boosters, a coin doubler or unlocking a new world, all these things, if done right, enhance (improve) the playing experience and provide true rewards to the player. And players who feel rewarded don't feel exploited.
For the vast majority of freemium games, a paywall concept is not suitable. More often than not game developers end up not only facing enraged users, but also harming their very own financials by trying to monetize players too early or too aggressively (or both), thereby simply driving them away before they ever spent any money.
If you have a truly sticky game and want to make the most money you possibly can, there is no way around paywalls. Despite all the complaints, paywalls are not per se wrong. We have heard producers in major game companies say that "you are not monetizing aggressively enough if you have anything above a 2.5 star average review." Others say that if you do not get the occasional curse from your players, you are not engaging them heavily enough. It all depends on the developer's individual perspective and goals. And, if implemented into the right game, there can be no doubt that paywalls are responsible for many of the Top Grossing games out there today.
But you can also monetize your users without exploiting them. Your game economy will be appreciated if it offers an enhanced experience or accelerated progress for money, not the ability to progress at all. Always allow players to move through the game without having to pay money if they are willing to spend ample time. Yes, you might not monetize as well as with a more rigorous approach and leave some money on the table. But you might have also done yourself a great favor and monetized your game perfectly by withstanding the temptation to overdo it. Either way, your game will probably be much more liked, and you might make up for the decrease in initial monetization by increased user retention and therefore increased LTV (user lifetime value). Moreover, there is strong chance of improved user acquisition courtesy of great reviews. After all, these are the factors that are largely responsible for how much money you make at the end of the day. A game is kind of living thing, and it gets better the more carefully all competing interests are balanced.