The limits to AB Testing

Last week was about all the ways to test a game – broadly broken into quantitative, qualitative and technical testing.

  • The technical testing should test that the game is not buggy.
  • The qualitative testing is using small numbers (below 100) of people that you follow very closely or even meet in-person.
  • The quantitative testing is where you just treat people as statistics, and go for large volumes (1000 or above).

As I wrote last week, all of these are needed as they give unique viewpoints into how the game works. This week, I’m going to have a deeper look at the quantitative testing part, where we do A/B tests on large groups of people. This approach has its limits, but it can be stretched surprisingly far, if you want to.

Here’s how I visualise the problem myself: if you put features on the x- and-y axis of a graph, and the commercial potential on the z-axis, you will get a landscape like this. (Of course, it’s way more complex than what we can actually draw with a 3D landscape like this, but for illustration it works).

Say you are convinced that combining an endless runner with gacha mechanics is the best idea ever. This means that you envision there to be a large, unconquered mountain at the intersection of these two features.

chart_IIIYou then make a first version of the game, and drop it on some unsuspecting test players. From there on, you iterate on your game, always trying to improve it slightly by running a series of A/B tests.

In these tests, you have (at least) two different versions of the game: A and B. Randomly, you assign either version to a group of players. The easiest way is to advertise to get a group of, say, 1000 players. You then assign 500 of them to play version A, and 500 to play version B. A simple way to do it is to generate a User ID number, and say that even IDs get version A and odd IDs get version B. From there, there are a ton of more sophisticated stuff you can do, but this is the bare bones version.

Now you will look at which group played for longer, or spent more money. After watching them play for some time, you will have more information to feed into the development team to make the next version, with the next A/B test.

For those of you who studied optimisation, this might feel familiar. It becomes close to a version of the steepest ascent algorithm. This means that you might be able to optimise your way to the nearest hill (local optima), but you will not be able to jump to a nearby even higher hill, since that would require you to go downhill a short distance before you start climbing up again. It’s like a blind person climbing a hill. He will get up on the hill, but cannot see the higher hills nearby.

Steepest

So, how well can this approach work in real life, and what are the limits? Some of the wildest things we have done are fairly bold tests. On an upcoming level-based puzzle game, we were unsure about the difficulty and how fast we should introduce new concepts. To find an answer, we tried out 3 versions: the original one that felt a bit slow for us, another that had dropped every second level (and thus introduced concepts twice as quickly), and a third one that had dropped out 2 of 3 levels (introducing new concepts three times faster). In this particular test, the original fared best, closely followed by the double speed version. The triple speed version was way worse – apparently confusing people with too much info.

We had another fairly ambitious test when developing Benji Bananas. One of our role models for the game was Tiny Wings, another Jetpack Joyride. If you have played the two of these games, you will know that while both are endless runners, they have different way to end the game. Jetpack Joyride will kill you as soon as you make a mistake, and force you to start over. Tiny Wings, in contrast, will forgive your mistake, and let you continue. It will just cost you a few seconds of time, and eventually time will run out, which ends the game.

So, which one should we adopt for our cute swinging monkey? Should we be forgiving or should we be harsh when the player makes a mistake? We thought we knew the answer, but wanted to test anyway. Actually, I have since asked whole roomfuls of game professionals which way they think we should go, and about 2-to-1 favour our own intuition.

JetpackVsTinyWe were fairly sure we should be forgiving and go the Tiny Wings route. After all, it’s a very cute and casual game. When we tested it, however, the Jetpack Joyride inspired instant-death version won out.

We did not believe our data, we were sure we must have made a mistake somehow. So, we improved both versions (and especially the timed/Tiny Wings version) and ran another test. With the same result.

We still did not believe it, and polished up things a third time, only to get the same result a third time. After a few months of wasted effort, we finally accepted the data and moved on.

After 9 months of tests like this, we finally had a working game. Some measures had improved by a lot over the course of testing and iterating. For instance, measured by how many players completed 100 games or more during a few weeks after downloading, we had improved from 0.5% when we started testing to 20.5% in a version before launch. That’s about 40X improvement – so much for A/B testing only being about small improvements!

 

A few more hints if you too decide to do some wild testing:

One concern is that you might tarnish your brand or the game brand when doing this. There’s an easy solution for it. You simply invent another name that you use during the testing period. If you are really concerned, you can have another company account as well. And if you are super concerned, you can switch out some key graphics so that no one can recognize your famous IP. With these measures, you get to test things out completely anonymously. The game will speak for itself and no positive or negative brand associations will tarnish your data. If the ideas your testing turn out to be bad, you can just quietly kill things with no bad PR as a result.

I would, however, suggest that you do not involve money in these early and risky tests. As long as you are giving away free entertainment that no one can in any way pay for, I think it is fair to run a few tests and see how people react. When you have paying customers later on, you need to be way more risk averse.

 

Until next week!

Torulf

Testing and Iterating

I have a great idea: let’s throw away half of everything we do!

This is about the process of testing and iterating on a game until it works, or you decide that it will never work well enough. There are a huge number of ways to test a game, all with their own weaknesses and strengths. You should likely use a combination of several of these. The constant iteration and testing means that you will design and implement a lot of things that you end up throwing out. It’s frustrating, but it works.

Testing out a game will usually begin with a small number of other game experts discussing the high level drafts. At this point, I am convinced that you should already be talking to others about the idea. It is more likely that you lose money because you made the wrong product, than it is likely that you lose money because someone heard about your idea, copied it and stole your market. Just talking to others in the industry might very well help you make a much stronger concept to start with.

Once you have some first prototype, you can start testing it out on friends, family and other unfortunate people you happen to meet. At this stage, they can give you some general pointers about how interesting they find the concept, and help you roughly figure out who might be the target audience and who definitely is not. Just remember that is is very, very rough at this stage. Do not assume that your friends are in any way a representative sample of your customer base.

When we are a little further along, we have often been testing games in the lobby of our nearby university. Of course, the sample of people is again clearly skewed, but we can catch early UI misses this way.

We take an smartphone or tablet loaded up with our latest game version in one hand, and our own smartphones in our other hand. Then we stop a random person in the lobby, and ask them if they would like to help us out by playing our games for a minute. We hand them the smartphone with the game, and record a video of their fingers (and voice) with the other smartphone. Then we just say nothing, apart from encouraging them to speak their minds.

It is quite common for the first test to reveal that 7 out of 10 participants had trouble at the same spot in the tutorial. We fix that, and then go back to do 10 more such tests.

A more automated way to get such tests done – as well as going a bit deeper into the game – is available at PlaytestCloud.com. They don’t stop people in lobbies, but rather have people test play a game while recording what is happening on their screens and what they are saying. The game company then gets a video of the whole thing, and can watch and annotate that back at the office. It is a very useful service.

We have also done some more traditional user experience testing with several cameras, one way mirrors and questionnaires. While they work, they are quite cumbersome and, in the end, no more useful than the lobby testing or PlayTestCloud.

The deepest of the qualitative testing we do, is in collaboration with our nearby university. Here we wire up people with an Emotiv EPOC brainscanner, and a Tobii eyetracking device in front of them.
https://emotiv.com/epoc.php
http://www.tobiipro.com/

Together with videos, this allows us to see exactly what they are experiencing and where they are looking. It is useful for pinpointing some very specific problems in the game.

So far, it has been all about user experience testing. Of course, you should also test the game functionality technically. On the Apple side, there is a somewhat manageable set of devices. On the Android side, there is not. (Our Benji game has reported over X thousand device versions that it runs on).

TestDroid is a convenient service where you can test out your app on a huge number of different Android devices/versions. We simply make the game play itself and record it doing that. There are, of course, multiple other options as well for how you might outsource technical quality assurance, and a lot of companies offering such services.

At this point, we have tested the game out conceptually, technically, and with a limited number of players that we have listened carefully to. It is now time to go for larger numbers and start working statistically.

We try to go into pre-alpha soft launch with our games as soon as possible, and then develop the games in iterations, gathering feedback all the time. We release the game in some place on the other side of the world (to make sure our friends do not influence the data), and advertise to get small cohorts of users. Usually, we buy some 500-1000 users in each round we test.

To have a look at what these users do in the game, we need some analytics software integrated. So far at Tribeflame, we have made our own bare bones version, as well as integrated a number of others like Flurry, Game Analytics, Google (Play) Analytics, DeltaDNA, etc. Some solutions are very basic, while others are quite comprehensive. The important part is that you can see at least some basic numbers about where you lose players during the first sessions, and you are able to track retention numbers over the first month.

The different forms of testing will each give you it’s own unique look into some aspect of how the game works. None of them will give you the complete story, but they complement each other nicely. The soft launch metrics of thousands of players and show you how people behave with good certainty, but is does not tell you why they behave like that. In contrast, small groups of players that you meet face to face, or bring in through PlaytestCloud, will be able to describe the problems much, much better, but on their own, they are only a small biased sample. Together these two approaches give quite a good picture of how the game works.

Social features in Games

How to best use social features in games is changing. It is now less about reaching real-world friends for virality, and more about forming in-game communities of strangers with retention as the goal. Let me explain.

The big boom for social games came with Facebook. Games like Mob Wars came in 2008, while Farmville took off in 2009. This first wave of social games were engineered for virality above everything else. They kept pestering their users to post to their friends, and to get those friends to also start playing the game.

The social features of these games were not really that deep. The games behaved sort of like my 2 year old son. Here, he has loudly demanded that his uncle plays with Legos with him – only to then completely ignore said uncle while happily playing next to him. They are both doing the same thing, but with very limited interaction.

IMG_0507

That still has some value, even though there was widespread scorn for the term “social” when describing those games. There is social proof in having friends doing the same thing you do. The mainstream consumer starts doing something only when all their friends and acquaintances are also doing it.  

These games used a variety of ways to get people to invite their friends. There were suggestions that you brag about every achievement you got in the game by posting as visibly as possible on your Facebook wall. There were walls to unlock more gameplay that could only be passed by connecting to 3 or more friends in the game. And there were ways to send gifts to each other, in the hope of triggering the social obligation of reciprocation from your friends. (Have a look at Cialdini’s book “Influence: The Psychology of Persuasion” for more on tricks like these.)

All this was done to achieve a good “k-factor”, which is the measure of virality. The k-factor means “how many new customers does every existing customer bring in”. There’s an excellent explanation of it here.

In short, if your k-factor is above 1, that means that the game spreads on its own. You just need to seed it with some customers. Say you bring in 1000 customers through featuring and advertising. If the k-factor is 2, they will bring in 2000 of their friends, who will in turn bring in 4000 of their friends, etc. Eventually the whole world plays your game! (Or, what actually happens: the k-factor declines over time).

If the k-factor is below 1 (which it usually is), then it still means that your marketing is cheaper. If you spend $3 per download to get people to download your app, you will eventually get 2 downloads for that price is your k-factor is 0.5, bringing your effective cost per download to $1.50.

So far the early focus on getting the virality up by bringing in the real-world friends and acquaintances of the players. Early mobile games also tried to boost virality with similar methods, but it was way harder to get it to work well. New games are more focused on retention rather than virality.

To get virality, you should focus on the player’s real world friends, but to get retention, you want to build new in-game connections between strangers.

Social features are good drivers for retention, but only when some demands are met. Players can come back to a game for a variety of social reasons. If there are clans or guilds, players will feel a social obligation to play and contribute to their clan. With competitive features, people will be comparing their own progress to peers and try to keep up.

The problem is that both of these only work with players who are at roughly the same level. If I start playing any of the King games right now, it will not inspire me much to see my wife at level 245. If anything, I might get disheartened and think that I will never be able to catch up.

Similarly, when I play Clash Royale in my friend’s clan, I am actually dragging him down. He’s way more interested in the game than I am, and is also playing it a lot more as well as better. Which means that I should not really be in his clan. It would be in his interest to have better players than me in the clan. If he keeps to the clan that I am in, the social features will quickly become a liability rather than an asset. He is likely to stop playing, just as I stopped playing. If he moves to a clan with his own level of players, the social pressure is kept constant, and he is way more likely to stick around.

I think that this is a universal rule: it is unlikely that your friends are interested in exactly the same games as you are, and that they are equally skilled at them. Therefore, we can build games that try to get people to invite their real-world friends, but that is for short term virality. For the long term, we should transition players into making new friends in the game. Friends that share their interest in the game, and are playing at the same level.

The use of Game Conferences

Where to travel and what to do there.

There are quite a number of game conferences available. Why are they so popular, and should go to any of them?

At a short glance, it almost seems odd that the games industry is so keen on sharing their insights. After all, at these conferences, we are really telling our competitors how to build better products and business models. In most industries, that is not thought of as prudent business behaviour.

So why do companies do it? And why am I wasting your time with this blog, which is really about the same thing?

Most of all, it is about attracting great partners, by signalling competence. It’s really sort of the same thing as the peacock’s tail – it’s expensive for the peacock dudes to grow, and makes them easier prey for predators – hence only a really strong peacock can do that. And the peacock girls love such macho show offs.

By spilling the beans on all your business secrets at a conference (or a blog), a company can signal to investors, potential employees, publishers, etc. that they know what they are doing. They are even cocky enough about their competence that mere imitators, who only follow the advice given, will never be able to catch up with them. Therefore, they can afford to tell competitors their secrets – in exchange for a more respected position in the industry.

From my, admittedly short, experience with this blog, it seems to be working! We’re getting better people to ask for jobs at the company, and better companies to ask for potential partnerships.

Now that I’m done comparing myself to a peacock, which conferences should you go to? There’s a great list of most of them, here:

http://www.pocketgamer.biz/events/

http://www.eventsforgamers.com/calendar/

 

If you look at visitor numbers, the really huge ones are the ones aimed at consumers. There’s E3 in the US, Gamescom in Europe, China Joy in China, etc. There’s also Spiel in Essen in Germany that is about board games – the non-digital stuff.

These events have hundreds of thousands of people attending, but most of the attendees are players rather than game developers. Attached to them will, of course, also be business meetings for companies.

Among the developer events, the largest is GDC that is held every spring in San Francisco. Actually, it’s this week, and I’m here in San Francisco right now. Other, slightly smaller ones, are the Pocket Gamer Connects events (in London, Helsinki, Bangalore, Vancouver…), the Casual Connect events (Tel Aviv, Amsterdam, Singapore), Develop in Brighton, Nordic Game in Malmö, White Nights (St. Petersburg, Helsinki), Mobile Games Forum (London, and Hong Kong) and many more. GDC itself also has smaller versions in China and Europe. With all of these, how do you pick which ones to attend, and what should you do at them?

 

There are mainly three things one can do to keep busy at a games conference. You can

  • listen to the talks to learn new things
  • meet other companies to scheme about world domination
  • get drunk at somebody’s party or dinner in the evening

 

Most likely, you will pick two of these. Doing all three is likely to fail – so plan ahead which is most important to you.

Let’s start with the talks: if you are entering a new part of the business (like moving from console to mobile, from premium to F2P, becoming a VR pioneer, etc.) this is a great way to learn from other people’s mistakes and can save you a lot of money. Every year, I seem to get at least a few epiphanies by listening to some of these talks. All in all, this is pretty straightforward: figure out what is most important to you (tech talks, art talks, business models, game design, etc.) and just go listen to them.

Even more important than the talks, are usually the meetings. It is way more efficient to meet a whole series of companies at one of these events that ‘everyone’ is attending, compared to flying out for one-on-one meetings at each other’s offices around the globe. You can find potential ad networks, publishers, sub contractors, ad networks, localization agencies, investors, at networks, etc. – all in one place. Did I mention that there are likely a few ad networks there too, willing to meet with you?

Try to figure out before hand what is important for you – meet with those people – and make sure you have at least some time to catch your breath in between. I actually find it good to also meet with a few of the companies that you will likely not partner up with in the end. You get to hear more views on where this business is headed, and might find some ideas that you did not know existed.

Last, the dinners and parties. These are great for networking, and a way to get you more contacts that can help you out at some point. A lot of times, I have been at some event that I did not think was super useful, but it lead to another not-that-useful contact that led to a third – that in the end saved our asses when we really, really needed it.

These events are a sort of numbers game – every year you will get a few more contacts and get invited to a few more dinners. Slowly you become one of the insiders that everyone knows, and get invited to even more of the “exclusive” events. People you meet at these events can help out a lot. Without those contacts it’s just quite a bit harder to succeed in this business (just like in all other businesses).

 

One last word of advice:

When you’re choosing what conference to attend, pay attention to the signal to noise ratio. The larger ones are not always better – it also gets harder to find the relevant people you want to talk to at those huge events. Conferences with a few hundred people can be excellent, if everyone there is worth having a discussion with. Of course, getting on the invite list for one of the dinners attached to the larger events can also get you to a place with amazing signal to noise ratio. If you’re not already getting those invites, try holding a talk at the event – or writing a blog. I hear that can work.

The Price of Entertainment

What 1 minute of Spotify pays the artist, vs what 1 minute of mobile gaming pays the developer

It often strikes me that creative businesses have a lot in common with each other. Likely the clash of commercial demands and artistic demands often will lead to similar situations. A few weeks back, I wrote about what games can learn from storytelling in movies. Let’s expand a bit on that and have a look at several other “commercial arts”.

One thing that we will find across at least movies, music, games, books and ads (advertising counts as one of these arts!), is a tug of war between creative and analytic. (The same might apply to architecture, design, painting and sculptures as well, I have not checked.) For all of them, there is one faction arguing for making decisions based on cool data, while another faction ridicules them as unimaginative robots and praise creativity instead of conformity.

Let’s take advertising as the first example. One classic book on the art of selling stuff is Ogilvy on Advertising, where Mr. Ogilvy (founder of one of the largest ad companies in the world) explains how different ads work statistically. “People read headlines 5 times as often as they read the body. People remember ads with news 22% more than ads without news.” He also states: “If you are lucky enough to write a good advertisement, repeat it until it stops selling.”

On the other hand, you have e.g. Luke Sullivan’s “Hey, Whipple, Squeeze Thisarguing more for the creative side. The name of the book comes from an effective, but annoying ad from the -70’s, where Mr Whipple was selling toilet paper. That ad ran for a long time, as it just kept selling – exactly as Mr Ogilvy suggested. As Sullivan writes: “In 1975, a survey listed Whipple’s as the second-most-recognized face in America, right behind Richard Nixon… To those who defend the campaign based on sales, I ask, would you also spit on the table to get my attention? It would work, but would you?”

Ads are an interesting study that in a way is similar to games. Another I already wrote about is movies. There you can find long debates for and against the very formulaic script of “The Hero’s Journey” with Jungian archetypes on top. Or, have a look at how The Economist just analysed movies, here

They give a great formula for making a hit movie – but also end with the statement “ But do it for the money, not the plaudits: such a film would have just a one-in-500 chance of carrying off an Oscar for Best Picture.”

 

For music, I believe making a Billboard Top-100 hit has a lot of similarities with making a App Store Top 100 grossing hit. A team making mobile games can still be quite similar in size to the teams making billboard hits. And in both cases, the numbers guys churning out polished, but quite formulaic stuff will more often win the chart positions – with the occasional artistic rebel breaking all the rules and succeeding in spite (or because) of it.

The Swedish music producer Max Martin has the best track record in modern times. 54 of his songs have hit the Billboard top 10 chart positions! You really want to have read here on how he does it.

 

Let’s end this with an interesting comparison: what is the price of entertainment per minute for the different art forms?

Spotify claims to pay about 0.7 cents to the artist each time someone listens to a song.

With 3.5 minute songs, that’s 1 cent every 5 minutes or 0.2 cents per minute.

 

A good mobile game would follow “The Devil’s Rule” of 666, and make about 10 cents Average Revenue per Daily Active User (ARPDAU). As each player is then spending 6 times 6 minutes, or 36 minutes per day in the game, it comes down to 10/36 = 0.28 cents per minute. That’s less than 50% difference from what music pays per minute!!

Of course, not all games will have 10 cents in ARPDAU (believe me, I’ve made games with way less!), but then again, not all games will hit the 666 rule either. Less successful games not only have lower ARPDAU – they are also played less minutes per day.

According to Netflix’s quarterly report, their customers spent an average of 568 hours watching the service in 2015. That would cost the customers 12 * $7.99 or 0.28 cents per minute – exactly the same as for mobile games!

Movies and premium games still manage to ask for a much higher price per minute of entertainment provided. Apparently, the average price for a movie ticket last year in the US was $8.43. Let’s say the average movie is 2 hours long. That would make about 7 cents per minute – or about 25 times what mobile games cost.

A premium game is similar. It might cost $50 for some 10 hours of entertainment, which comes down to about 8 cents per minute – similar to movies, and much, much higher than mobile games. Of course, the variation is immense for premium games. Some people might play them a lot longer, and thus get a much lower cost per minute – but a lot of people also pay for premium games that they end up playing way less than 10 hours.

Why do premium games and movies succeed in charging so much higher prices for their entertainment compared to music and mobile games? I would suggest two things: Production costs for movies and AAA games are higher per minute of entertainment provided. A 2 hour blockbuster movie will likely cost more to make than 35 music singles (also about 2 hours).

Also, they require more focus from the consumer. Both mobile games and music is something that people do a bit on the side, with less than their full attention. Playing a match-3 while watching TV, or listening to music while working. In contrast, movies and AAA console games are immersive and will demand your full attention during several hours. They’re a more intense form of entertainment, and thus the price can also be more intense.

Chain reactions of Luck

Why match3 is a really really good core game.

You might have noticed that the match-3 mechanic is really popular on the top grossing charts of the App Store and Google Play. Why is that? It just happens to be a really good core mechanic.

Before we dive into why match-3 is such a good mechanic, let’s look at something that isn’t a good core for a mobile F2P game. Let’s actually look at something that is a terrible idea: a traditional puzzle game.

As an example, let’s take Tribeflame’s own game from 2012 called Light the Flower. In this, we invented a cute mechanic where you moved mirrors around to guide sunlight onto flowers that were languishing in the dark. Here’s the trailer for that game:

This was still a premium game, and we could not convert it to F2P in any sane way even if we wanted to. Why? Think about the progression – we’re going from easy puzzles to harder and harder puzzles. Puzzles where there is only one or two correct answers.

Either we have a smart player that is just going to breeze through these challenges until they get bored, or we have a… umm… less smart… player that is going to get stuck in one of them. And when they are stuck, there’s not much we can do apart from telling them the answer, which will likely just make them feel “less smart”.

In short: the progression here just sucks!

 

If you enter some luck into the game, we will be able to fix this. Now, the less smart players are actually just unlucky (or so they keep telling themselves), while the lucky players are smart (or so they keep telling themselves).

There are actually surprisingly few games on the top charts that combine luck with skill. Some 20% of the games on the top charts are variants of match-3, and have such a combination. Then there are about 25% that are casino games where there is basically no skill involved at all. The rest (little more than half) are really deterministic games at their core. That is, if I were to replay an attack in Clash of Clans with exactly the same army and deployment strategy, I would get the same result.

For the level based puzzle games, the balance of luck and skill is crucial, however. With the match-3 mechanic there is an extra bonus: chain reactions of luck.

In a game like Bejeweled, I can make one move somewhere on the field – trigger an explosion that triggers another explosion that triggers a third and a fourth one – and eventually the entire field blows up. This serves two purposes: it’s the jackpot that players crave to make them feel really, really good, and it is what gives them enough hope to keep playing a session that started badly. Even the very last move can turn my luck when there are chain reactions involved!

Just how much luck the player has in a match-3 can actually be completely decided by the game engine. In this way, the game can help a weak player advance, and give some extra challenge to a good player. The jewels/candy/fruit/what have you that fall down from above can be calculated to give a certain result.

To catch the games doing this, try playing a match-3 daily for some time, and then stop for about two weeks. If the company did their analytics correctly, they will assume that you are about to churn out of their game, and cease being among their valued customers. To stop you from doing that, they will want you to have a great session that makes you feel good if you decide to give the game one last chance. Which means that the first session after the two week pause will likely be an amazingly lucky one where everything is going your way!

 

There might be an optimal difficulty curve for one individual in a puzzle game. The problem is that it is different for all individuals. In a game with a component of “luck”, it is possible to adjust the game to the individual, and that’s what a lot of the match-3 games are doing.

 

To sum up, match-3 is a great core mechanic because of these reasons:

-The UI is a very simple and direct one swipe

-There are Chain Reactions of Luck that the player craves

-The player can win with the very last move, there’s always hope

-When the player lost, and bought an upgrade to continue, it is unclear how many extra moves are required to finish and win. That is, the player can feel close to winning even though he/she is actually more than 10 moves away from it

-The game engine can control the luck with what drops down

-The game can be interrupted at any time without the player losing

-The game is suitable for one handed play in portrait orientation (you can play standing on the bus)

-The game has quite small footprint (size, loading time, device requirements)

 

Nothing is quite perfect, however. These are the main weaknesses that I can see in most match-3 games:

-There is nothing permanent built by the player. Only the level progression gives the feeling of accomplishment, rather than e.g. a village the would likely give the player stronger emotional ties to the game.

-There are no stats that can be upgraded by 10% at a time. This is why Puzzle and Dragons pairs match-3 with the dragons that can be upgraded in this way. That’s a way stronger meta game.

-There is no Player vs. Player (PvP) to drive competition and spending by the most competitive players.

-They are somewhat weak when it comes to being brandable. Most match-3 games look the same with a quick glance.

Guest Appearances

I gave a Keynote at Pocket Gamer Connects in London a while back, based on the 666 post.

 

And, Pollen.VC did a story on my thoughts on retention: http://insights.pollen.vc/app-business/measuring-real-retention-for-real-results