Still working to restore things from a backup of the old website | sign in

Game Design - A Case Study of Pokemon GO

2016-08-08 12:55:00

A couple of weeks ago the smart-phone game Pokemon GO was released in my region. I played Pokemon on my Gameboy when I was a little boy and my friend had already pointed out how all the people walking around us were playing the geo-location-controlled game. Being free and easily available in the store, I of course decided to try it out, as did my friend.

Unfortunately, it wasn't, and isn't, a very good experience. I still play the game every single day, and I'll write more on why later. But as a developer myself, I really feel like Pokemon GO is a terribly good example of a mistakes in game design which should not happen. Things obvious to me appear to be completely unclear to the makers of the game; Niantic Labs.

Even though they've made a similar game once before, they still clearly do not have the experience required to develop games which are actually fun to play. My hope is that they, and others, will read this and make more enjoyable games in the future. Or change their current games - please Niantic, I would so much love Pokemon GO to be an awesome game!

Releasing an unfinished product

This should be well-known by now, but unfortunately it's becoming more common since releasing updates is so painless today. In the case of Pokemon GO, since it relies on walking around outside, Niantic obviously had to release during summer. But why not next summer? They were probably pressured to release it before the next iteration of the regular pokemon games as part of a marketing strategy.

It's also important to note that it's very likely that they agreed to this release schedule ahead of time. They thought it could be done in the short time frame because all they had to do was copy their old game Ingress. Not so. Or maybe so. After all, that it's a copy of Ingress can be seen clearly, in fact, too clearly. An other lesson here is thusly: Make sure you have the time to finish the product. And then add some more time to that, unforeseen problems will arise, they always do.

Is the game unfinished though? It was released with a large number of major bugs - that is, bugs which significantly affect regular gameplay - not edge cases or cosmetics. The game totally lacks a so called "end game" which is the part of the game which you play after you've gotten good at it. And there's a disturbing lack of features most people would expect like trading and and PvP. And of course, all the faulty game mechanics I'll list in this article. Additionally they said themselves that only 10% of the game is done, and the major version number is 0 - it's not version 1 yet. Personally, I wouldn't even call it a beta-test, it's clearly alpha quality.

But how bad is this? Really bad. You can only release a product once, that's it really. You can re-release, or make a "new and better" version but it's nothing like the true release in terms of how many people you get to reach out to with your product on the first release. You have one chance to make a good first impression because that impression will stick. If you make a poor first impression then that's what people will remember, they'll leave, and most won't come back even when you've fixed everything.

Server load and queues

One of the primary problems which plagued Pokemon GO for weeks following the initial release in the US and the release in Europe, and the one in Canada, was the fact that you couldn't play because they servers were overloaded. Now, if you're completely unknown and you make a completely new game without any hype and your servers get overloaded because you didn't predict such a storm of users, you can be forgiven.

But if you're already a known successful developer (sure, maybe Niantics previous game Ingress doesn't count as a success - it's more of a proof of concept) or you're releasing something there's already an audience for, like when you use the Pokemon Intellectual Property, or there's any kind of hype or public interest then you really need to handle the situation much much better.

First of all, use a scalable architecture. This means that you'll automatically use, and pay for, as many or as few servers as you need. You should of course also have a set of your own servers as part of this architecture to keep costs down. Pay attention to your design though! Always look for the weakest link of the chain. For instance, it's common to forget (or ignore) scalability for the login servers. It's easy to think "meh, all they do is log people in.." but you will regret it. I'm not completely sure where Niantics bottlenecks were, but they certainly failed.

If you do mess up on this though, and people can't play, you really need to provide a substitute. It's actually very simple though: Communicate. All you need to do is tweet "We're sorry, our servers are overloaded, we're working on it." with some variations a couple times a day. That's it. You can do more, and I would highly recommend it. But a little goes a very long way in this case.

But most importantly: Queues! They've probably existed as long as humans have (if not longer), they're really simple, and they work. If you're making a game and there's any chance a bunch of people will want to play it, spend a single development day making a basic queue system. If you never go over capacity then it wont be a big loss, but if you ever do it will make all the difference. Instead of a game which no one can play because the servers get overloaded (and everyone keeps spamming the servers to get in anyway), you'll have a game actually being played by as many people as you've built capacity for. And the rest will be in a nice queue, which is better than not knowing when, if ever, you'll get to play.

The big issue with server load (which is true for many other systems as well, like the highway) is what's called peak load. That is, the highest load, not the general load. You don't just have peak load on the release of a game, it also happens every evening/night when people play the most and even more so on the weekend. That means being able to deal with high peaks is useful not only on release but on a regular basis, plus when you have any kind of event in the game. Implement a queue, and you wont have to worry so much. Plus you can give players in the queue tiny rewards (like xp) so they don't feel it's a total waste. And you can also incentive playing at low peak hours with similar bonuses in order to even load out more. (You really don't want to do the opposite and penalize players during peak load.) In either case, don't release an online game without a queue system.

Unrecoverable mistakes

When you start playing a game you're going to make a lot of mistakes, and even after you've played for a long time and gotten really good at it, you'll still make mistakes. Of course, when you make a mistake, you should have an unfavorable outcome, but if you can never recover from a mistake then that's very poor game design.

Pokemon GO suffers from this issue to a large extent. For instance, when you use stardust to power up a pokemon, unless you're already at a very high level (level 30+ probably), and the pokemon is too (near the end of the semi-circle) and the pokemon is of a species which is very powerful, and is already evolved so you know it has the right move set and it also has a high IV (basically a random hidden bonus to CP).. unless all those things are satisfied, you've wasted it, because stardust is limited, and you can't realistically recover. You will forever be behind those who didn't make that mistake because you'll have to spend your time gaining back that stardust while they'll power up their next pokemon.

There's a similar issue with evolving any pokemon (except for the common/crappy spices'), you need to wait and make sure you have a good one before you do it, or the candy will be wasted. Also, Lucky Eggs, if you don't use them to make 60 evolutions at the same time, you're gonna fall behind, or you have to buy more. Same thing with incubators, you really should save the 3-use ones for 10km eggs and only use them after you've hit level 20, or you'll have to buy more to catch up.

According to these mechanics, the player is actually expected to figure these things out, without any hints from the game, before ever doing them (even a single mistake can be significant) and additionally, the player should then actually not take part in a large part of the game until much, much later, which is even more bad design.

Punishing progression

A lot of people have voiced their concern over how the game becomes more and more difficult to play the further you get. Of course, that's how a game should work. But there's a fatal flaw in it's implementation. As you get a higher level, you can also find pokemon with a higher CP, and these are "harder" to catch. I put "harder" in quotation marks because it isn't actually harder to in terms skill - you simply get worse random odds. For example, giving a Razz Berry and throwing a Great ball with Curveball and Great Throw bonus multiple times and the CP 450 Seel, which you really don't care about because you have it in CP 600+ already, still breaks out and eventually runs away. This doesn't make the game more difficult, it makes it more annoying.

A second part to this issue is the lack of an increased reward accompanying the higher difficulty. As a game gets harder, it should also offer more reward. If it does not, you realize that you're being punished for progressing - the game is in fact doing the opposite of motivating players to continue playing the game. Catching a higher CP pokemon should give more XP, and more candies. I doesn't have to be a lot, but it should be noticeable and make it worth it. Especially since you're most likely to transfer it anyway, because you already have a better one.

Encouraging bad play

There's one thing in particular which stands out among all others when a game has been poorly designed, and that's when it's beneficial for the player to play "badly", or even do the opposite of what the game intends you to do. It clearly shows how the creators are not game designers, or gamers, and probably haven't even played their own game to any significant extent.

For instance, as noted above, trying to catch a higher CP pokemon doesn't really give you any more reward than a low CP one, but it's much harder. It's therefor in the players own interest to ignore high CP pokemon and just catch the low CP ones most of the time. Especially when rare evolved forms spawn, unless they happen to be of an uncharacteristically low CP and you never evolved it yet. Those spawns should be sought after, not ignored.

An other thing which has also been noted above is how the game encourages you to not do certain things in the game until much, much later, like boosting, evolving and hatching pokemon. It's a lot more fun to do it often, but it's a lot better for your progression to do it after you've already probably gotten sick of the game.

The most telling issues though surface when we look at PvP, which now only exists in the form of "taking over" gyms. It isn't just a little easier to take over a gym, and not even a lot easier. It's massively incomprehensibly easy to take over any gym from any one of the opposing teams as soon as you're getting a higher level. This is because you can use 6 pokemon to beat a single one. As long as you can win against their one best with 6 of yours then you'll take it over sooner than later. And yeah, 6 against one really is as easy as it sounds.

On the other hand, defending a gym, or "training it" as it's called requires you to not only play well, but also play for a much longer time in order to succeed. This is because you must win one-on-one, and you also don't get nearly as many points, even if you manage to win with a pokemon half the CP of your opponent.

Because of this, many people have a second account on a different team, or one person in a group of friends is part of a different team, and they'll use that to take down their own team so that they can then win against the other team, instead of bothering "training" to power up their own team.

This also means that defending a gym is entirely impossible, which in turns begs the question, why should you even bother taking over, or defending, a gym? And it's true, after you've taken over a few gyms and lost them a couple seconds after, literally even before you've crossed the street, you really don't care about gyms anymore.

An other similarly ridiculous thing occurs when your pokemon has been defeated while defending your gym, or been defeated trying to train your gym. It hasn't been knocked out, but it's very low on health. You now have to use potions to heal it, but potions are kind of rare. However, Revive is not. In fact, most people are constantly trowing Revive away when their bag gets full.

Revive wakes your pokemon up from their knocked out state, and heals half of their health. But if your pokemon isn't knocked out, it also can't heal it. So will you spend your potions, or find an opposing gym, attack with your low health pokemon, with no chance of winning, so they'll get properly beaten up and knocked out, so that you then can use Revive on them? Yes you will because you need to save your potions. In fact, if you could, you would use your own fists and beat up your own pokemon to knock them out so that you could then use Revive. Clearly, someone didn't think this through. Revive should probably heal a pokemon up to half it's health regardless.

Don't make random chance the basis of your whole game

In one sense, I can't argue against random chance. After all, it's human nature to be completely fooled by it. Games of chance, simple dice, casinos, lotteries and " and you can win an iPhone!" are only some examples of how we've let other people deceive us through random outcomes for thousands of years.

We seem to inherently be completely focused on what we imagine could happen, and not on what in all likelihood will happen. Random chance has always been a part of many types of games, although more serious competitive games like Chess and Go are completely without chance.

Random chance is a powerful tool to provide a varied experience in a game. You can use it to make sure the same thing doesn't happen over and over again. The problem arises though, when you use random for everything, and actually cause the same thing to happen over and over again because of it. Let me explain.

When different pokemon appear randomly, you provide variety to the game experience. But because of random chance, some pokemon might not appear as often as others, even though the chance of it appearing is actually the same. Even if you throw a dice 6 times, you probably wont get all the different numbers - you'll have to throw the dice again. You may in fact be so unlucky to have to throw the dice over and over again.

In Pokemon GO, you might therefor have to walk around the same area over and over again, because it just so happens that whichever pokemon you're looking for didn't spawn this time purely out of random chance. When you find the pokemon there's a random chance to catch it, even if you can somewhat improve your odds, it can still both break out, and run away.

Then the CP of the pokemon is of course randomly chosen as well, and so is the IV (hidden bonus). In order to evolve a pokemon you then have to be lucky to find it and catch it several times, plus its attacks are chosen randomly then as well and you can chance them, which means you might very well have to start all over again.

Because of this, the supposedly varied experience very quickly turns into a completely soul crushing desperation of doing the very same thing over and over again hoping for an eventually luckier outcome. But we certainly haven't gotten any better at thinking rationally about random chance.

The random pokemon of doom

Many people are chasing pokemon with "perfect IV", probably totally oblivious that with the three IV values being randomly chosen between 0 and 15, it creates a total of 4 096 combinations, of which only a single one is perfect. I'm not gonna go into the math here but that actually means that if 10 people were to catch a pokemon more than 2 800 times each, only 5 of them would get one with a perfect IV.

I just turned level 26 and I have caught just over 3 000 pokemons all together. And most of them are of course the common ones. Since it doesn't matter if you catch a Rattata with a perfect IV, you have to catch thousands of pokemon of the most rare and powerful types. Now we don't just have to deal with random chance, but also low random chance. And then of course we need to find one with a reasonably high CP too and then it'll have to get good attacks when we evolve it.

Actually, let me calculate that for you. Let's not bother with perfection and instead just say we want the IV to be at least 80%, and CP to be at least 70%, but we do need the best attacks which is about 1 in 5 and we're looking for one of the top 10 highest rated pokemon right now which I know according to data mining have an average of 0.5% chance of spawning (including their evolutions), but let's say we can improve it to 2% by looking in the right area.

If we assume we're able to catch them all before they run away, we're looking at a number in the order of a single pokemon out of 4 200 seen. So we still end up with a number like the one above. That's just for one. And presumably we'll need 5 more to fill our attack team, and then one more for each gym we're holding.

Giving unfair advantage

Games are about competing. Some people only compete with themselves, but most fun game play comes from competing with others, or competing with the game itself, but that's of course based on how well people in general can play it, which means you're still indirectly competing with others.

Whenever there's a competition, the question of advantage always comes up. One might argue that the winner of a competition always has an advantage, obviously they're superior to others at that particular skill set, that's why they won. But that's generally accepted, the question is really about unfair advantage.

What's unfair then? Well, any game is just an arbitrary set of rules everyone has decided to follow, which means that what's unfair is equally arbitrary. When you design a game you must therefor think carefully about what "skills" should give advantage in this game, and what will the players consider to be unfair advantage, causing them to lose interest.

Often times most players will argue that gaining advantage with money is unfair. In Pokemon GO, the biggest advantage you can get with money is hatching eggs for rare pokemon and their candies. Exactly how big an advantage this is isn't exactly clear. Hatching eggs is a slow process, but with money you can hatch almost 9 times as many eggs and the better eggs do yield more rare pokemon, with better IV and more candies to evolve and boost faster.

Very few people have vocalized any issue with advantage from money in Pokemon GO, even though several high level players have stated that eggs are the way to go for the endgame. Indeed, the full power of buying Incubators with real money to hatch more eggs probably isn't well known among most people because random chance is such a complicated subject to begin with.

Pokemon GO also hides a lot of information about how the game works. Pokemon level, IVs, the actual stats (which become CP), speed of an attack, how much energy it generates (for special attack), how a defending pokemon attacks differently than a player and has double HP, and so on, and so on. That means it's a game with unknown rules.

Anyone can agree how easily a game you don't understand can become frustrating, and when you don't know the rules, then you obviously can't understand them. But the even bigger problem is that the rules of a game can be figured out through extensive testing (and hacking). It's a problem because it now gives the minority who's "in the know" a major advantage.

You might think of it as a honest skill to be able to figure it out, and many agree, but using hacking to do it is always considered an unfair advantage. Additionally, even without hacking, only a few people will actually figure it out, everyone else (who're still in the minority of all players) will just read it on some website.

An other big advantage is simply put; time. Given what we know about about random chance in Pokemon GO from the sections above, it's clear that unless you spend a huge amount of time playing the game, you're not going to be able to randomly collect any competitive advantage.

And of course, combining knowledge, time and money gives you the greatest advantage. This a terrible problem. It's a problem because most people don't have nearly as much time nor money to spend on the game as some others do, and they don't realize you're "supposed" to search through third party websites to know what the game keeps hidden.

And since there's basically nothing else in the game to give you any advantage (what would normally be called "skill"). The majority of players will end up thinking "I can't compete with people who play all day long, or spend real money, and I simply don't understand why I can't win" and lose interest in the game.

Then, when the number of players drop off, the people who spend a lot of time and money won't find it satisfying to be better than others anymore, because no one's playing. And the whole game dies.

Limiting the power of money and time, and removing much of the games reliance on random chance, as well as make the rules known to everyone, would distribute advantage much more fairly among a larger number of players. Introducing some skill-based mechanics would provide alternative advantage for those with little time and money.

For instance, in the real Pokemon games, type advantage does twice as much damage which means that strategically using one pokemon instead of an other in a specific situation gives a significant advantage. In Pokemon GO the type advantage is very small.

Additionally, it seems Ninatic has used random chance as a substitute for actual game content. That is, instead of providing varied and interesting game play which keeps you engaged, they've slapped together a bunch of simple mechanics with a lot of random chance to provide variety instead. But it quickly becomes boring. If the game can keep the the player entertained, it can be enjoyed even if you're disadvantaged against everyone else.