Coming out of Focus
Today I’d like to talk to you about…yes,
FOCUS GROUPS.
There was a time when I was at my former employer during the building of the Gears of War franchise that we leaned heavily on Microsoft’s internal testing labs. They had one way mirrors, for fuck’s sake, whole place looked like a police interrogation lab or R Kelly’s Bedroom. They’d bring in local “target demographic” (usually 18-35 year old males) and have them play a work in progress version of the game. We’d then get back an extensive report on what were the problematic areas, suggestions for improvements, and ratings on how fun each area was.
During the time this was useful…to an extent. At the end of the day we ultimately trusted our guts and used that to make a fair call on any given issue raised by focus groups. We shipped the games, usually to much fanfare and a good amount of acclaim and sales, and moved onto the next one. It was only after I’ve had the last year+ to reflect on the experience to realize that focus groups, when not used properly, are fundamentally a flawed way of looking at your game and I’m about to explain why.
Put simply: The party performing the focus group has asked these folks to come in and play the game. They already feel special, like VIPs, almost, entitled. “They want MY opinion? Wow, well then, I must be pretty darned cool!” They then sit down and play the game. Here’s the problem, though.
They don’t really play it.
Before I finish my point, let me fast forward to the present. I’m currently enjoying the hell out of “South Park: The Stick of Truth.” Last night I came to the Alien Spaceship Pilot and Co Pilot boss fight. As of this writing I’ve attempted this boss fight no less than 10 times, each time failing, and two times getting the pilot down to one health before he smites my poor little cartoon ass. I can skip the cutscene where the aliens get up from their chairs faster than you can say Unclefucker. I’ve yelled at the TV, tossed my controller, and generally scared the shit out of our dogs and prompted my wife next to me (who is still slogging through “The Bureau” for some godforsaken reason it’s like some alternate dimension Gears I think I saw fucking Tickers in there) to say “Wow. That looks hard.”
Know what I’m going to do as soon as I finish this blog post? I’m going to go back upstairs to our game room, fire up that damned game, and kick some alien ass. And I’ll tell you what, when I finally take out those two fuckers I’m going to be jumping up and down happily yelling “IN YOUR FACE YOU ASS PROBING MOO MOO MOTHERFUCKERS!” because…
THAT’S WHY WE PLAY VIDEOGAMES IN THE FIRST PLACE.
Disclaimer: Gears will always have a place in my heart, and again, I think Rod and co are going to knock it out of the park.
However, there has been some valid criticism over the last generation of games that they were:
- Too linear
- Too easy
- Too hand holding with tutorials and puzzles
I remember sitting there with designers looking at videos of gamers getting lost in just the simplest of maps. A bare bones map, and they have no idea where to go. So, we’d make things a bit more linear. They can’t figure out how to perform a move? Force a tooltip on them! (no one reads that shit unless it’s on a loading screen anyways.)
The problem wasn’t the game. The problem wasn’t that it was hard or too difficult. The problem was that the folks in the focus group didn’t put down $60 of their
OWN HARD EARNED MONEY to BUY THE GAME.
Their hearts aren’t in it. The first sign of something being a little tricky or confusing and they stop and write it down or it’s noted. It’d be as if someone who hadn’t run much in their life suddenly decided to go for a jog and found out it was too hard and was then denied the satisfaction of a completed run.
I get a good amount of shit for free. I didn’t get hooked up with South Park, I had to buy it myself with my own money. Now, I’ve been fortunate enough in my career that I can afford shit I want to buy, which is fucking amazing, but it doesn’t matter - that’s my sixty dollars that I worked hard for and I’m going to milk every bit of fun out of it, anal probes be damned.
People
behave differently when they know they’re being watched. And, while data is
good to a point, when you play too much to the data the tail winds up wagging the dog and you get watered down mushiness.
(Can you imagine if Dark Souls had gone through this ringer?! It would have been shit. “I died too much!” BOO HOO GO PLAY DANTE’S INFERNO YOU PRICK!)
On a side note, there’s a basic social interaction that comes out of asking someone what they think. If I was sitting at my favorite sandwich shop enjoying a pastrami on rye by myself and no one was watching me you can bet it’d be a far different experience than if someone was sitting across from me in a lab coat with a clipboard asking 500 questions about the sandwich. “How’s the crust?” “Well, normally I love it, but now that you’ve empowered me to act like a fucking expert I can tell you it could afford to be a little crunchier.”
Finally, if you ARE doing this and taking a survey, keep it AS SHORT AS POSSIBLE. Occasionally I’ll get an email from a company that is seeking feedback on something that I use, a service, or a product. And once in a while I go “fuck it” because I don’t have a job right now and I decide to take the survey. If I see it’s more than 10 questions, I say “nope” and go back to Twitter.
I’ll take a well managed and educated gut call over a million focus groups, any day of the week.