A Survey on Game Reviews, and Whether or not They Suck

[Decado]

Last year I recruited some Codexers -- along with some Obsidian forum posters, WL2 forum posters, some guys from the WL2 team, and a bunch of other random people I know -- to take a survey I was conducting. I was studying the way people use professional critic reviews, amateur reviews (from metacritic, places like that) and whether or not they found reviews useful or helpful to making purchasing decisions. I was also interested in studying what people's opinions on gamer reviewers or game journalists were, especially since so many of them seem to be quite mistrusted.

So anyways, I finished my thesis and it is published, but I also distilled the results down to a blog post. When I started the survey, a bunch of dudes asked if they could see the final results when it is done, so here it is:

http://thegwumps.blogspot.com/2014/01/respectable-men-and-women-have-polite.html

Some key points:

1) When asked which were more useful -- critic reviews, user reviews, neither, or both equally useful -- the respondents picked users over critics, 30.1% to 14.4%. 60.9% said that user reviews gave them information they didn't get from professional reviews.

2) Most of the respondents had pretty good opinions of both professional reviewers and amateur, user reviewers (this surprised me)

3) There was no statistically significant relationships between how much money a person had, and how much they relied on reviews to make decisions. Same thing with the amount of time they had available to spend on gaming, and their reliance on reviews.

Anyways, there seems to be a gap of knowledge between how/why gamers read critics, and what they think they get out of the process, and how that ultimately affects purchasing decisions. It was interesting to see that people considered user reviews to be "more useful." It was also interesting to see that for the most part, people have no beef with professional critics -- including most of the bros on the Codex.

 

[DragoFireheart]

Anyways, there seems to be a gap of knowledge between how/why gamers read critics, and what they think they get out of the process, and how that ultimately affects purchasing decisions.

What % of these people are Bioware/ex-Bioware fans?

 

[Lemming42]

Interesting survey.

I don't have a problem with professional game critics (although I value their opinion as much as I'd value the opinion of a random guy off the street), but I can't stand these "gaming personalities" that have come about as a result of "professional" critics. Jim Sterling is the example that leaps straight to mind.

 

[Cowboy Moment]

At this point I basically consider "professional" reviews a tool of publisher marketing, they cannot be trusted when it comes to games with large marketing budgets. And for niche games, review outlets rarely have people knowledgeable enough in a given genre to give an informed opinion.

To offer a simple example, you will sometimes see reviewers lower the score due to a bad UI (most recently happened to Blackguards) in a niche game, and often rightfully so, but I dare you to find me a major review site that lowered Skyrim's score for the same reason (and Skyrim's UI is fucking atrocious). It just doesn't happen.

The most painful thing about it is that if they did, they'd immediately find themselves under siege by fanboy brigades from neogaf, reddit, and other peasant gathering places, fueled by the rage that their precious game didn't get 10/10. Fucking degenerates.

On a related note, I'm not surprised at all that Codexers are, in fact, a lot less edgier than they appear to be.

 

[Jasede]

S-, shut up you nigger.

 

[sea]

The only reliable and trustworthy reviews are obviously the ones I write.

That's interesting data, especially regarding how user impressions of reviews are not affected by things like discretionary income. It seems to me that reviews may be less about motivating users to purchase games and more as a way of affirming existing opinions post-purchase, or as a warning should a product not be up to par. And it's quite clear that reviews-as-entertainment are a pretty big thing too.

 

[RK47]

reviews-as-entertainment

is what i watch / read them for.
games aren't worth full price nowadays.
With the kind of discounts steam offer these days, I just ignore every hype and news about the game till months later.
So all those previews, and day 1 reviews don't really hold much of my attention.

 

[Clockwork Knight]

The most painful thing about it is that if they did, they'd immediately find themselves under siege by fanboy brigades from neogaf, reddit, and other peasant gathering places, fueled by the rage that their precious game didn't get 10/10. Fucking degenerates.

These people are what caused the Doritos movement to begin with, not publishers. They already made their mind on a game (possibly even before it comes out) and want to be told they have good taste. Why would you deny them that? Give them their 10/10.

This is why I think it's possible many (p)reviews of AAA+++ games aren't actually being paid for, there's no need for that. Give the obviously awesome game a low score and readers will leave in droves to a reviewer who knows what he's talking about.