Game reviews are useless.

[DarkUnderlord]

... or at least, according to Gamasutra they are. Apparently EA earlier this year <a href="http://www.thetechherald.com/article.php/200808/210/Declining-Metacritic-scores-irk-EA-boss">expressed concern at their low average scores</a> on the site <a href="http://www.metacritic.com/">MetaCritic</a>. Former Eidos president Keith Boesky has <a href="http://www.gamasutra.com/php-bin/news_index.php?story=18562">told Gamasutra why EA should just ignore those reviews altogether</a>:
<blockquote>"This overall score, or METASCORE, is a weighted average of the individual critic scores. Why a weighted average? When selecting our source publications, we noticed that some critics consistently write better (more detailed, more insightful, more articulate) reviews than others. In addition, some critics and/or publications typically have more prestige and weight in the industry than others. To reflect these factors, we have assigned weights to each publication (and, in the case of film, to individual critics as well), thus making some publications count more in the METASCORE calculations than others."

I get why he does it, and he likely had the best intentions, but it doesn’t work. The critical view is subjective. Doyle's determination of the critic's value is also subjective. So we are really getting a third generation facsimile of a subjective view of the quality of a title. If you factor in the uncertainty of the gallant, but flawed effort to convert A to F scales to numerical equivalents, Sir Francis Galton would certainly call foul.</blockquote>
Subjectivity? In reviews? The horror! Personally I ignore scores and sift through the negative user feedback. If all I see is "dis gaem suks", chances are I'll enjoy it. If however, I see a well-written, articulated, multiple paragraph answer on just what's wrong with the game, chances are I pay attention.

Spotted @ <a href="http://www.gamebanshee.com">Brother None's Fairy Kingdom Playground</a>

 

[Dhruin]

That reviews are subjective isn't his point; his point is that because Metacritic selectively weights reviews, they are skewing the average, which would otherwise (according to him and the citations) produce an accurate picture.

 

[DarkUnderlord]

Actually his point is that reviews are useless to gauge "quality" and that we should use sales instead as there's no correlation between the review score and final sales. He even says as much:

So, back to the Jerry Maguire moment. I guess we really shouldn’t rely on the Metacritic as an indicator of quality or shareholder value. Quality should be measured the old fashioned way: sales.

Effectively he's saying reviews are useless at the one thing they're meant to do. That is, provide you an indicator as to the quality of the product. If you couple that overall "point" with his comment about "subjectivity", then he appears to be pushing the argument that "we should be using sales, not reviews!" to judge how awesome games are. Particularly when the article was spurred by EA's own push to improve the quality of their games and the fact they needed some way to prove that increased quality (which is why they chose MetaCritic). Keith's argument is that they should be using sales to judge quality and ignoring reviews altogether (hence all the paragraphs calling bullshit on the correlation between sales and review scores).

 

[Saint_Proverbius]

My favorite thing about reviews is that a site will give a review of 10/10 and a few weeks later talk about how the game wasn't really as good as the hype. Or in the case of GameSpot with Oblivion, they pointed out lots of flaws in the play through video and didn't mention hardly any of them in the review.

Reviews are pretty meaningless, especially the reviews that are posted within the first week of the game coming out.

 

[denizsi]

I don't know who and where, but one reviewer on a game site had bashed this AAA title throughly, implying it was hardly worth buying and then gave the game a score of %95. I've found that to be the best way of maintaining a professional attitude without making yourself a target for the industry cut-throats. Suits only care about the score, most moron will only look at the score to make a decision and only the curious will read the review text. Win-Win.

 

[elander_]

I just bookmarked that Metacritic site. If EA thinks they are selective for giving them lower scores then they must be doing the right thing.

 

[J1M]

This is just EA coming up with excuses for why their games score lowly. Using his logic I could argue that Entertainment Tonight is the best show on television because it has the most viewers, etc.

It's kinda sad that sitting on their piles of money they are complaining that people didn't think their game was good. People who care about quality may not have thought it was good, but if you really cared about their money you wouldn't be making Madden #22.

 

[Gerrard]

Win-Win.

Win for who?
This will make masses of retards buy shitty games encouraging the developers to make more shitty games.

 

[Brother None]

Spotted @ <a href="http://www.gamebanshee.com">Brother None's Fairy Kingdom Playground</a>

Slander!

I'LL SUE.

But seriously, you completely missed his point, DU. He was saying:
1. Gaming discourse attempts to show a direction correlation between MetaCritic score and game sales for the sake of economic observers and investors - this is patent nonsense.
2. Because MetaCritic selects and weighs by its own discretion (for instance, speciality sites like Watch, Codex and Banshee are ignored), it is useless as a measurement of critical response.

Now he argues that in the case of EA, the trouble is EA communicating with its investors in showing a positive way forward - he thinks the fact that EA uses MetaCritic as a tool for this is nonsense because of the above.

That doesn't mean he thinks critical response is useless per se, he says that A) it is made more useless because MetaCritic doesn't function as it should (and he's right) and that B) when communicating with shareholders critical response is not a relevant measurement for long-term strategy. This last point is contentious, but basically true in most industries, though I'd feel unsure as to how valid it is for entertainment industries.

 

[Angthoron]

This last point is contentious, but basically true in most industries, though I'd feel unsure as to how valid it is for entertainment industries.

Seeing how popular things like Reality TV and various "forget-after-watching/playing-shiny-buckets" types of entertainment are, it's pretty valid. Mass production based on marketing department research is safer and generally more profiting than something that actually receives a fair bit of attention. Sure, the critics might like the latter better, but by and large, the critics tend to praise two-bit entertainment at least as much as they praise something that's actually praiseworthy. So why bother listening to critics while the sales figures keep soaring?

Now, after they plummet... But they won't, as seen with The Sims' expansion packs.

"What's on this afternoon?" he asked tiredly.

She didn't look up from her script again. "Well, this is a play comes on the wall-to-wall circuit in ten minutes. They mailed me my part this morning. I sent in some box-tops. They write the script with one part missing. It's a new idea. The home-maker, that's me, is the missing part. When it comes time for the missing lines, they all look at me out of the three walls and I say the lines: Here, for instance, the man says, 'What do you think of this whole idea, Helen?' And he looks at me sitting here centre stage, see? And I say, I say —" She paused and ran her finger under a line in the script. " 'I think that's fine!' And then they go on with the play until he says, 'Do you agree to that, Helen!' and I say, 'I sure do!' Isn't that fun, Guy?"

He stood in the hall looking at her.

"It's sure fun," she said.

"What's the play about?"

"I just told you. There are these people named Bob and Ruth and Helen."

"Oh."

"It's really fun. It'll be even more fun when we can afford to have the fourth wall installed. How long you figure before we save up and get the fourth wall torn out and a fourth wall-TV put in? It's only two thousand dollars."

Ray Bradbury, Fahrenheit 451

 

[denizsi]

Win for who?
This will make masses of retards buy shitty games encouraging the developers to make more shitty games.

Let's save the world.

 

[Brother None]

Seeing how popular things like Reality TV and various "forget-after-watching/playing-shiny-buckets" types of entertainment are, it's pretty valid. Mass production based on marketing department research is safer and generally more profiting than something that actually receives a fair bit of attention. Sure, the critics might like the latter better, but by and large, the critics tend to praise two-bit entertainment at least as much as they praise something that's actually praiseworthy. So why bother listening to critics while the sales figures keep soaring?

Eh. Bit of an odd example, since the big success story of reality shows aren't the ratings, they're the incredibly low costs, and the fact that you're rarely if ever tied down to any individuals.

Don't have that in gaming. You could argue Bethesda "release a game and let the fans fix it"-attitude is of the same sort.

 

[Mareus]

I just bookmarked that Metacritic site. If EA thinks they are selective for giving them lower scores then they must be doing the right thing.

There are still problems with the metacritic. They get their score by calculating the AVEREGE score of all scores on the sites, but they do not take into consideration many sites who do not have numerical scores in their reviews (like codex). Now, while this is better then just reading a review at Gamespot, it does nothing more than calculate the averege neglecting the objectivity of the score. This is a problem because if you take into cosideration that most sites have been dumbed down so that big companies can sell their products better, you still do not get the objective score. You just get the averege score which has nothing to do with the real value of the game. I used to just take into cosideration the user feedback scores, but lately even that section of metacritic has been dumbed down allowing many of the assholes to give scores 0/10 for really good games. It is my oppinion that no game should get 0/10 unless it really, really, really... really sucks. Even retardilion should not get a score 0/10 because it is just not argumentitive.