Steel cage deathmatch fight: Tom Chick vs. Bruce Shelley

I have nothing against Bruce Shelley. Except that I think he’s too literal with his math.
Sometimes, three fifths is not the same as 60%.

Bruce’s explanation for some of the less enthusiastic reviews is amusingly bean counterish and a little tangential, but Sandy Petersen, game developer extraordinaire, knows the true cause is press bias against Ensemble:

Not fair! They liked TA and Myth more! And since we didn’t garner as many 9+ reviews, that must mean they hate us! followed by yet another appeal to the fans, who surely can recognize quality games better than those hateful, biased game reviewers. And let’s agree to ignore the fact that at least 2 of our 4 major releases are considered RTS classics.

Not only is he too literal with his math, but he’s also implying (or just outright saying) that disparate reviews suggest that one reviewer is off-base, and that sales are the best metric for a game’s achievement. Both of which are so absurd that I’m surprised to see them uttered by someone that I respect as much as Bruce Shelley.

Sandy Petersen’s bitching is even worse. Age3’s average score is 8.5? Oh, boo-fucking-hoo. However will you survive the Nerf slings and arrows of critics saying that they like your game, but then refusing to admit that it’s GAME OF THE YEAR… INSTANT CLASSIC?!? They might as well have given Age3 an average score of -15! And they liked Myth better! And Total Annihiliation! How could they say that we’re not better than two games that many consider to be among the best realtime strategy games ever made? If we can’t be the BEST RTS EVAR, then we’re taking our ball and going home!

It’s times like these that I miss Dr. Crypt.

So it did abso-frickin’-lutely not deserve 80%, either?

Aren’t 1/2 stars allowed?

Well, while I’m not sure I would go as far as to say “hate”, the Age franchise is generally and consistently undervalued in the press compared to the C&C and War/Starcraft franchises. Sometimes you see them talk about the “big two” RTS franchises and omit us altogether, despite the fact that we’re as big, if not bigger, as a franchise.

Also we continually wrangle with the fact that the press and fan boards do put a huge premium on innovation that is not matched by market enthusiasm or our actual fanbase that doesn’t post. So we continually get ripped for making decisions that are completely sensible ones to make, IMO. Please note I am not saying all of our decisions are defensible ones… we certainly make our share of mistakes. :)

Also Ben, I know it’s the stock in trade on message boards, but I think you’re representing both Sandy and Bruce a bit with a bit of overboard histrionics don’t you think?

I totally see Tom’s point with the review scores, but the fact of the matter is that a lot of people do look at things like Gamerankings and Metacritic, so it’s not like that disparity is entirely without consequence.

I don’t think even all the reviewers in game industry can come to an agreement as to how all the different rating systems translate in relation to each other.

Honestly? I think I’m cutting them more slack than their comments deserve. They are bent out of shape because they think Age3 should be garnering better reviews. But it’s not like people are panning the game, or even close to it. I don’t think I’ve seen a single review that didn’t give it a thumbs up, and several reviews have been extremely favorable. Don’t you think that bitching that your game “only” has an average review score of 8.5 is just a wee bit prima-donna-ish?

All right, you don’t have to answer that. I guess it’s a good thing that I didn’t review it for anyone, though. I liked it less than Tom did.

I totally see Tom’s point with the review scores, but the fact of the matter is that a lot of people do look at things like Gamerankings and Metacritic, so it’s not like that disparity is entirely without consequence.

What does that even mean? “Without consequence?” Are writers supposed to consult with each other prior to submitting their reviews to all the various publications, to make sure their scores all fall in line so that they don’t enact these mysterious and forboding consequences? What are these consequences, exactly, and what do you propose as a remedy for them?

That’s a fair point, but it’s also true that a lot of people look at the scores in one or two magazines they trust without looking at Gamerankings and Metacritic. And those people would not have been served if AOE3 had been given a 4 or 4 1/2 star rating: such a rating would imply the product was better than the reviewer believed it to be. This could all be solved if all game reviews used the same standardized format, but I hope we can all agree that would be the worst possible outcome for gamers and reviewers.

You don’t see major restaurants complaining that the Michelin guide “only” gave them two stars. Game makers need to calm down and realize that different publications use different standards. And they also need to realize that, when defending themselves from “sub-par” review scores, pointing out that the scores are not, in fact, sub-par is a much more effective tactic than going on the offensive and bitching out the reviewer for hating Microsoft.

I think I’m cutting them more slack than their comments deserve. They are bent out of shape because they think Age3 should be garnering better reviews.

Not even - they think Age3 should be getting better scores, which is the whole problem. No gamers really give a shit about the text of the reviews, just the single numerical value at the end. GameRankings just gives people what they want, but in my opinion that does a disservice to the writer AND the developers.

Itd be nice if people cared about the reviews, because a 6 is not a 3 is not a B-, but they dont. I happen to think its because of the generally shoddy quality of videogame writing (does anyone actually READ those 3000 word reviews on IGN?), and Bruce Shelley is just doing the same thing most gamers do when we are reading reviews.

This is probably why Metacritic has developed some kind of “secret formula” that weights their average, as opposed to Game Rankings which does a “real” average.

I don’t know about news, previews, etc, but the reviews for your games, (courtesy of Gamerankings, Bruce’s favorite site) contradict your claim that the series is undervalued. Let’s take a quick look at your two best:

Age of Mythology: 90% 1/3rd of the scores below 90, bottoming out with 3 70s. 5th best RTS on the PC.

Age of Empires 2: 92% less than 1/4th of the scores below 90, bottoming out with 1 70. 4rd best RTS on the PC.

Age 1 has an 87%, and 3 has an 82%. Still not looking undervalued to me.

Personally, I don’t care for the all the aggregating, but Bruce, Sandy, and many gamers unfortunately do. Of course, many of the issues in this thread would go away if gamers would actually, you know, read the review text instead of just compulsively aggregating scores. But apparently that’s too difficult or time consuming or something, so, until then, best of luck with the numbers game.

Edit: jfletch has a point. Video game writing is, on the whole, quite shitty. This certainly contributes to the numbers game.

I agree with Tom more than I disagree. The GameRanking system is a silly way to equalize scores, and Tom was quite clear in his review & on the boards that parts of Age 3 were great, and parts were problematic, so the whole beast was tough to score. I don’t understand Petersen’s bitching, since CGW gave the same score to the original Age of Empires (the one everyone was biased against because of M$) as they did Myth and Total Annihilation (4.5 for all three games).

However, I believe Tom overstates the strength of a 3-star rating. If we convert the stars to a population percentage (e.g., a score of 70% means that 70% of all games reviewed by the magazine got that score or lower), a 3-star CGW rating is equivalent to 48%. Maybe Tom gives Age 3 a qualitative “thumbs up,” but he assigned a score in the bottom half of the rankings. By comparison, PC Gamer’s 91 ranking becomes 95% on a population basis. Most other reviews fell somewhere between these two extremes. For what it’s worth, my database of reviews has Age 3 at 91% on a population basis, and includes both CGW & PC Gamer.

What have we learned?

  1. Ensemble made a well-crafted game; it probably won’t be an enduring classic with most gamers, but all RTS fans will check it out. Overall reviews reflect that, but there’s variation. You actually have to read the text to see why there’s variation.
  2. Ensemble’s designers, like all humans, seem to minimize their contribution to their perceived failure and would rather externalize the problem. I’ve been a subscriber to all three PC magazines for years, and I don’t see a press bias against the Age of series. It’s clear to me C&C has been third place since Age of Empires 2 came out, and Red Alert faded.
  3. Tom gives average scores but believes they are good. Tom may still be confused why he and his partner had a fight once when he replied, “You look fine, honey. I give the outfit 3 out of 5 stars!” to an innocent question before a dinner date.

Considering that reviews are personal opinions whereas sales are at least a factual representation of something, I’m not sure I’d consider the two as equal extremes. The sad, but true, reality is that critics in many fields discount mass popularity or even view it as a negative. If it appeals widely, they reason, it must have sacrificed something to be successful with the “lowest common denominator.” The only way to really know how something will be viewed long-term is to wait for the long-term.

As such I can see Shelley’s frustration. After he and his team spent years on a game that is selling extremely well it should not be a surprise that he feels some reviews are off base. He’s suffering from the same desires that often aflict popular movie stars - having won the box office they now wish to win the critics.

Good analysis, Sidd. I want to see how Tom replies to point #3. ;)

They must have gotten some crap from Microsoft for the score average.

Holy crikey!

So a three star score is in the lower 48% of rankings however it’s still a thumbs up?

Has the game business just become Hollywoodised and no one is brave enough to say they thought something sucked anymore?

Translation of Tom’s article into Hollywoodese:

Hey Shelly baby I loved your game LOVED it, yeah I was a bit critical of the bad parts you know the beginning, the middle and the end but baby can I tell you have talent! Yeah that most dissapointing game of the year stuff that’s just all talk baby, I can’t wait to see your new project. Don’t call me I’ll call you, Ciao (kisses Bruce on both cheeks).

The Gamerankings thing really is sad. A lot of stock is put in how that number turns out. Sometimes quite literally. Actually, it’s not sad so much as it’s fucking pathetic. I would be happy if Gamerankings just up and vanished one night.

Xemu, thanks for chiming in. I am mystified that Ensemble feels the Age games are underappreciated. You guys aren’t mentioned among the founding fathers (Warcraft and C&C), but you’re easily among the best of the next wave and one of the biggest and best perpetrators of the genre as a whole. I five-starred Age of Mythology without a second thought! That’s a 100% according to Gamerankings, and I bet you guys weren’t scratching your heads over the disparity between my 100% and whatever the guys at PC Gamer gave it. :)

Seriously, though, I understand being disappointed at 3 stars. You guys did a lot of awesome work with Age III. I wish certain things were better. Even more, I wish some of the patching had gone differently. But that’s neither here nor there, and I know it’s not even your project.

Sidd Budd’s math makes my head heart even more than Bruce Shelley’s math. Why on earth would you convert a rating to a population percentage?

I’m not sure how I’m “[overstating] the strength of a 3-star rating” when I point out that I consider it a thumbs up. I have nothing to do with the rest of the ratings, so the fact that the game scores below the average for the rest of the reviews in an issue means beans to me.

Go peddle your fancy statistics on Gamerankings! :)

  1. Tom gives average scores but believes they are good. Tom may still be confused why he and his partner had a fight once when he replied, “You look fine, honey. I give the outfit 3 out of 5 stars!” to an innocent question before a dinner date.

Dude, please. Give me some credit. I know enough to know when you’re supposed to inflate a grade.

-Tom

I don’t know about anyone else, but I don’t consider a 3 star review a thumbs up. It’s right there in the middle and, I’d even say, it’s at a risk of falling off the 2 star ledge (2 star = never touch this game; it’s evil). 3 stars is a neither a thumbs up or thumbs down. It’s ho-hum. It’s totally and completely average. In that I don’t have time for average games, it’s almost a negative review in that respect.

That said, the moaning by Bruce Shelly seems disingenuous, particular when he uses sales numbers to prop himself up. Maybe he’s been talking to Tom Brady. “We don’t get any respect.”

I consider 3 to be “above average” - it’s a qualified endorsement. It’s also a score that means that I need to read the review very closely. Why does he/she not recommend it stronger? Are the concerns the type of things that can be fixed in a patch? 3-stars means “CAUTION: I liked this, but have some concerns.”

Re the innovation argument, I endorsed AoE3 with a good score (on a hundred point scale) but did take points off for lack of real innovation mostly because it seemed a step back from AoM. It wasn’t that Ensemble didn’t do new things; it was that they went back to doing the old ones. In many ways, it seemed a retreat from the advances that helped pioneer in AoK and AoM.

I can almost forgive pretending that Rise of Nations never happened. But to ignore your own history?

Troy