7-9 scale - the discussion

Games
1

Where is honest game reviewing now?

The “7-9” scale being talked about is definately in effect at the second most influencial, and, unfortunately, “respected”, game review site on the internet (IGN). Several other lesser sites follow this ideal. It sickens me that sites will give a complete and utter shit game, that everyone agrees sucks completely, a 7.0 or higher for god knows what reason (Path of Neo 7.8? True Crime NYC 7.8?!?! What the hell is going on here?)

The only consistantly trustworthy sites left are, in my opinion, Gamespot and Gamespy. Gamespot is, de facto, the end-all-be-all of console game scores. They hit the mark 90% of the time. The PC reviews, however, are a little hit or miss (Vietcong, for example, was an amazing PC game- and they nitpicked every minor problem with it and made it out to be crap). Gamespy seems a bit better rounded handling PC review scores.

There are other sites out there, such as Eurogamer, who judge games harshly. Unfortunately, Eurogamer is almost too harsh in certain instances. However, if Eurogamer says a game is good, it most likely is. I don’t think I’ve ever disagreed with a Eurogamer 8-9/10 score.

Yet, Eurogamer is sometimes so overly critical of games that they miss out on who may actually enjoy the game. I can’t cite an example right now, but there have been a few games that Eurogamer has slammed that I and my friends/coworkers have found extremely enjoyable.

The only happy medium I’ve seen is CGM, mainly with their PC game scores.

They are extremely critical of games, and spell out exactly why a consumer would or would not like the game. They also weigh each side as equally to as possible, allowing in that someone may enjoy the game even though the reviewer didn’t.

CGM’s console reviews are great in the fact that they cater to a PC audience, and try to break down if a PC gamer would be interested in the game. Unfortunately, I sometimes find them a bit harsh in shutting out certain games to “protect” the PC audience from something foreign.

In summary, I find these sites/magazines the only respectable reviews left on the internet/in print-

Most important part of the post

From most respecable to least:

CGM
Gamespot
Gamespy
Eurogamer
Yahoo!Games! (GamesDomain?)
OPM
OXM
1Up
CGW
VoodooExtreme
EvilAvatar
ConsoleGold (whatever it’s called now)
PCGamer
IGN
GamePro
MaximOnline

2

And this is why I like the meta-ranking sites like Gamerankings.

Not only do you get a good “aggregate picture”, it’s also easy to see how much each site varies from the average score. These stats confirm that certain sites (IGN among them) are usually scoring games higher than the average of all reviews, and allows you to discount the offendingly high scoring reviews.

3

I basically only use the meta sites now. Individual reviews are too variable, and I’m usually only interested in finding out whether something is a plausible purchase for the price. (To some extent, the cheaper it is, the lower an aggregate score I will tolerate.)

4

I don’t think we’re trying to protect the PC audience (except maybe financially), just looking at it from that perspective as a PC gamer. All too often there are console games that get very high review scores in console mags because those folks simply haven’t even played a PC game in their lives. Either that or they do so infrequently they’re unaware of a partcular console game being super-derivative of something on the PC.

I mean, for a PC gamer, something like Rainbow Six: Lockdown, that I reviewed in last month’s mag, just isn’t going to cut it at all. It’s dull. That’s true for me as well since I am a PC gamer as well as a console one. I guess that’s probably the best way to view those reviews… the guys writing them are hardcore PC and console gamers so you know we’re coming at it from the perspective of someone pretty knowledgeable of all forms of gaming.

I’m sure Tom or Steve can tell you more if they pop in here.

–Dave

5

How about we, the people of this board who I consider very bright individuals for the most part, pay less attention at the numerical value of a review and more attention on the actual text of the review.

6

I already do that. :)

I’ve started buying just about every game magazine on the market. I like to know who’s doing what and how well. I think it’s also helping me be a better writer. With subscriptions so cheap for most print mags and even cheaper if you get them through Zinio (the Ziff ones), there’s no reason someone like me can’t stay informed.

–Dave

7

Just a random comment to the original post: I have no idea how you could have come away from my Vietcong review having inferred I “made it out to be crap.” I really liked that game. Maybe you’re referring to the sequel or the expansion pack. Thanks for the kind words about GameSpot’s reviews, though. With the so-called next generation of games upon us, I think most publications are taking the opportunity to reevaluate the way they review games, so hopefully readers will see a positive trend in the next year. Ron Gilbert’s blog at grumpygamer.com recently made a very interesting point that the quality of game reviews may improve as the content of the games themselves does. A cerebral game like Killer7 forces a critic to write about it differently than some typical FPS, so as long as game makers continue to produce interesting stuff, I have faith that the quality of game writing will steadily increase as well.

8

Hey Greg, whatever happened to that New Thing in Game Ratings you said you couldn’t talk about yet a while ago?

9

Errm, HonestGamers and Digital Entertainmentand News and that’s about it I’m afraid.

The rest are really terrible.

I give GameSpot a 2.2/10.0.

10

The problem with the 10 point scale is that no one uses a 5 as the baseline for an average game. Five point scales tend to be a bit better about the mid-point being the score for an average game.

Just think how many two-star games and even 1-star games you see scored at places that use a 5 star game. On a 10-point scale those scores would be 20s and 40s – something you almost never see.

11

The site that has these gems from its Blitzkrieg II review?

As good as consoles are for other genres of games, the PC’s gaming acumen seems solidified by the fact that war simulations are irreversibly connected to the platform.

If you were asked to think of another war-based RTS game, chances are that you would choose the Command and Conquer series. Blitzkrieg 2 is the complete opposite to that series, which isn’t necessarily a bad thing.

If you want to be successful in this game, you need to be tactically astute. You are given a small core of units, which it is your mission to command them.

Troy

12

Well, you gave it a 7.9, which is on the lower end of the 7-9 scale.

13

Well, yes and DEN does have awful layout. But they take their time to actually play the games, which is more than can be said for most sites, and the ratings seem far less arbitrary. They also appear to have the reviewers review games in genres they like.

If I read another review which starts with, “I hated the first one, and this one is even worse” I’m going to scream. That means you shouldn’t be reviewing it. That’s like sending a Fangoria reviewer to Pride and Prejudice: “No gore, lots of irritating chicks giggling and fruity accents. 2/10”

14

I’m not sure that this is unique. Every place that I have written for has asked what genres I felt most comfortable writing about and I’ve rarely written about any genre I clearly stink at (like FPS).

As I read the terrible opening paragraph of DEN’s AoE 3 review, I suspected that maybe it was just this one writer.

Then I read another writer claim that Act of War has an unlimited resource model and cites the oil wells as an example. In fact, the wells are finite in capacity and can be exhausted. Only prisoners can bring unlimited funds, but slowly.

And another refers to Imperial Glory as yet another military RTS set in Napoleonic Europe. I had no idea we were drowning in them…

And someone gave Superpower 2 a 7.8.

I think I’m done with DEN.

Troy

15

7.9
good

16

Eh? I think Gamespy’s reviews are some of the best among the websites, but I think there’s this thing that hardcore gamers have against them because Gamespy will forever be associated with the product that started the whole thing. People aren’t willing to take their reviews seriously because of that.

I can tell you that I’ve been working with all of the writers at GamerDad (I’m the EIC over there) to try to improve all their writing and one guy in particular who clearly plays the games thoroughly, but had a lot of trouble getting it into text, has improved dramatically over this past year. For those that aren’t aware, our reviews are no different than any other site when it comes to the main body, we just tack on the parental part at the end in the Kid Factor. So don’t feel like you’re not welcome if you’re not a parent.

Anyway, everywhere I’ve ever worked with has put people on the reivews that like the genre or are even fans of the games. That probably leads to some score inflation sometimes, but it at least gets you well-reasoned text when it comes to the good and bad because the people want to play those games in the first place.

–Dave

17

7.9 is above average for Gamespot; only 30% of all their games get a score of 7.9 or higher. Not all sites are equal when it comes to the 7-9 stereotype.

EDIT: Removed inaccurate reference to PC Gamer’s revision to their scoring system. I thought they had changed the descriptions given to the different percentages, to encourage more stringent scoring. Upon closer inspection, it appears they didn’t change the old scoring system, but merely added a graphical bar (like a horizontal thermometer), to their reviews for easier visual discrimination

18

I was thinking about this last night, and I had an incredibly brilliant idea so hang on to your seats, ladies and gentlemen.

OK. Get ready.

A normalized rating scale. Not a shodily normalization like the aggregate sites do, but a within-site normalization that doesn’t even give a raw score, merely how many standard deviations above or below that site’s mean the game is. Reviewers could be given any scale they wanted to use, since all scores would be statiscally comparitive, score inflation of all games as a whole could not happen.

So? How do you like them tasty delicious Granny Smith apples?

19

Well, you gave it a 7.9, which is on the lower end of the 7-9 scale.[/quote]

I can’t imagine how anyone could extract what he really thinks of the game from that review. Most of it is just carefully neutral description of game features–the sort of thing you’d expect to find in a preview, or bullet-pointed on the game box–without any commentary on whether he thinks those features hurt or help the gameplay, let alone how. The few statements of opinion that are there are often concealed in mind-achingly labyrinthine prose full of qualifiers (that mostly boil down to something along the lines of “it’s the sort of thing you’ll probably like, if you like that sort of thing”). I mean, what am I supposed to make of statements like this:

The events of the campaign are predictable enough in and of themselves, but they may take you by surprise as they come up, and they’re the sorts of experiences you’d probably hope to get out of a first-person shooter set during the Vietnam War.

So the campaign is predictable yet surprising, and probably what you’d expect, depending on your expectations. That’s illuminating.

I had to read this sentence three times just to figure out what he was trying to say:

it’s possible to get Vietcong running smoothly if it doesn’t run smoothly on its own, though at this point it’s not unreasonable to think that PC games shouldn’t require this sort of fine-tuning.

I’m surprised that he didn’t add that it’s also not unreasonable to not think that games shouldn’t require this sort of fine tuning, depending on what you think. Or not.

There are a few genuine bits of critical commentary in there, but if you trimmed the review down to just those, it would only be about two paragraphs long. And it would be better for it.

20

Every time I see a “may” or “might” or “hope” or “probably” in a GamerDad submission lately, I cut it right the fuck out. I hate those words in reviews and I’m trying hard not to use them myself.

–Dave