You obviously didn’t read my post long enough. Unless that’s out of 50, in which case, you speak the absolute truth.
It’s curious that the game press seems to be the only entertainment press using these weird and differing review systems. Almost all movie reviewers use 5-stars or letter grades (which pretty much perfectly map to each other). I can’t remember ever seeing a movie or music review using a percentage system. (I’m sure someone will find one, however.) Why did anyone feel the need to invent a different system in the first place?
I still contend that there’s a tendency for people to think you can quantify game reviews because the games themselves are so much about math. (And all the people playing and reviewing the games probably have strong computer/math backgrounds.) So, with a bigger number you have more variations, which means more elements were considered to reach that final number. Or something.
Like there’s really a difference between a game that gets 92% or 9.2 and one that gets 88% or 8.8. The actual gap between those ratings is practically a rounding error, but most people would probably say there’s a meaningful, quantifiable difference between the two scores.
While I have no intention of dropping ratings since people actually like them, I think one side effect of dropping ratings is that almost every review would read like a list of why a game sucks, even the great ones.
One thing giving a rating forces the writer to do is better justify their overall opinion, as in the text has to match the rating. The only time I send reviews back to writers is when I don’t feel they match. At that point, they have two options: change the text, or change the rating. If they think it’s a 4.5 star game and the text is too negative, either downplay the negatives, bump up the positives, or drop it to 4-stars or 3.5-stars.
If you pay close attention, most reviews fixate on the negatives of a game, I suspect as a way to avoid criticism. It’s also a lot harder to write incredibly positive reviews. We’re so scared of being labeled fanboys that we only show true passion for something when we hate it. (Also, you can be a lot funnier when trashing something.)
So, even in a 5-star-equivalent, non-rated review, I’m guessing the text would be 30% positive, 70% negative. Just the positive parts would be “wow, this is great,” and the negative would be downplayed.
But the reader would still walk away with, “Holy crap, that’s not a great game.”
I like the 7-9 scale because it’s intuitive and ingrained in our brains since youth (and apparently more places than just the US). Switching to a scale where 5 is the “average” score is really confusing. A-F, 5 stars or 7-9 are standard, understand and straightforward.
I’m with Steve on this.
When I got to my position I inherited a 1-10 grade that actually try to use the entire range. Our sister mag about digital photography has just introduced their ‘new and improved test’ which is a 1-10 system weighing heavily on the 7-9 part of the scale. They use decimals, so it’s really a percentage scale and right now I’m editing a camera test from their magazine to my format and even with the expanded, technical and long explanation I really don’t se how any expert can distinguish between 10 cameras ranging in picture quality from 60% to 82%. That is I see the difference between the 6 and the 8, but not the many cameras ranging between 7,9 and 8,2.
And games are even less quantifiable than cameras imho.
But geeks like their numbers and the worst offenders are those sites that put everything into subcategories to give everything an air of science.
And then there’s the whole aggreate score calculation…
Oh god yes, absolutely I see the confusion. We spent our entire podcast this week talking about it. It would appear to be a completely Quixotic venture, in fact–especially without the context, yes. The context is there in the print mag, and I know it will be there soon (if not already) online. But with or without an “explanation”, the idea that we can actually fight the accepted wisdom that anything below a 7 is “D” or “F” is going to require endless explanation. I don’t know if that makes us “wrong”. But good god is this tiring. My ideal solution? Letter grades.
What exactly was wrong with the 5-star system? Why would letter grades somehow be better, especially since the stars match up perfectly? (The only other rating system I’d consider would be letter grades, not because of matching up but because they have an obvious real-world corollary that requires no explanation.)
I suppose wanting to normalize ratings with other Ziff publications makes sense, but it seems like those other guys should have to justify their own ratings systems instead of changing the one that most reviews in all other forms of entertainment uses.
I like the grades better because you can do B+/-, for instance, and you know it’s still a B rating either way. A similar star rating would have to be 4.5 or… I don’t know, 4.1?
And to answer your other question–you answered it. It was about getting in line with the other Ziff publications. So, the scoring system we’re using is the same one shared by EGM, Official Playstation Magazine, and 1UP.com.
Must say first that I whole-heartedly agree with the first post - the point of a score is to communicate something to the reader, and redefining the vocabulary you use to do that (even if it’s explained elsewhere) is a failure to do that. And…
…the whole point of a magazine is that you don’t have to read it all to get some value out of it. Even more so with a website. 90% of the audience are there to skim, and dip into something that catches their interest. With no scores, nothing does - they have to base that decision on information gleaned elsewhere or the superficialities of screenshots.
I haven’t been here long enough to be familiar with Drastic’s 1,400 other posts, but I really really like the idea that he might just jump into a thread whenever wild hyperbole is called for to make a point.
I’m with you on not knowing what the difference between a 7.9 picture quality and an 8.2 is, but using that scale on things as huge, complex and artistic as games is completely the opposite. There’s a gaping chasm between those two, particularly since so many games fall in that upper quartile of the scale.
Just quickly, I’d like to bitch for a sentence or two at all scoring systems involving fractions. If you need finer distinctions, why the hell did you pick a scoring system with only three/five/ten options? There’s a site that marks out of twenty, but allows quarter-points. WHAT.
ANYWAY: Most games reviewed in our mag land in the upper half of the score bracket, but we do use the full scale. We have 40% games, 30% games, 20% games, 10% games, all the way down to 1%, which we’ve only awarded once. In fact we’ve even used zero, in that we gave a game N/A on the grounds that it quite literally wasn’t playable. A 50% in our mag says don’t buy it, because who wants to spend £30 and hours of their life on something mediocre when there are so many 70-90 games available? But it doesn’t mean it’s worthless, and it doesn’t mean there isn’t an interesting difference between that and the 1% game. The degrees and flavours of awfulness unrecommended games come in are some of the richest sources of amusement in games writing, and we need that half of the scale to rip them apart all over it.
For me personally, there’s also a modest principle at stake. Brave games that are ruined by a few flaws like The Ship (61%) shouldn’t be lumped in with utterly worthless cash-ins like the Big Brother game (1%), or offensively egregious date-rape sims like Leisure Suit Larry: Magna Cum Laude (3%). And if you stretch the 7-9 scale to fill all the numbers, that’s what happens. The many talented people who did great work on it have earnt the right not to be kicked, and I’m glad our scoring system let me reflect that while I didn’t recommend buying it, these guys are doing interesting things that could become great things down the line.
Just heard the GFW podcast and I must say that you guys are fighting a losing battle. No one cares about average or even above average. People want a good game (“B” grade) or a great game (“A” grade). Anything less is somewhat of a failure. I do agree with you Jeff that letter grades are the best way to go IMO.
Well, yeah. On a 7-9 scale, a 6 is off the bottom–complete crap, don’t even think of going there. Of course someone who thinks the game isn’t utter garbage will disagree with a less-than-seven rating.
Daily Radar - Hit, Direct Hit, Miss, and poop (whatever it was called).
A variation on the whole number 1-5 scale. Since virtually every site or mag I have read that uses that scale uses the whole scale, I prefer that one. No mag or site will ever use the full 1-10 scale, sure they may make some fancy pronouncements about doing it but eventually their learned behavior from school will kick in and it is back to 7-9. It’s 7-9 for a reason, after all.
Direct Hit - Everyone should buy this game
Hit - Buy the game if you have an interest in the genre or subject matter
Miss - Rent or buy if you’re very interested in the subject matter but beware
Poop - Avoid at all costs
