Stop messing up my searches for truly awful games with the merely mediocre! How else am I supposed to get my daily dose of schadenfreude?
One site shouldn’t have that much of an impact, since it’s offset by zillions of others handing out your standard 7-9 ratings.
Metacritic weighs some reviews more heavily than others, but even then, unless all of its “more important” sites have equally low ratings, it won’t have that much of an impact in the long run. If this review is first, though…
No need to “woot” us at all, as that died with the last CGW. This Dark Messiah review of Tom’s, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this “4” score is our score.
Yep, we got shit for taking the scores away, and now we’re in shit again for bringing them back and not abiding by the 7-9 scale! Yay!
Yes, I heard that Bill Gates personally ordered GFW to put scores back in, so he could add several points to all of the MGS products.
Letter grades don’t translate internationally. Even Canada has a different letter grade scale than the States.
Every opportunity that arises, eh?
On a scale from A to blue I rank this idea as a 3.
I dislike ratings, but if I were to choose a system, I’d go for a simple “Yes or No” system like Ebert’s, or Green/Amber/Red lights. The latter, I think, because I don’t think it’s often done.
Sellout! Bowing to the pressure of the man!
I forget who it was that used it, but my favorite scale ever has been “buy/rent/skip”.
Yeah yeah. You’ll give me shit no matter what I do!
I think they normalize the scores do you do not have that problem.
Personally, the only numbers that I really cared about in PCG or CGW (when they had stars) were either 90 and above (they really liked it) or below 50 (especially 20s and teens) because I love to read about games that were being trashed. I also enjoyed the CGW when the stars were all the same for every game. Is that worth something now?
Anything between that was all about what the reviewer wrote. Don’t get me wrong, I was always interested in what the reviewer said for any game that wasn’t a total dog. But essentially, I looked at numbers first for games that I wouldn’t play anyway. Games that I might be interested in were all about the reviews, not the numbers.
The reviews in any publication weren’t the end all and be all for me. Sometimes I picked up a game that got a bad review, and enjoyed it. But most of the time the reviewer and I agreed about a game.
Yeah, I didn’t like the CGW “we don’t do numbers, but here’s what a bunch of other guys gave this game” but I still bought it. Hell, I didn’t kill my subscription while Vederman was giving out Ann Landers advice either. And that pretty much SUCKED.
Guess I’m just a glutton for punishment.
Rock on, paper publications! You have one idiot that will keep buying your stuff.
Skip the scores all together. (And don’t do a half arsed job like CGW/GFW).
If you can’t be bothered to read the review then why buy the magazine?
Personally, I find that the magazines still have some good stuff to read, just not in the review/preview’s section (with very rare exceptions, notably PCG exclusives).
There is other stuff in these mags that are still worthwhile for me to keep a subscription, namely the opinion articles or other stuff (Tom Vs Bruce in CGW for example).
But as for relying them for reviews and most previews…the internet just has them beat.
Can I suggest this entire blog?
[INDENT]2005-12: Systems for Collective Choice
2005-12: Collective Choice: Rating Systems
2006-01: Collective Choice: Competitive Ranking Systems
2006-08: Using 5-Star Rating Systems
[/INDENT]
Using 5-Star Rating Systems
In Collective Choice: Rating Systems I discuss ratings scales of various sorts, from eBay’s 3-point scale to RPGnet’s double 5-point scale, and BoardGame Geek’s 10-point scale.
Of the various ratings scales, 5-point scales are probably the most common on the Internet. You can find them not just in my own RPGnet, but also on Amazon, Netflix, and iTunes, as well as many other sites and services. Unfortunately 5-point rating scales also face many challenges in their use, and different studies suggest different flaws with this particular methodology.
First, one study using Amazon data has shown that many undetailed ratings (where the rater isn’t required to add any additional information other than the rating they select) show a bimodal distribution. In other words the distribution of ratings tends to cluster around two different numbers (e.g., 1 and 5) rather than offering a normal distribution where the ratings cluster around a single height (e.g., 3). Thus the median of these ratings is not an accurate reflection of product quality, but instead is a statement of conflicting opinions.
My new project at work is something related, and it’s surprisingly complicated.
I think the blog entry on the 5-star scale ignores the problem of selective reviewing. Regarding his examples, fans of a show will always rate episodes highly because even the worst episode has the setting and characters that they like. Consumer reviews tend to be 1 or 5 because consumers only bother to post reviews when they’re either irate or enthusiastic.
Game reviews tend to skew upwards because video games have a significant technical component that affects rating, and the technical quality is likely to be high among those games that a magazine bothers to review in the first place. Some reviewers (like Tom Chick) actually do give a game a low rating if they just plain don’t like it, but most will set 3 stars as a baseline for a technically competent game, even if they think the gameplay is mediocre.
Whether one should do that or not is actually a difficult question. There’s a similar problem with reviews of action films. Many genre fans (myself included) will think that basically any action film is at least okay if the stunts and special effects are good, but newspaper reviewers will often completely ignore these aspects and rate all movies exclusively based on story, characterisation etc. Result: most action films get trashed indiscriminately. Is that a service to the readers?
Jeff Green, I give your post a score of 750.
Steve “the suit”, I give your snarky post a score of 50.
Do you guys want to guess top end of your respective scales, each has a different set of numbers…
Jeff, can you understand the confusion of just throwing those numbers out there with no context? I welcome the cutting of content from other sources, I welcome the return of scores, but the Tom’s 4 hangs out there like the numbers above, nonsensical in the current scoring system.
Chet
This isn’t entirely new. I’ve actually been doing this for CGW/GFW ever since they’ve asked us to go back to numerical ratings. Sword of the Stars and Dark Messiah were both a 4. Joint Task Force and Stronghold Legends were a 3. The Age III expansion was a 9.
I presume their other writers have as well. I know Shawn Elliott’s 6 for Battlefield 2142 is an example of that.
But I understand the confusion, so hopefully they can clarify – if they haven’t already – that their 1-10 scale is different from the typical 7-9. Because it probably would have saved me a fair amount of grief if I’d given Dark Messiah and Sword of the Stars the 6.blahblahblah I would have given them on a 7-9 scale.
Aww, who am I kidding? Once you go below a seven on any scale, you’re basically kicking puppies to anyone who digs that game. :)
-Tom
Are you honestly surprised? I guess I didn’t make my point very clear in the initial post but I’ll try again: I can understand if you don’t want to use grades at all, but I can not understand using a non-standard scale. As a matter of fact I can’t see why anyone would want a non-standard scale. Sure, there is no offical standard, but since the 1-10 scale has been used for quite some time in game reviews readers will have certain expectations. The only reason I can see for using a single numerical rating at all is for the reader to quickly and easily see roughly what the reviewer thought. If the reader thinks: “ok, a 4 is a 6 everywhere else meaning that a 7 is…?” then I think you’ve made a mistake.
When I read Tom’s review I thought the grade would be in the “flawed but with some redeeming quality, enjoyable for some” area, which usually means a 6/60. I was very surprised to see a 40, because that is normally the “crap, plain and simple” category. The main reason I was confused is of course because I just followed a direct link without first reading the review score manual, but I don’t see why anyone should have to.