| Title | Kotaku calls out Quarter to Three for aberrant review scores |
| Author | Tom Chick |
| Posted in | Games |
| When | April 11, 2013 |
I was fortunate enough to be part of Jason Schreier's thoughtfully written and incendiarily titled article on Kotaku, Metacritic Matters: How Review Scores Hurt Videogames..
Read the full articleSessler has a point, even if he might be making it with a little too much drama. A publisher using Metacritic or any aggregate as an objective measure of the quality of a product is unfair, and just plain stupid on their part. I would not be happy with my review work being used for that purpose; I don't blame him for feeling that way, as well.
Simple and fun solution: get a large number of critics to agree to write what they actually feel in the body of the review, but give every game a perfect score for a quarter or two. Once you demonstrate that the pot can be poisoned like that, the publishers will stop paying attention.
I read the Kotaku article earlier and knew you'd have some opinion of it, Tom. Glad to see you respond here.
Tom, I respect your opinion and I agree with the two core beliefs you listed. It should be obvious on its face that an aggregation of opinions doesn't get you anything but some kind of stew. However I do believe that you (and that's plural you, anyone who reviews games) are playing with fire just by using numeric or numeric equivalent scores. You're taking something subjective and assigning to it an objective constant value that really doesn't apply to any scale beyond the arbitrary one that every reviewer carries in his or her head. And as funny as I think the comments on your reviews can get, I can almost wrap my head around how someone could be confused thinking that over a long enough time and with enough inputs, a game's score must be asymptotically approaching some general opinion value "x." What that means, though, is where I diverge from them.
It makes me sad that the same people who saw your HALO 4 review and flooded in to leave their mark will never read this.
If you think of video gaming review scores as analogous to grades in school, the system makes perfect sense. 90s and above are A's and represent outstanding work. 80-90 is a solid effort. 70-80 is an acceptable output that means your work is acceptable but completely unexceptional. And below 70 or 60, depending on the education level (or caliber of game you are expected to deliver), you start failing.
There is a certain amount of work that you have to put in before something becomes an acceptable 7. Most AAA games fulfill these baseline requirements of competence and polish. There's metric tonnes of shovelware that gets released every year that most publications don't bother to review. These would get F's if they were reviewed, but everybody knows they are shit, so it's not worth it. Once you achieve competence, then you start getting judged on originality, innovation, story, balance, etc. That's where your 70-100 ranges separates the C students from A+ students.
So I don't really have a problem with the scoring system in wide use. I personally wouldn't want to ascribe a score to a game like Year Walk, but it's a convenient shorthand for seeing what people think of a game.
I just wish AAA games had better writing and stories, so that a run of the mill shooter like Bioshock Infinite didn't garner 9s and 10s by virtue of the fact that its story isn't about aliens or war or war with aliens.
+1. Your reviews make sense because you explain your opinion, Tom. It's why I come to QtT daily. Thank you for making your voice a part of the conversation and for speaking your mind.
Tom, you're like the gaming press' Lord Varys: widely distrusted, but quietly trying to save the realm.
Your last paragraph reads as though you believe Adam Sessler and yourself to be at odds in your arguments, but after listening to his most video on the subject and reading this post I don't actually see much conflict between your ideas. You [apparently] want a change in how critics score/review games and how consumers process those scores/reviews. Sessler [apparently] wants the way game developers react to game scores/reviews to change. Those changes don't seem to be at odds with each other.
This is what is wrong with video game reviewing these days:
http://www.metacritic.com/game...
Critic score: 82/100
User score: 4.2/10
Actually, the H4 review, linked from metacritic, is what got me reading Qt3
These products are not submitted for the approval of critics in the same way school assignments are submitted for grading (in fact the whole analogy is actually fairly disgusting.) Metacritic is a tool for consumer usage, not a teacher's rubber stamp.
The vast majority of books, films, and music that have been produced and made available have all undergone their medium's respective rigmarole of elbow grease to get to a point where they're sold to you (in other words, that mysterious baseline of having become “acceptable”), and yet these media don't get free passes just for showing up. Not every movie you see in a theatre is three stars out of five just because its makers jumped through all the necessary hoops to get it on the silver screen. Not every song composed is a good song, nor every book published a good book.
And yet in normal conversation about other media, there is no hesitation to voice your opinion. No one says, “I hated this movie, but I still rate it well because it was 100 minutes long, has a beginning middle and end, and lots of people worked really hard on it.” No one hates a book only to begrudgingly reflect favourably just because it was 400 pages long and was ultimately intelligible. Why should I hate a video game and then adjust my appraisal because it fulfils an arbitrary checklist of quality?
I don't like Bioshock Infinite at all, but I don't give it a pass just because it's “competent and polished.” Most big budget Hollywood blockbusters are “competent and polished,” but they don't deserve high marks unconditionally just because somebody threw around enough money to make them functional products. FM radio is competent and polished, my mother's harlequin romance novels are competent and polished, the stuff lining the walls of the Louvre is competent and polished, and neither I nor anybody else has to like it automatically just because it ticks all the boxes of a “competent and polished” product.
(EDIT: Funnily enough, I seem to recall Mr. Chick's review of Fallout: New Vegas to be fairly positive, only to be branded with the lowest available score because of its bugs. I'm sort of curious as to where that review falls on the continuum of critically thinking/writing about games, since that score seems to have reflected the experience with the game as a consumer product and not the subjective experience. By contrast, fast forward to March 2013 and let's pretend that Tom actually adored Sim City: would its score reflect the 1/5 consumer experience or the personal 5/5 subjective experience?
Oh dear, I'm fretting about reviewer scores far too much now!)
Don't give a rating. Rock Paper Shotgun doesn't. Let the words tell the reader all they need to.
I wonder if this has something to do with gamers feeling looked down upon by the culture at large. I've noticed a certain "we must support the developers" fellow-feeling kind of thing...similarly, comic book fans who feel they must "support the industry" even when their industry mostly produces junk. They get an A for Effort, even when their product is C-quality at best.
Yes, you're basically just stating your opinion when you review games - which is fine.
The issue is that you're letting your immediate emotional response rule - rather than being fair about it. Sometimes, it helps to sit back and think about things in a larger perspective.
No one can claim to be fully objective - and it's very obvious that you're not trying to be. But I do think a useful review needs to strive for the objective - even as out of reach as that may be, especially if your knowledge on the subject is limited.
Yes, you're reviewing "video games" - but that doesn't mean you can't take them seriously or that you don't need to think things through, rather than letting your emotional response take over.
Interestingly, you do put out the occasional competent review, meaning a review where you've actually thought things through and where you're not going for the controversial opinion you know will get you the attention you so obviously crave. So, it's not so much a matter of you being incompetent - but you being too much of an attention whore. At least, that's what you seem to be - to me.
A useful review needs to "strive for the objective"?
Are you an idiot, or are you just unable to read the article you're responding to? I'm sorry tomchick doesn't like some game you like and you're all butthurt about it, but please crawl back into whatever Halo review comments section you crawled out of.
Tom I've always thought of you as my Pauline Kael. Perfect pic for this piece.
A wonderful read as always Mr.Chick. I do applaud you for using the full scale when reviewing games or speaking your mind freely for that matter. I always believe that if any medium's criticism is to be taken maturely, then I believe one should be able to express their opinions freely. Me and a friend of mine write articles detailing our headaches and heartaches so I do understand the notion of being able to express with the utmost intention of allowing creativity in the medium to emerge (I would give the name of the website that my friend had setup but that would be kind of embarrassing compared to what you write).
However the problem with using scores, is not that publishers or readers may use it with idiotic intent. But most perhaps; I feel that a subjective opinion cannot be represented in an objective manner whatsoever. Which is why, when critics rate films and music using scores instead of critical insight, I feel they may have missed the point somewhat. Nonetheless, I do support both your yearning for more heartfelt criticism and Sessler's stance on the abuse of using such scales.
" how many reviewers pretend ridiculously towards objective insight
measured on a bell curve that spots a game 60 points for just showing
up, but deprives them of 100 points for not being perfect."
Or it's automatically over 80 because of the hype it generated before release, like Spore or Dragon Age 2, which is even more pathetic.