I’m not disagreeing with that.
But I also disagree with the statement that reviewers are somehow obligated to review release code, especially when there’s a day 1 or week 1 patch available that changes things massively. I know that even in America broadband is now used by the majority of people, and I suspect the ratio is even higher for gamers.
Cory may well feel that developers need to be punished for releasing code that they know is buggy, and this is a sentiment I sympathize with, and certainly some reviewers are free to adopt this policy, but I think in general it does gamers a disservice. It’s also unrealistic to set a deadline on reviews.
arakyd
3842
What is this formal review you speak of? Does it follow some sort of rule set, as the word “formal” implies? In a “formal review” do you, for example, actually make it clear how many hours you spent with the game? Can your readers actually expect to see something like that in the future? Because in that “Impressions Of,” there is no clear indication of how much time you’ve spent with the game. What I see is a lot of statements about the game as a whole, not one of which is positive, and a last sentence implying that you aren’t going to be playing it again for a while. It reads exactly like a final judgement of the game in its current state, in the form of a string of pulled punches.
I don’t know if this style of writing comes from being funded primarily by game advertising or what, and I don’t really care. I don’t read RPS any more, and posts like this are one reason why. I would rate it as “a bit shoddy” and “unfinished” both. Maybe you can make a profit that way. Brad Wardell claims to have made a profit with Elemental. I just wonder how long such products will last in the market. My guess is that they hang on for a long time, as the market itself continues to decline. A depressing prospect, but maybe it will create some room for stuff that’s actually good.
Cubit
3843
What part of the word “impressions” is hard for you to understand, arakyd?
Gamespot touched on this, though the reviews morality of that site are certainly questionable. The back story was one of the paintball FPS games got a horrible review. The developer sent a missive that they should re-review as a patch was forthcoming. Gamespot’s response was that reviews were based off the game at time of delivery.
My interpretation of this was upon the day it hit street, but pushing out patches on day 1 is far more common now than then. But my take is the game is reviewed based on whenever the reviewer gets around to it. It is not like they should play a game sans a patch if one is available.
The lack significant of reviews for Elemental is confounding.
Oghier
3845
There was a comment to the RPS ‘Impressions’ article that I thought quite apt:
So… what’s the point of playing a fantasy game, if it’s not crazy? That’s the good part of these type of games! Being a undead Lych crushing life with your zombie minions, or striking hard in enemy territory with terrorist elves troops, or making a magical ring with invisibility gifts in your forge, or summoning elder gods from the sea, or creating a godlike being. Stuff that makes you go “woahh! awesome” and makes you forget about the bugs and balance problems.
This was why people loved MoM, despite the braindead AI. It had so many over-the-top, awesome strategies and moments that you forgave its shortcomings. Everything was overpowered.
Along with all the basics that have to be addressed, I hope the revamp to the magic system isn’t afraid to climb way out on some limbs.
I agree that citing lack of internet access isn’t really a good reason for reviewing release day code. I’m not the one making that argument.
Why is it unrealistic? You should hold back because the developer or publisher asks nicely? No deadline at all? Really??
Er, most game review sites policies say they only review gold masters and not patched versions. So that does mean that the editors agree. For example, GameSpot has a “After the Fact” review that will review patched versions which is considered a different review than the gold master review. http://www.gamespot.com/misc/reviewguidelines.html
Here’s IGN’s policy (“IGN only reviews retail product or games that have been deemed “final gold code” by their publishers”) http://games.ign.com/ratings.html
Heck, 1up alluded to their review policy in the Elemental review itself. I know PC Gamer’s policy is to only review Gold code. I know it was GameDomain’s policy.
Is it “punishing” a game to review a game without patches? Not nearly as much as a publisher punishing buyers by putting out a half-baked game that needs to be patched (multiple times in most cases) to be playable. As I said before, it’s pretty common industry practice to review the gold code version of the game. Anything else becomes extremely problematic. Imagine someone doing a review on v1.05 of Elemental - do they need to state that their saves aren’t compatible with v1.06? What a publisher ships is what gets reviewed. Anything else creates too many problems, too many appearances of favoritism (“you didn’t review the game until they patched it a week later so it could get a better score”) and doesn’t appropriately reflect the value the buyer gets when they pick the item off the shelf.
I take it you’ve never been a paid game reviewer. Of COURSE there are deadlines - those sites that publish first get more traffic. More traffic = more money and reviewing is still a business. It’s not ideal - you’ll find threads from me blasting incomplete reviews - but it does happen,it has to happen and it’s certainly not “unrealistic”.
I don’t mean editorial deadlines.
I mean some sort of broad deadline by which ALL reviews must be in, to prevent the developer from patching the game to respectability before reviewers can severely mash it so that Cory feels satisfied that they will never release an unfinished game again :)
You’ll note that a few posts earlier I did say:
Why?
Reviews make the most traffic in the first few days of a game’s release. I’m certain that a 6 month old review or re-review will do less in traffic. This may be offset by mitigating circumstances (like an expansion pack or big patch), but I suspect just as often it’s a writer or editor who’s played the game in its patched state who says “hey, it actually turned out really good. Let’s give it some press.” Or perhaps a developer asks really nice for a re-review. I’m not sure how the internal decision making is made, but I’m quite certain there’s a reason sites don’t offer re-reviews as standard material for their audiences.
I may not be a reviewer, but I’ve lurked here long enough to know a few facts :)
Obviously not all though, as you pointed out by the gold masters policy. Yikes :x
Cubit
3851
Why do you keep attaching my name to attitudes I have not expressed, papasmurf? Knock it off.
Fair enough. But who is punished more?
[ul]
[li]If you review an under-baked gold master, the publisher gets bad press.[/li][li]If you don’t review an under-baked gold master, the users get a shitty product out of the box.[/li][/ul]
Seems like a damned if you do, damned if you don’t scenario…except that it’s the publisher who decided to ship an under-baked gold master in the first place. Since it was their responsiblity and their decision, I don’t see any reason to hold off on reviews so they can make their product look better in reviews.
Of course, we’re now getting into the game journalism thread territory here. So I’ll just say it’ll be interesting to see if any review sites break with their own policies to review a patched version.
Yeah, but most users don’t deal with that shitty product out of the box for very long. Patch culture is here to stay, even on consoles.
Are you really doing your readers a service by reviewing a game that might have gone from constant CTDs to a playable title with a patch that took you 5 minutes to download?
Jab
3854
Any info on when boxed copies are being shipped out? I still haven’t received my collector’s edition and have not gotten any emails regarding it.
Why not review it with the latest available patch and mention in the review that you are doing just that? That way, someone reading your review who is interested in purchasing the game will realize he or she needs to get the patch. Then if that person doesn’t have broadband or has metered access, he or she can factor that into the buying decision.
I think you’re overestimating the amount that bugs will drive down a review score. Except in the most egregious of cases, most reviewers won’t take more than 10-15% off for bugs. For example, you’ll notice in the 1Up Elemental review that it talks more about game balance than bugs. Bugs are never going to take a 90% game to a 50% game.
Furthermore, to use your example, what happens when the converse is true? When CA came out with their 1.2 patch that rendered the game unplayable for a fairly large number of users of one type of video card, that wasn’t mentioned in any review. This is just one (specific, real-world) example. There’s just a whole host of issues that you open up when you start reviewing patches, many of which I’ve mentioned in other posts. You have to draw the line somewhere and the easiest and most logical place is the game upon release. That’s why most review sites have that policy.
So, yeah, I do agree they are doing their readers a service. A review is one person’s (hopefully somewhat trained) opinion of the game that was released to the marketplace. As with any review, YMMV. But it’s not the reviewers job to paint the game in the best light. It’s their job to render an opinion. And that’s what they do - bugs are a fairly small part of the overall review process. Go back to that 1Up review of Elemental. Does anyone dispute that a C+ isn’t reasonable, whether it be the V1.0 or the V1.05 version?
Plus, should we really expect anything less? If an e-book were released and it was missing two chapters, would you argue that the reviewer should wait for the last two chapters before reviewing the book?
Again, I’m not the one saying that metered access is the reason for holding a review.
My position is one of fairness. Let’s say Civ 5 releases in a state as half-baked as Elemental. Now, since Civ 5 is using Steam, you could argue that you have to get all the updates anyway, so why not review it after Firaxis gets a couple of patches out the door, right? So what about Victoria 2? If you let Elemental and Civ 5 slide, shouldn’t Victoria 2 get the same shot? How long should you wait as a reviewer? First patch? Second? A month? Two? Six, in the case of Sim City Societies?
What happens when you review one game after three patches and another after one patch? Was that fair? Do you give Firaxis less time to bake their patches than you gave Elemental? They have a big publisher paying them, right? If the developer says the next patch will really hit it out of the park, should you wait?
These are the issues I would have with a review site holding their reviews for patches.
I don’t think you should wait at all, but at the same time I don’t think it’s unreasonable to apply a patch that’s available at launch, or even one that comes out while you are in the process of reviewing the game. If you do that, then you should mention in the review that you are playing a patched version, and specify which version.
Because it opens the process up to a whole host of issues, not the least of which is the charge of favoritism. “You waited for patch 1.2 for game A so you could give it a better score, but you reviewed ours at Gold Code.” As an editor, you don’t need that kind of hassle.
Furthermore, some game review sites get Gold Code masters and begin their reviews prior to any patches being released. So it’s not like you’re going to stop your review and start over because a patch came out, no matter how important it is.
Most games have multiple patched versions nowadays. It’s probably more confusing to have to mention the patched version that’s being reviewed since it’ll be different than the one most users play anyway. It’s just one more variable and an unnecessary one at that since all it does is adds more confusion.
It’s just a lot easier to draw a line in the sand, say you review gold code and then let the readers do the research on whether patches address the issues you bring up when reviewing the gold code. That’s what game forums are for. :)
Oghier
3860
I certainly believe it’s easier, but it’s not the best answer for the vast majority of consumers. For people using reviews as part of their buying decisions, they want to know what the game will be like if they buy it and play right then. Most will patch up to the latest version, particularly if the game tells you ‘update available.’ A review based on an obsolete version may be easier for you, but it’s less useful for most of your readers.
