Valve has a plan for review-bombing on Steam


#82

And I think you should have tools to help you in that. But ultimately user reviews are going to be based on the factors that user cares about and malfeasance by the creator is one of the possible factors, and should be allowed to be.


#83

“Leaving a bad review without buying the game” is effectively what people constantly get away with by way of the refund system, since Valve doesn’t care if you buy and refund a dozen games while logging just enough playtime in them (again, six minutes to be eligible to leave a review - I don’t see how you can think this isn’t “gaming the system”) to leave a review.

Of course, there’s a very obvious solution there as well: like with the review system itself, Valve needs to dedicate more manpower to their refund system, rather than having an automated system process everything. (They have plenty of incentive to dedicate resources here, too - I know plenty of people who use Steam refunds as a way to try games, which Valve explicitly doesn’t want.)


#84

It doesn’t matter what Valve does. Quantity based review systems like in Steam are inherently flawed. Showing more data might reveal the clear cases of review bombing, but does nothing to address the underlying problem.

That problem being that I don’t know the reasons for a reviewer’s up/down vote. If a game has a 1000 recommendations that are all given for reasons I personally don’t care about and a 100 downvotes for reasons I do care about, the game will still show up for me as ‘overwhelmingly positive’. Yet if I were to go by that recommendation I end up with a game I will hate.

Now you might say, “that’s what reviews are for”. And yes, that helps a bit, but for it to be truly helpful I must assume that; A) Everyone leaves a review along with their vote. B) That the reviews are truthful. C) I’ll have to read most of them.

Point C alone is enough to defeat the purpose of having a review system like this in the first place. The whole point is to have a single score for me to look at and be able to tell if it’s something I want or not. But it doesn’t do that because it lacks crucial information about the why of that result. Without knowing that the final score is just a tyranny of the majority number, which is why review bombings are so effective in the first place.


#85

I’ve always been in the user reviews are worthless category for a long time, and mostly for the reasons you cited.

Reviews by people whose tastes I know, not even agree with just understand, are far more valuable to me.


#86

But this is i would argue an outlier in probability, in that the thing you care about not being congruent with the score. A game with a huge number of positive votes is far more likely to be good than a game with a very small number of positive votes. A game with a huge number of negative votes is far more likely to be bad than a game with a handful of negative votes.

Of course, there’s genre, right? You might see 95% positive with Rocket League, but think to yourself “I don’t like sports games, or cars, or rockets”. In which case, yea… i guess that score doesn’t work for you. There’s some common sense you have to apply here.

To date Steam Reviews have been, by far, the most accurate in approximating the “hit or miss” level of a game, far moreso than Metacritic. Quantity on the scale that Valve had really does possess a quality of its own. The large numbers of reviews are supposed to “proof” against outliers. Now Valve is seeing the semi-organized mobs of angry gamers that others have been dealing with for years, and it doesn’t exactly know what to do without sabotaging its own system.


#87

This histogram is going into the wrong direction. What it’s doing is exposing the raw voting data to the users; the right solution is likely to be doing some more sophisticated processing of the votes to produce the rating. But showing that histogram of raw votes is going to make that harder - anything other than a raw average will now appear to be inconsistent with the actual votes, driving people mad.

What do I mean by processing? One option would be to weigh each vote by the inverse of the amount of votes case in the same time period (not sure whether that time period would be a day, a week, a month, or if it’d depend on how popular the game is; but let’s say a week for argument’s sake). So for each week, compute the average score. To produce the topline score, average together all the weekly averages.

If there’s a short-lived controversy causing a huge spike in votes, those votes will have a huge effect on the “recent” score, but just noise for the total score. If the game has genuine problems, it’d rack up downvotes at a similar pace at all times, and those problems would be properly reflected in the total score.

To review-bomb in this system, you’d need a fair bit of dedication. People would need to commit to the bombing for a period of months, and everyone participating would need to know exactly which week to vote one. It’d still be doable in theory, but need a lot of organization for a slow payoff. Maybe not the strengths of a lol-driven 4chan mob…

The above model is a bit simplified (you’d e.g. want special treatment of the pre-launch period). But the main point is that there should be ways to fix this properly, as long as you accept that every vote is not equal. Here Valve is doing exactly the opposite.


#88

I think determining the right solution requires some assumptions on Steam’s goals that perhaps aren’t accurate. I suspect they want sales, first and foremost. If whatever they’re doing doesn’t serve that primary interest, I’d be stunned if they go ahead with it. Perfecting a review system so it will provide a personalized and/or weighted score based on numerous factors may be more useful to the consumer, but not necessarily for sales.

Providing the histogram to show when a concentration of downvotes may have occurred is also useful to the customer beyond review-bombing, as long as they bother to look. For example, one of my long-time favorite games is Sword of the Stars (warts and all). It’s long done exceptionally well in Steam’s customer review scores. However, of late some negative reviews started showing up. This led me to the Steam discussion forum, where I learned many recent purchases were getting an error for an invalid key and therefore weren’t able to play.


#89

But their current system is already not optimizing for sales.

Why did Steam introduce user reviews? Because they stopped any form of curation, and the store might as well be a pile of shit. The reviews were one way to let people quickly figure out which games are worth a closer look.

Then some publishers started gaming the system, by giving away free games in exchange for positive reviews. The scores of those games were now artificially boosted especially during launch. Good news for Valve, right? Higher scores -> higher sales -> goals of the review system achieved! Maybe in the short term. But in the long term, those kinds of shenanigans would have destroyed the customer’s trust in the reviews. So Valve tried to close that loophole.

Review bombing is basically the same thing, just in the opposite direction. Again if review bombing is allowed to work, people will eventually lose trust in the system. If that happens, it stops providing any value to Valve. So of course they’ll try to solve it somehow. I just don’t think this histogram solution will actually work, and it has the downside of locking them out of better ones in the future.

But wouldn’t the “recent” score already have done that? Knowing that in the last month people have been feeling negative about the game is indeed valuable information to the potential buyer. Knowing that there was a three-day spike in the number of reviews six months ago is mostly worthless.


#90

I’m not sure how it locks them out. I mean, they can just say “screw reviews” and dump the whole system tomorrow if they felt like it.

Sure, but what about three months after the issue was resolved? People can see the blip and then note it’s back to normal (or at least Paradox hopes so).


#91

So review bombing is now always a negative thing (impacts, not score)? There is no positive review bombing? By definition it has to be negative for people to lose trust in the system.

Before you can propose any solution, you have to define the scope of the problem. How many true review bombs have occurred on Steam? How many products have been significantly impacted by such an action?


#92

PlayerUnknown’s Battlegrounds gets the review-bombing treatment from Chinese players.


#93

For context, from what I can tell from reading a bit, PUBG previously didn’t support Chinese players at all, and the Asian server they just added to try to solve that problem apparently sucks a lot (and not just for Asian players).

In other words, this is a bit of a different situation from the last time Chinese players review-bombed a popular game, which involved Endless Legend and a Chinese mod that included a link to pirate an older version of the game the mod was compatible with.


#94

Saw a guy with 500 owned games on steam and around 250 reviews.

This is how nearly all the reviews look like:

Should be possible to weed out this sort of thing easily?

Alternatively have a user-customized option so you can ‘flag’ certain reviews to be considered the opposite, or discarded, to get an accurate score. I guess the updated Recent Reviews etc display defeats this, assuming there isn’t a lot of “users” who do these kind of reviews.


#95

what is wrong with this?


#96

Well, it’s completely unhelpful. But not outside of the reasonable parameters of use.


#97

There’s a lot of shitty games being given thumbs up that shouldn’t have it? But I guess the majority of users will still review it correctly and give them the ‘shit game’ status they deserve, and users like the above are more akin to those who reply to spam messages and buy stuff through it - thus helping “it” stay alive.


#98

It’s not your place to decide what someone felt about a game though? Maybe they enjoyed it for whatever reason and want to show that with a recommend but are too lazy to write up a full review.


#99

~200 similar ones is suspect. Surely there is a subscene on Steam that somehow rewards people that give “up” reviews to inflate scores.

And my place is to say that they are wrong, and hoping for an ignore function on steam to stop them influencing scores ‘negatively’ with their malfeasance .


#100

I own close to 1000 titles on Steam. I’ve left one review. -1-

I think I’m one end of the spectrum (which spectrum I’ll leave up to the armchair psychologists, lol). On the other end, you have people who leave reviews for everything. Somewhere in between, I suspect there are people who just leave thumbs up for those they like. Of those, perhaps some feel like they have to type something or they want to be silly and repeat themselves.


#101

I wonder how that spectrum maps on to support for likes.