4177

Yes, of course. It just struck me as funny!

4178

They figure that it will hurt new up-and-comers more than it will hurt them.

4179

They can also probably write the new legislation. It’s not like Congress knows how the internet works.
They’ll have to turn to experts. Oh look, Facebook has lots of them.

4180

These all seem like things that could be solved for outside of one post I shit out while feeding a baby this morning.

In terms of Facebook, “uncurated” just means doing away with the “Top News” sorting and going to time-based, most-recent-at-the-top posts. It means no suggesting friends or groups to me. Ads presented to me are random.

I don’t think it’s unreasonable to put more of the onus of spam filtering or moderation on the user.

And I see it the opposite way - I think the vast majority of people would never opt in. As long as you require adequate transparency about what it means to opt in, I don’t think anyone would.

First, this is what malicious actors already do. They’re already much better at it than anyone else, which is why watching a Warhammer 40k video gets you Nazi propaganda in your feed.

And second, maybe it’s not code - maybe it’s big data lakes or whatever. Make that anonymized and publicly viewable. At the very least, there should be transparency and specificity about how their models work. Not just “we look at the things you like and recommend other things you might like,” but “We know what kind of posts make you stop scrolling and for how long, we know what you click on and how long you spend reading linked content, etc”.

It’s not detached from reality – Steam does this:
image

What would Facebook’s look like? “We suggested you should join this Nazi group because you watched thirty Ben Shapiro videos in one day and because you clicked the heart reaction on the picture of your uncle’s Norse rune tattoo.”

It doesn’t have to be comprehensive, but it needs to pull back the veil a little bit so people can make the connection between their browsing habits and the shit that gets recommended to them. In some ways, it’s like the warnings on cigarettes.

A concrete example would be me saying that I don’t want to see the Daily Wire, so in addition to the algorithm knowing that I don’t want to see the Daily Wire, it also understands that I have some degree of pushback for things that are closely related to the Daily Wire.

When you’re talking about regulating entities like this, you come really close to running afoul of free speech protections. I’m no First Amendment stan, but it is broadly applied and uniquely American – it’s not going anywhere. But there’s no such protection for harmful technology, and for what it’s worth, I consider algorithmic content delivery to cause more harm than good.

I do consider a search engine that prioritizes paid ads to be bad. I do consider an e-mail client that scans my mail for keywords so it can present ads to me to be bad. And I definitely consider massively popular social media platforms that rely on easily-manipulated algorithms to control what people see to be bad.

I think when the idea of curated content first came about, it seemed really benign. “Oh, this customer liked this book, so I can recommend her other books she might like.” “Oh, this customer just watched this anime, maybe he’ll like this other anime.” But then the corporate owners of the algorithms realized they could be fine-tuned to maximize profit and get paying customers/advertisers. And the users/subjects of the algorithms realized they could exploit them to achieve nefarious ends. And the end result is decidedly not benign.

You could argue that humans are just gonna human and therefore we shouldn’t regulate anything, but in Facebook’s case, we see that they are taking human nature, amplifying it, and intentionally radicalizing people for the sake of clicks and profit. It’d be great if everyone was a perfectly educated and engaged digital citizen who knew how to be a good steward of their personal data. But they’re not, and society shouldn’t suffer as a result.

4181

Facebook does do thing, but the reality is that they hide the shit out of it.

If Facebook suggests something and you block it, it will tell you why it was suggested to you. Which, of course, you don’t see if you don’t block it. I imagine there is some way to find it, but it’s deliberately hidden. The fact that it exists at all though gives Facebook an out when they get called on it. “Users can see why something was suggested to them!” Technically true. Realistically, it’s hidden information that almost no one will ever see unless Facebook fucks up and pushes the wrong thing to the wrong person and they’re tech savvy enough to push back.

I think making algorithms more transparent can work. Mostly because, as you mentioned, it does work. Steam does it. Youtube does it a bit (though again usually when you’re responding with “what is this horseshit doing on my screen?”). Facebook already does it, but hides it. Make it so algorithm derived pushes have to have it stated why it’s being pushed to you on the push or something.

But I also completely don’t trust Congress to do it right. Because Facebook and Google will end up writing the law at the end of the day.

4182

A good first step might be just to require all algorithms to be open. Like Gun Violence studies, the best first step is to have actual information from the source. Allow people smarter than us actually to see what is going on.

Another step I would like to see is to require companies to publish all ads so that they are publically available. Maybe even see who is targeted and how should be available to researchers and the media.

4183

But my response to that still stands, in that you’re going to have a multitude of problems in trying to apply those laws to achieve the limitations on speech that you want to limit here.

The big issues you run into, are that the supreme court has only allowed for limitations on speech in very narrow cases, of what is legally defined as commercial speech. So you run into problems trying to limit what you and I would both perhaps consider bad, wrong stuff, because it is still protected under the first amendment. If the government were to enforce punishment against an organization for promoting some speech, but not others, then I believe you’d run afoul of the same free speech protections as it would amount to the government using its power to favor certain speech.

Something like removing section 230 protections, specifically for algorithmically pushed content would avoid most of those problems, as it does not require the government to treat any speech differently. Instead it simply allows 3rd parties to sue for things like defamation if the algorithm pushes stuff that they lie about (and then normal defamation laws would apply).

Now, the important thing to note is that the actual impact of this would just be to totally get rid of algorithmic content suggestions, the actual content of them becomes moot.

I believe that the reason why this was pursued, was because it fell under the umbrella of commercial speech, as it took the form of an advertisement that Match was sending out in email in order to encourage a commercial transaction.

Plus, the case appears to contain some other misleading business practices that they are being sued for.

Also, all that being said, the FTC hasn’t actually won this case, and the DOJ apparently released Match from a subpoena they had been issued previously, so we don’t know whether it actually falls under the existing limitations even on commercial speech… but it’s kind of moot because it’s different from FB’s actions, as FB isn’t pushing false information as ads for facebook, as Match was.

4184

I agree this is a problem especially for search engines. I had not considered that, but it does seem like search results would fall under the umbrella of algorithmically generated content suggestions.

This is kind of the problem I’ve been trying to point out… when you try to limit speech, it’s very easy for you to accidentally stomp on things that you didn’t mean to… and then that overly broad limit on speech ends up making the law rejected by the SCOTUS.

Also, for the record, I will murder you if you try to prevent me from using Google… I remember the dark times before a functional search engine. I will not go back.

4185

Mostly I think you’re making plausible points, but this I think misses the mark. There’s all sorts of ways in which this kind of transparency might help qualified third parties analyse and audit the ranking algorithms being run by social media companies, even if almost no individual users would know what to do with it. It might even enable some kind of automated tooling with which people could shape the behaviour of ranking algorithm to better suit the users real preferences.

4186

Picking this post because it looks the most convenient to rejoin, not to argue with jsnell in particular. Because, well, while the anger is justified, jsnell and Timex are basically correct that legally and technically, it’s not simple. The games Steam recommends or the topics Discus (or even Wordpress) recommends below are basically the same basic idea, even if dumber, so the line must start permissive and then close. Or you might add intent somehow, but that’s hard to enforce because of free speech - though protecting and rewarding whistleblowers with proof could help, and we should do more of that.

There’s still things you can do, like having the default view being the content you explicitly subscribed, or having a human readable topic unsubscribe (after opting in) with examples - even neural networks aren’t a blackbox. It will require a new ML model, but tough shit.
Or you can force open the market to external entities to make better sense of those numbers for users via some API, although that’s more of an EU thing. They’re quasi-communication monopolies with huge influence in most other markets, there should be obligations associated with that, like any utility - which they de facto are for many, many businesses.

Let’s be honest, it’s mostly volume and big promises, they’re still not particularly smart (big yet). Otherwise, nobody would be talking about external manipulation. What they can easily do, however, is pick up on engagement, because those are very distinguishable numbers that you can get some high probabilities from, and they directly influence profit. And that’s something you can snipe at the edges from a regulatory standpoint, including highly correlated variables, at least if there’s an agency with some level of transparent access to the inner workings - maybe filter less from actual contacts, give less relevance to popular content, show more colder hits, something along those lines; I’d have to read up on the state of art to see what’s there to mess with, but you also regulate in some like it was done with probability tables in gambling machines.

4187

A point of clarification here…

Most of the claims of external manipulation are imagined. It gains traction because it’s more enthralling to imagine some nefarious guy with a handlebar moustache pulling levers behind the machine.

Jsnell is correct that a component of the “problem” here is the fact that computers are in fact better than humans at identifying certain types of patterns, given enough data. Whether they are “smart” or not is debatable, but they are able to see complex patterns that elude humans.

Most of the issues being discussed in this thread are a result of these systems, devoid of any human bias. There are cases where human bias crept in, but that’s the exception target than the rule. The only real guidance from humans is, “show people stuff that they want to see, and will make them stick around,” or as you described it, maximize engagement.

That’s why it generally works on everyone, not just one political group.

4188

Sure, that’s also true, but like a lock, you only need one, and it’s hard to make either resist state actor sized burglars.
As to the models, we don’t know what the objective was. Maybe their data scientists focused on revenue, with the downsides being just natural effects of a big enough social network (digital or otherwise). Or maybe they (or, at least, the heads) did notice a correlation with personal political motives and were elated, fiddling in small ways in A/B experiments.
Either way, we do know they love society damaging content and would very much like to keep it, while having the pockets to keep the status quo to a significant extent - up to and very much including buying startup competitors and, err, greasing wheels. And that’s a pickle we must find a controlling point on the acidity.
(trying too hard on the metaphors today)

4189

Nobody thinks it’s simple.

4190

Yep, and the simpler and more transparent the algorithms are, the more harmful they will be, if only because they will be easier for the bad guys to game. If your algorithm is driven 100% by closeness in network and produced engagement, you will get a cesspool in social media. It’s only by bringing in more complex and subjective measures like the trustworthiness of a source that you get any improvements at all.

Social media is mainly stupid; it just reflects human nature, which is awful.

The only thing that can motivate the social media giants to clean up their act is shame or altruism. Probably the best short term tactic is forcing them to make a lot of data available to selected outside scientists. The outside experts won’t be able to force Facebook to change anything, but the publicity they generate might encourage at least the appearance of altruism even at the cost of some profit.

4191

I don’t know that that’s true, at least based on some of the posts made in this conversation.

4192

From what I’ve seen, the main issue isn’t really any specific content, but how it is presented.

For instance, the problem with Facebook isn’t that they have Nazi posts or whatever, or even that they recommend it to some people that it will appeal to.

Rather, the problem is that the algorithm will nudge everyone towards increasingly radical views. It doesn’t really care what extreme views it’s moving you towards… But it’ll radicalize you by showing you increasingly extreme current, slowly making you more and more receptive to increasingly extreme views. It didn’t show you the Nazis on day 1, because they weren’t that close to what you were watching, and you probably wouldn’t have watched it… But by day X, you’re primed for the Nazis.

And what’s worse, because it does this to everyone, it is creating a society where more and more people are at the edges, and completely unable to work with others who exist on the other edges.

And I don’t think it really is driven to push any of those ideological extremes… It doesn’t care. It’s just that people who are at the extremes, are more engaged.

I don’t even know if it’s designed to create this radicalization, it may simply be an emergent property of suggesting stuff, and perhaps a natural tendency for more extreme content to be more engaging to average people. Simply showing people similar content may naturally result in a progression towards ideological extremes.

4193

If I say e.g. we should return to the moon, it is a suggestion to do something that is both possible and not simple. Complicated doesn’t mean impossible.

4194

It kind of is. Normal users have no idea of just how much spam and abuse they’re being shielded from by the very algorithms you want to ban. If it was let through, basically every single user-facing site and service would become unusable. (I work on abuse and account security, so I’ve got a decent idea on what the raw stream of sewage looks like). And that’s in the current situation, where the economics of spam at least are kept in check by those algorithms. It’d get even worse the moment that filtering spam becomes regulated.

Second, it’s just not possible for people to handle the spam and moderation problem themselves. 99.9% of the people do not have the expertise for handling their own spam. The few that remain do not have the data for it. I guess you might be imagining something like a “this is spam, don’t show me things like it” button. But that is just the kind of algorithmic content curation that you want to ban, since “things like this” is not objective.

What would such a transparent opt-in dialogue would say, and why would we expect it to discourage the majority of people from opting in?

I think it would say “click here if you want to see more videos like this”. It would be true, and basically every single user would click on it, because the reason they went to YouTube or TikTok or whatever was too see that video, they liked it, and they would like to see more.

What benefit is there to the average user to know just what features are fed into the model? I honestly can’t think of any. But the benefit to the malicious actors is massive. Abuse is an iterative and adversarial process, where the attacker has most of the advantages. If you additionally make give them the benefit of information assymmetry, by forcing the defender to reveal their data and their methods, the task becomes impossible.

And believe it or not, none of these companies actually want to be hosting Nazis. If nothing else, nobody will want to advertise if there’s a risk of the ads being shown in that kind of context. That’s why they have thousands, maybe tens of thousands, of people working on trust and safety. Yes, even Facebook.

If you want to get a feel for what this kind of work entails, and the challenges, the leaked Facebook TnS report about January 6th is well worth a read. I don’t love how Buzzfeed framed the story, but the report which is quoted in full at the end of the article is a great example of anti-abuse work, and one of the very few public examples:

Right. So that is not what you originally asked for, which was “as much specificity as possible”. Steam it as most “give me some plausible explanation” for why it was recommended rather than really allowing for you to understand why the decision was made. For example: why is the game you’re looking at similar to GTA V and Project Cars? As-is, they could justify literally every recommendation as “this is similar to GTA V”.

Ok, so that’s a good change in scope, because it’s now implementable and could be understood by a normal person.

Would it actually achieve what you want? I think your concern is the gradual radicalization via recommendations, right? The recommendations will ultimately just tell you that the recommendation is for something very similar to what you’ve already liked. You’re basically suggesting some kind of an intervention “hey, do you understand that these people whose content you’re watching are basically white supremacists”, but timed to happen early enough that the victim is shocked by that.

You won’t get that level of introspection or insight just by reporting the raw facts of why a recommendation was made. You’d need some very different system, that’s basically completely orthogonal to the recommendation system itself. An interesting research problem, but seems very hard to regulate into existence.

What you were proposing is going to apply to every search engine, no matter how it is monetized. What is the uncurated ranking of search results that they’d use? It simply does not exist. The closest you could get to is every search just returning 10 random links, whether they are related to your search terms or not. But even that wouldn’t work since there are subjective decisions going into just what pages make it into the search index in the first place.

Interesting idea. I guess I’d be a bit worried that the incentive for the researchers is to find something they can sensationalize. An “everything is basically OK” result might be publishable, but would sink into obscurity.

4195

Heh, even the science about engagement is driven by engagement.

4196

This is one way to address the problem. If you prevent people from monetizing disinformation, the incentive to create it diminishes. This will stop some of the social media posts that just point people to a YouTube video that was made solely to drive revenue.