You probably read the news about the Meta settlement. Most of the coverage led with the number, which is not an insane journalistic choice. The number itself is news. $17.1B is the largest state consumer-protection settlement in American history outside of Big Tobacco.
If you read past the number, you probably got to some of the design decisions Meta agreed to. Hidden like counts, cosmetic-surgery filters banned, chronological feed option, two-hour default cap… These are all good, sane design decisions that any company acting in good faith would’ve shipped already. Or at least any company that gave a shit about its own internal research on the irreversible damage their product was doing to teenage girls.
But the four clauses buried deeper in the news coverage (if mentioned at all) are the ones that have my blood boiling. And they all circle one deeply problematic issue: an age assurance framework.
But predictive surveillance is not only used by good actors. And it’s not only used to figure out how old you are. Once these machines are perfected, you can point them in a myriad of different directions. “Is this person pregnant?” “Is this person undocumented?” “Is this person organizing protests?”
Meta “commits to utilizing and continuing to improve its soft matching models,” leveraging “device IDs, phone numbers, and e-mail addresses” to identify unlinked accounts.
Translation: a federal consent judgement just ordered Mark Zuckerberg to get better at figuring out which accounts belong to the same person across devices. It’s called cross-device identity resolution.
I don’t get to narrate this from the cheap seats. I built the pipes. I sold them twice, to companies that used them exactly as designed, and I run a permissioning company now — which means I sell the aspirin for a headache I helped cause.
Which is also how I know what comes next. Nobody builds a classifier for one label. You build the pipeline for the label you can defend in public, and every label after that is a configuration change. Age is just the Trojan Horse.
It’s surveillance of the most terrifying order, and this week the most surveillance-y company on the planet was told they HAVE to get better at it.
So yes, Zuckerberg has to pay a historic dollar amount. But it probably was worth the price tag because it comes with the obligation to do exactly the thing he already wanted to do more of. And now when someone objects to him doing it, he’s got a court-ordered excuse. Sounds like: “I’m under a court order. It’s 2026, dummies. Of course I have to use AI to solve this problem.”
You can refuse to hand over your data. You can refuse to allow others to sell your data. What you cannot do is refuse to be guessed about.
Now let’s acknowledge that the plaintiffs in this case weren’t wrong. The internal documents were damning, the harm to kids is real and documented, and “the privacy implications are troubling” is a garbage answer to a parent who has buried a child. But the thing is, almost every protection in this settlement makes for a better and safer product without having to figure out how old anybody is. Killing like counts, banning cosmetic-surgery filters, offering a chronological feed. Why not just ship those to everyone?
Instead the settlement orders Meta to build a machine that guesses things about you based on how you behave. And to see why that’s so devastatingly problematic, we only need to look at a different news story that was heavily covered this week: ChatGPT for Teens.
OpenAI is facing down its own lawsuits over dead kids. And Altman decided to be proactive: we’re going to have a different product for teenagers than the one that we have for everybody else.
It doesn’t sound like a terrible solution until you ask the obvious question: how do you know who is a teen and who isn’t?
Wait for everybody’s favorite answer: surveillance. Predictive technology that watches you, what you read, the words you type, the speed at which you type them and makes a pretty smart guess about how old you are.
Somebody I used to work with built a company that uses similar technology to predict dementia. It infers cognitive decline based on how fast you type, what you type, how frequently you delete.
Now, if you live in a state or a country with data protection laws, you might think you’re, well… protected. But you’d be wrong.
You can refuse to hand over your data. You can refuse to allow others to sell your data. What you cannot do is refuse to be guessed about.
And while those inferences might start innocuously enough, that’s not where they stay. “I don’t care if ChatGPT infers that my 13-year-old is a 13-year-old.” “I’m delighted if a company is able to get somebody help for cognitive decline before they get behind the wheel of a car on the same road that my kids drive on.”
Sure.
But predictive surveillance is not only used by good actors. And it’s not only used to figure out how old you are.
Once these machines are perfected, you can point them in a myriad of different directions. “Is this person pregnant?” “Is this person undocumented?” “Is this person organizing protests?”
The distance between “safety feature” and “state surveillance” is just a hop and a skip.
Which brings me to the clause of Meta’s settlement that might have infuriated me most of all. About five billion dollars of Meta’s payment isn’t guaranteed. Meta owes it only if TikTok and YouTube adopt the same terms. Five billion dollars to buy an industry standard of surveillance.
And Section IX.G says nobody gets to argue about it later. Both parties waived the right to challenge any term of the judgment, including on constitutional grounds. Legislatures tried this in Arkansas, Utah, Ohio and Georgia. Courts threw it out every time on First Amendment grounds. This version can’t be thrown out, because the only two parties with standing signed away the right to ask.
What couldn’t survive as a law now stands as a contract.
Ya gotta love lawyers and judges: Frequently wrong, never confused. And always much too brilliant to ask for input from people who could help them think through the implications of the claims, opinions, and orders they’re filing.
I’ve been worked up about this all week — spittle balls at the corners of my mouth, the whole bit — so let me be precise about what I’m actually claiming:
Surveillance everywhere, under the banner of child safety. And a court-ordered alibi for one of the worst actors in technology. That’s the trade we made this week. We’re all hopelessly Zucked.
Definitely the most thought provoking reaction piece I've read on the court decision. If an over engineered solution that has the fox guarding hen house is not helpful, what does the pragmatic solution look like that still arrives at the desired outcome?