Unconventional

AI is not normal.

Which is to say, it doesn’t follow the norms we’d built our systems and processes and interactions around — and that points both to great opportunities and huge challenges as we increasingly deploy it across our lives and work.

Knowing which is which is the critical question.

(And credit where credit is due: The key insight in this post was entirely generated by Claude. Whether you find that exciting or terrifying is up to you.)

I was reading Yuval Noah Harari’s rejoinder in the FT to Argentina’s president Javier Milei’s proposal to legalize “non-human companies” run entirely by AI. The danger, Yuval notes, is that AI-run companies aren’t constrained by the same considerations that the humans in a human-run corporation are.

Hitherto, corporations have been run by human beings possessing a dual nature. Human CEOs are corporate entities who care about the success of the corporation and fear things like bankruptcy. But they are also biological entities, who care even more about their freedom and happiness and fear things like spending ten years in prison. An AI CEO would be a purely corporate entity, and it is unclear what kind of sanctions could keep it in check. If it faces bankruptcy — which is equivalent to its death — it would presumably be willing to do anything to avoid that fate.

Which is to say, it isn’t simply the stated rules and regulations that keep society on track; it’s the unstated assumptions and norms — and practical resource limitations — that prevent us from coloring too far outside the lines, both for better and for worse. At least in general.

And as we’ve seen from Donald Trump, someone willing to ignore the norms can get a lot done; and also do a lot of damage.

Think of AI systems as that, but on steroids. On the positive side, they’re tireless and can parse huge amounts of information; that’s allowed journalists to dig through massive document sets, find needles in haystacks, and helped us stay much more informed than we could if we relied only on our biological processes. We can create much more personalized stories, and many more of them, tailored to people’s needs and interests. It holds out hope that communities too small in the past to have their perspectives represented in stories can be better seen in coverage in the future.

But it also means that accountability is more diffuse; a reporter whose name is on a story has to juggle a host of considerations when they write a piece: They know, if they’re a local reporter, that they’ll likely have to see the subject of the story the next day, and that puts some pressure on them to ensure the story is fair and reflects multiple points of view. They’re on the hook if there’s a mistake; they have to think about how to manage a source relationship. None of this is baked into a simple “is the story true?” rubric that the best AI-generated content would adhere to.

Another way to put it: When we write rules and guidelines for how AI systems should behave, how much do we assume it will also follow the unwritten norms that have historically constrained us? Because, if they aren’t written, they probably won’t constrain the AI systems.

And that — again — is both good and bad.

Here’s what Claude said — on its own, without prompting from me, as I mulled this idea with it:

The norms that AI blows through productively are almost all norms that were really constraints imposed by scarcity — of time, attention, expertise, geography, resources. Removing those constraints is genuinely liberating if the values underneath the norms are preserved.

The norms that AI blows through destructively are almost all norms that were really accountability mechanisms — the reputational stake, the reciprocal relationship, the human in the loop who had something to lose. Removing those constraints is genuinely dangerous because the values underneath the norms don’t survive without the mechanism that enforced them.

That’s a useful way to think about when AI is helpfully extending our capabilities, and when it may be unhelpfully dismantling assumptions we’ve all come to depend on.

And there are grey areas as well: One of the main complaints about AI systems is that they’ve abused fair use and copyright laws by hoovering up content to train on; that’s functionally the same process that humans use to learn, but unbound from our short attention spans and need to sleep. Is that fair? Unfair? Do the rules written for one species work for another species?

Having AI systems also means we don’t need to select targets of investigations as carefully as we once did when we only had a handful of reporters; now we can unleash bots to dig through every public filing of every legislator, hunting for any anomalies — major, minor, or otherwise — we can uncover. Is that a good thing or a bad thing?

I don’t know. It’s definitely a different thing.

I know, too, that similar technology can be unleashed on anyone — including journalists.

And beyond the world of news and information: AI systems are ruthlessly efficient at finding gaps and loopholes in systems — that’s fundamentally what Mythos is said to do when it hunts for vulnerabilities in code. That capability has been disabled in Fable 5, the new model that Anthropic just rolled out — and which it just shut off in response to a White House edict — but imagine if you took that single-mindedness and applied it to every legal contract, every financial instrument, every transaction and find the holes, the unwritten rules and norms that have always existed but would have taken humans years to surface.

One of the attractions of AI systems is that they can surmount the limitations flesh and blood humans have; but in the process we should be careful they’re not also surmounting the unspoken and unwritten safeguards we depend on.