Surface Tension

What does the world look like if you look at it like an LLM?

I used to joke, when I was running The Wall Street Journal’s and Reuters’ budgets, that I saw the world the way Neo did in The Matrix: as streams of green numbers running vertically, although in my case they represented costs, rather than the humans in front of me. (And yes, I was not popular. I also see a therapist.)

It’s a way of looking at the world that’s optimized for an outcome; in my case, to ensure we were using what limited resources we had in the best way possible. And when I was in the Singapore army, I learned to look at pastoral scenes in terms of fields of fire and ambush locations. Different use case.

I have investigative journalist friends who are optimizing for other outcomes; they see the world in documents: What files and records exist that can shed light on a particular event, rather than just the frail human memory? Yet others — my friend Giannina Segnini comes to mind — see the world in terms of databases: Ask her a question, and her mind churns, looking for the data structures and data stores that might offer an answer. These are great journalism techniques, and examples of the mental jujitsu that distinguishes a certain type of great journalist.

So how does — or should — AI make us look at the world, and how best can we take advantage of its capabilities? I think there’s at least one mental shift — hiding in plain sight — that can help us rethink what’s possible to do.

To take a step back: What do LLMs — Large Language Models — do well? (And for the purposes of this discussion, I’m ignoring all the other Gen AI capabilities, like image and video generation, protein folding, drug discovery and the like.) As the name implies, it’s a language model, and it is in fact very good at languages — analyzing, summarizing, reading, writing and so on.

If we break that down — and this will be useful in a bit, so bear with me — that analysis capability means it can edit text, both at a conceptual level as as a simple proofreader; it can extract and structure text; it can summarize. Because it has such a huge context window — functionally, memory — it means it can look for and find connections across multiple documents, and far more effectively than most humans can. It can “understand” — however you want to use that term — nuance, subtlety, and increasingly, humor, irony and sarcasm. And because it’s multimodal, it can ingest documents of all kinds, from PDFs to videos to Excel spreadsheets and more.

But you knew that. Where are we going with this, you ask? Patience, I counsel.

It can translate and transcribe, which we knew. Because it can handle language, it can also “reason” and “infer.” And it can generate language as well, which means it can write (so far only middling well, but good enough for most use cases); it can create multiple versions of stories from the same set of facts; it can create to-do lists and instructions.

But the key point to note is that code — programming language — is language. So everything that it can do to English, it can do to coding languages. Except better, because code is a more templated system of communication, and with clearer outcomes that you can test.

And because code is what runs all our digital surfaces — websites, phones, laptops, databases, etc — we have a system that can talk to all of them. And since the entryway to most of our systems — nuclear power plants, car navigation systems, UPS delivery records — is code, those are just more surfaces that an LLM can, in theory, look at and talk to.

Perhaps this all sounds obvious. It was to me when I wrote this out on a whiteboard with Adiel the other day, but up until that moment I hadn’t thought of it that way. I was focused on what each new use case was that I had not thought about the scope of possible use cases. And now it’s clear: LLMs can reach, read, and understand any digital system they have access to.

There’s more: Because code executes, LLMs can write code to do things, including write more code to do more things; that’s the simple — or complicated — version of saying it can become agentic: It can set itself off to do things autonomously. Which means, in theory, LLMs can not only read and understand digital surfaces, they can operate on them as well. It’s one thing to have a system that can, say, read files from a database; it’s another to have one that can write to it as well, and even more, to do it autonomously; and, if it has permissions, to completely rebuild the entire database.

That’s both terrifying and incredibly empowering at the same time. That’s why we worry about AI agents set off to tirelessly attempt to hack into systems around the world. (Update: They already do.) It’s why we worry about rogue systems that have been badly programmed turning on humans because they misunderstood a prompt and decided that, in fact, they should shut off the life-support systems in a spaceship.

On the other hand, it means a journalist can look at the world with much more expanded reach.

And to focus on that upside for a moment: What does it mean to see the world in terms of digital surfaces that you — or your agent — might be able to access? What does that mental model get you?

Giannina, who I mentioned above, doesn’t really see the world in terms of databases; she sees it, as she memorably once explained to a packed session at an investigative journalism conference in Manila, in terms of information being gathered. She doesn’t ask, what databases exist? She asks, what information was captured? If a form needs to be filled out, for example, there’s information being captured. If that information is captured, then it’s data, and if it’s data, it’s stored somewhere. Her job is to go find it and get access to it.

What if we took that mindset and turned it into looking at digital surfaces? Wherever there is one — and there are digital surfaces everywhere — that’s another place your agent can, in theory, reach. What are the journalism use cases for that? How would we see reporting opportunities if we went around the world with LLM-shaped glasses?

I’m sure there are multiple smart ways of deploying systems like this, but here’s a simple example: We tend to think of looking for information as a time-based activity, meaning, we check to see if something has happened, rather than wonder if it might happen in the future. That makes sense; we don’t have the bandwidth to keep checking to see if something has changed. (To be sure, there are systems that financial newswires use to poll corporate sites regularly, looking for new announcements, but those are generally very specifically and finely tuned to look for one thing.)

But agentic systems turn that on its head. If you’re musing about some event — a CEO saying something specific in public — you can build a system that can lurk around until that event happens and alert you. Better still, if you wanted it to be some kind of complex event — six CEOs saying the same thing, in iambic pentameter, within two weeks of each other — you can do that too. And so much more.

Because the world runs on digital surfaces. We just have to see it that way.