Restructuring Structured Journalism: A Manifesto (of Sorts)

Well, this is a blast from the past.

Well, OK, not really. When I coined the term “structured journalism” a decade and a half ago — at least I’m staking out my claim to be one of the first to name it — it was to frame a core idea: That newsrooms were leaving too much value on the table by failing to reuse, reframe and recycle the information not just in stories, but also in reporters’ notebooks. And that readers would benefit too if we took that information and turned it into new story formats and delivery methods that could offer them more — or less — context as they needed.

Back then, the technology to do this was close to non-existent; some of us tried to build proto-structured journalism projects with what was available, not least Politifact; the late, lamented Homicide Watch DC and Circa; and WhoRunsHK when I was at the South China Morning Post, and Connected China when I moved to Reuters.

But that was then, and this is now.

The technology has finally caught up with the idea — and more than that, helped reframe it in a fresher, broader and more expansive way. Generative AI’s ability to parse, dissect, and reassemble words and information, at scale and at speed, is a game — indeed, frame — changer. And it’ll mean some deep — and uncomfortable — reflection about how we see ourselves and our place in the world.

Let’s pull back and put an idea on the table: Journalists’ core skill — or value to society, if you prefer — is the ability to think about what questions matter, to collect facts and information relevant to those questions, to bring analysis and context to those facts, and synthesize conclusions.

What about stories, you ask?

Good question. Historically — and currently — we take all that work we’ve done that’s described above and pour it into story-shaped objects to distribute to people. That’s how we brought our synthesis of all that information to our audience. We called them “stories.”

Those story-shaped containers have changed over time, often — but not always — reshaped by new technology, habits or conventions. They were trimmed for length to fit on dead trees thrown on driveways; edited to an ad-friendly segment length on a current affairs show; turned into a chatty pseudo-conversation to be streamed.

There wasn’t anything particularly sacred about the container or its shape; it had a purpose, which was to convey information and ideas effectively. That’s not to say it was arbitrary: Much of what went into it, and how it was structured, was based on how humans have always ingested and processed information. We look for relevance to us high up in the piece, enough context for it to make sense to us, relatable characters for ideas and concepts to stick in our minds, drama to keep us engrossed, and so on.

Stories used to be where most of the value resided, because that’s the only way we could bring our accumulated knowledge to readers and audiences. Stories didn’t change from reader to reader because, well, they couldn’t. And the only entities who could create them were us — humans.

But now machines can. And stories can change, and mutate, and adjust to different readers.

The containers aren’t just changing; how they’re filled is also changing.

And that morphs the core idea of Structured Journalism: That the value we create is increasingly concentrated in the work, the discoveries, the ideas, the insights that we put into the story containers. The value of the containers themselves — the stories — diminishes as AI systems grow in their ability to fill them well, and adapt them to different users.

There’s increasing value, too, in understanding users and communities and their individual news needs, because that helps us build the products and tools that can serve them the way they want to be served; whatever container is most appropriate at the time — a short story, a long story, a visual story, a podcast, a set of bullet points, and so on.

Sure, you say, this sounds like a nice manifesto that you could tack to the door of a cathedral somewhere, but what’s it to me?

It’s that the war for readers and truth and facts isn’t to be fought on the battlefields of stories. That ground is increasingly controlled by machines. Our advantage, our value, lies in collecting facts, bringing analysis and insight, and understanding communities and their needs.

Wait, you say — are you arguing that stories are dead? Aren’t stories and narrative structure how we create meaning and convey it to readers? Yes. And no. Some stories and some narratives are absolutely critical to how we convey meaning, and will remain so. Great 4,000-word, carefully constructed deep dives will have a place in this information ecosystem; they’ll inhabit what Tony Haile has dubbed the “archipelagos of trust,” niche pockets where audiences can find high-end smart writing, reporting, analysis or aggregation. For those stories, and for the organizations that create them, narrative is analysis. Narrative is the product.

But most news isn’t that. Most news stories are 400-word updates on what happened yesterday, or 200-word briefs, or 800-word news analyses. Most of the value they contain is in the raw information they carry, along with some context and analysis. And more important than their length is who they’re written for: They’re built for scale — to reach as many people as possible. They center information, perspectives and insights that matter most to whoever the reporter or the publication imagine as their key readers. And that made sense in a world where you could only do one version of a story and distribution was scarce.

But that’s not the same as serving everyone well. Maybe a story about a new tax to pay for a football stadium is important to a non-sports fan because of what it means to her tax bill; maybe what matters to a reader living near the proposed site is what it’ll do to his neighborhood; maybe a crusader against perceived corruption in city hall is focused on who’s getting the contract to build the stadium. Maybe someone prefers English written at a high school level, and someone else at a graduate level; maybe someone only has time, at that moment, for a 200-word summary, and maybe someone else has an hour to kill. All the information that the reporter has collected can be structured, reorganized and rebuilt to serve each of those readers better.

We couldn’t personalize stories before; we’re getting much closer to being able to now. We’re not talking about killing stories; we’re talking about how they can serve more people, better.

OK, but is personalization really better? What about editorial judgment? Won’t we dissolve into filter bubbles of one, and lose a shared sense of reality and community? Yes. But.

Mainstream media has ill-served so many communities for so long, simply because the economics and technologies drove it to build one-size-fits-all narratives. And so, yes, editorial judgment matters, but that judgment implicitly embeds so many unquestioned assumptions about what matters — and more importantly, what doesn’t matter. Ask the LGBT community how well editorial judgment served them in the early 1980s, when AIDS was ravaging the community and the story was barely a blip in the mainstream media. Customizing stories offers one way to reach different communities with perspectives that speak to them. It’s true that we risk splintering into separate realities; but the answer can’t be to hark back to the days when we acted as gatekeepers of how to see the world. We need to redouble efforts to find solutions — such as the prototype I built to offer broader points of view to readers — that can pierce those bubbles.

The future looks frightening; but the answer isn’t in the past.

Can this actually work in practice? Who’s building these systems? Who’s collecting this information? Who’s understanding what readers want? And is the technology up to it?

The honest answer is that I don’t have an answer to all of those questions. At the Tow-Knight Center, we’ve been experimenting and testing to probe the technology and brainstorm new processes, and we think there are paths worth exploring. And I know smarter minds than mine are also exploring these issues.

But we begin with the proposition that Structured Journalism moves us from a story-centric view of journalism to an information- and audience-centric view of journalism. And that will take some wrenching rethinking of our role, our work, our processes and our identity.

Who are we if we are not writers and craftspeople first and foremost; how do we see ourselves if our judgment of what matters is just one of many that we will help create? What happens if we have to invest as much time understanding our readers as we do understanding our beats? Can we reframe our role as stewards of the information that communities want and need, and the interface that helps them access it?

Perhaps the future isn’t clear, but we can see the shape of it as it advances towards us, and our possible role — and value — in it. It’s a world of huge risk and challenges, and it almost certainly upturns our current place in the information landscape.

But it’s also a world that offers real opportunities to serve communities better, and we’re early enough in its evolution that we can work to put public interest at the heart of how it develops.

But only if we’re clear-eyed about what’s coming, and where our value in it lies.

And a bonus post: Claude pushes back. ChatGPT pushed back against Claude pushing back (And I push back against both of them.)