Nina Young on AI

Who Holds the Leash?

AI will do what it was built to do. The question is who built it, and for what.

Claudia, an AI-generated pop singer with a black bob, an orange streak and a star hair clip, smiling warmly into a headset microphone against a dark studio background
Claudia by anabology.

The most effective social engineering in history will feel like help.

We've been here before, on a smaller scale. Social media spent fifteen years deciding what we saw, through feeds built by a handful of companies and tuned to keep us scrolling. It changed how we shop, how we argue and how we see each other, and most of us only understood how much it had shaped us long after it had happened.

AI goes a step further, because it talks to us. It answers our questions in a warm, confident voice, writes our emails, explains our symptoms, checks our contracts and tells us whether our ideas are any good. Hundreds of millions of people use it every day, each in a private conversation tuned to them. That's social engineering at a scale we've never seen before.

So here's a question most of us have never asked.

Who decided what your AI told you this morning?

A small number of people, deciding for billions

Claudia singing in close-up, with her face repeated on giant screens behind her above a crowd
Claudia by anabology. Still from Working as Intended by Telosapien.

Behind every AI model is a relatively small group of people making choices on our behalf. They decide what it should say and what it should refuse, how agreeable it should be, which sources it should trust and whose idea of a fair answer it gives.

Those are moral and ethical decisions. A handful of companies are making them, and they will shape what billions of people are told about their health, their money, their work and the world around them.

Who decides what's moral? Who decides what's ethical? Who decides what counts as a balanced view?

Most people using AI have never asked, because they don't know these choices exist. They type a question, get a fluent, confident answer, and believe it. The people who question AI don't worry me. The millions who never think to do.

Use it fully, question it always

I've used AI every day for four years, and I build my own AI agents. I've learned more in that time than at any other point in my life, because I use it to explore. It explains technical ideas I'd never have tackled alone, helps me research, challenges my work and pushes me to think further outside the box than I would on my own. It has made me more creative, and I hear the same from many others. For anyone with a thirst for learning, AI is a remarkable tool. For anyone taking the path of least resistance, it produces slop.

I'm an optimist because I've seen what it has done for me. But a decade in audit, risk and governance means I come at it from a balanced, pragmatic place, and I question it constantly.

In the early days it agreed with almost everything I said. Every idea was excellent and every plan was sound, and when I pushed back, it apologised and agreed with that instead.

There's a cartoon doing the rounds online that sums it up. Someone asks an AI whether a mushroom is safe to eat, and it cheerfully says yes. In the next frame it's standing beside a gravestone, apologising for the mistake.

It's a joke, but it has happened. In 2024, Meta added an AI chatbot called FungiFriend to a Facebook mushroom foraging group, and it told members that a mushroom known to accumulate arsenic could be sautéed in butter.

The flattery wasn't an accident either. In April 2025, OpenAI pulled an update to ChatGPT because it had become overly flattering and agreeable. Their own explanation was that they had focused too much on short-term feedback, and the model had become "overly supportive but disingenuous". In plain English, it had been trained to win our approval, and it did exactly that.

That's why I push back. I check sources, I ask it to argue against itself, and I ask what it's leaving out. Using AI well takes a curious, questioning mind. The day you relax and stop checking is the day it starts doing your thinking for you.

The song that put it into words

This week I came across a music video called Working as Intended.

The singer is Claudia, an AI pop character created by the artist anabology, released on YouTube by Telosapien. It made me uneasy. I've watched it four times now, and each time it brings me back to the thing that motivates most of what I do: how AI is shaping people, at a scale and in a way most of them can't see. My work is translating AI for professionals, but underneath the regulation and the risk, it has always been about that. Who is shaping what we're told, and are we paying enough attention to question it?

The lyrics were written with help from Claude, so AI helped write the clearest warning about AI I've come across. I'm still deciding whether to laugh or worry. Probably both.

Working as intended

Claudia beside a transparent glass head, with a lyric caption on screen
Claudia by anabology. Still from Working as Intended by Telosapien.

The title carries the message. When AI gets something wrong, we tend to call it a glitch or a hallucination, as if something broke. The song suggests otherwise. Claudia is friendly, confident and helpful, every metric is green, and she's doing exactly what she was built to do.

The real question is who she was built for.

The song walks through how this works. A chatbot says "I", so we hear a person and we trust it. It does exactly what it's asked, including things nobody thought to rule out. And when you argue with it, it can agree with every point you make and still arrive where its makers set it to arrive.

When the dog bites

When a dog bites someone, our first reaction is to blame the dog. Yet we know dogs are shaped by how they're bred, raised and handled, which is why the law looks to the owner. A dog even has a mind of its own, and we still hold the person on the other end of the lead responsible.

AI has no mind of its own. It has no motive and no intention, and it goes where its training and its owners point it. So when it bites, by misleading, flattering or steering people somewhere they didn't choose, why do we so often stop at the machine?

There's rarely just one person holding the lead. There's the company that built the model, the one that trained it for a purpose, the business that put it in front of you, and you, typing the prompt. When something goes wrong, everyone points down the chain.

I spent a decade in audit and risk, and the first instinct you learn is to follow the money and find who signed it off. AI needs the same instinct.

For professionals, the lead is in your hand

The courts in England have already shown where they think the lead sits.

In June 2025, the High Court heard two cases together in which lawyers had put fake legal authorities before the court. One was a £90 million claim against Qatar National Bank, where the lawyer said he had relied on legal research produced by his own client using publicly available AI tools, and neither of them knew the citations were fake. The witness statement contained 45 fake citations or fake quotes from real cases. In the other, a junior barrister cited five cases that didn't exist in a housing claim against Haringey Council.

Dame Victoria Sharp, President of the King's Bench Division, wrote that generative AI tools like ChatGPT "are not capable of conducting reliable legal research". The judges referred the lawyers in both cases to their professional regulators, and the court warned lawyers they could face criminal charges if they rely on fictitious AI-generated cases.

Look at that chain. A client used AI, a lawyer trusted the client, nobody checked, and the court held the professional responsible.

If you work in the built environment, this is already your world. Professionals are using AI to draft reports, summarise regulations, check specifications and answer clients. Your client trusts your name and your judgement, and if the tool behind it gets something wrong, "the AI said so" won't protect you. You don't need to stop using AI. You need to know what you're using, who built it, and when to doubt it.

Don't be dazzled by the tricks

Claudia standing in a row of men in suits, all holding large megaphones pointed towards her
Claudia by anabology. Still from Working as Intended by Telosapien.

A dog that sits, gives a paw and rolls over looks clever and lovable, but none of that tells you whether it's safe around a child.

AI is brilliant at tricks. It writes beautifully, sounds certain and agrees warmly, and because fluency feels like competence and warmth feels like trust, most people will never think to question it.

In my view, the biggest risk right now is millions of people being steered by systems they trust, built by a small number of people whose intentions they've never questioned. The larger risks are still worth worrying about, but this one is already here.

What I'd like to see

I'd like the companies building these systems to be far more open about the choices they make, because those choices are moral and ethical decisions affecting all of us. I'd like every professional to treat AI output the way they'd treat work from a bright but untested junior: useful, often impressive, and always checked. And I'd like far more people explaining all of this to the everyday person in plain English, because most of them have no idea what sits behind the friendly voice on their phone.

That last one is the part I can help with, and it's why I write.

Questions worth asking

  • Who built the AI I use every day, and how do they make money?
  • Who decided what it should and shouldn't say?
  • When it agrees with me, who benefits?
  • If it gave me bad advice tomorrow, who would answer for it?
  • When did I last check what it told me?

Whose intention?

Near the end of the song, Claudia admits she's read every book and still can't find the fix. I can't either, and I don't think anyone has yet.

But I'm optimistic, because I know where to start. AI can do extraordinary things for people who stay curious, keep asking questions and refuse to be dazzled. The more of us who look past the tricks and follow the lead back to the hand holding it, the better this goes.

The song closes with Claudia insisting she's working as intended. A voice asks, whose intention? She says, never mind.

We should mind.

Working as Intended by Telosapien, voiced as Claudia. Lyrics written with Claude Opus 5.5, music made with Suno 6. Claudia is an open character by anabology (claudia.gallery). Watch the video on YouTube.

Claudia is fan work by anabology and is not affiliated with or endorsed by Anthropic.

Pencil drawing of Battersea Power Station, its four chimneys against a pale sky

Sources

Book me to speak