← Back to Dave’s career
Case study · A large US video provider · 2026

Inspector Gadget

…or, David clones himself, hallucinations and all.

In plain terms: a detective's assistant for broken video. Give it a clue, and it digs through the records to find out what went wrong. (And like any AI, it sometimes makes things up. Those are the hallucinations, and much of this page is about catching them.)

An AI investigation helper (an “agent”: a program that can look things up and act on its own) that I built for a video-engineering team. It took what I knew about streams, telemetry and code, which had lived mostly in my head, and made it something anyone on the team could ask.

Role
Designer, builder, and sole maintainer (at first)
Built
Over about 4 months, alongside my regular work
Tools
For engineers: an AI coding-agent CLI, Python, a SQL warehouse, log search, and a real-time analytics store
~4months, first helper to handoff
6kinds of clue it accepts
Dozensof worked case files

Described in general terms. No proprietary data, code, customer details or colleagues' names appear here.

The problem

You know something's wrong. You can't prove it.

When streaming video goes bad, the cause can sit anywhere: the player, the manifest (an HLS playlist or DASH MPD: the file that tells the player which pieces of video to fetch), the packager (which cuts video into those pieces), the CDN (the network of servers that delivers it), ad insertion, the device, the network. The evidence is spread across several analytics systems, each with its own field names, plus raw session logs, plus the code.

One manifest you grab looks fine. So does the one from just before the ad break. But you know that something happens in a ten-minute window, and you can't show it. After twenty years of this, I could usually find it. The trouble was that I was the tool, and there was only one of me.

What I wanted was a black light for bad streaming experiences, one the whole team could hold.

What it looks like

Hand it what you've got

Everyone got the same starting point. You paste in whatever you have, and it picks the right playbook. Below is its opening screen, redrawn; you don't need to read every line.

🔍 How do you do? I'm the Inspector Gadget!
Hand me what you've got. I'll take it from there:
Error code or symptomlook it up → scope it → diagnose it
CSV of error dataerror rates, stability, model/platform comparisons
Session (visit) IDpull the full event timeline → classify what happened
Session JSONautomated triage: errors, buffering, frustration patterns
Experimentvariant-by-variant readout, quality metrics, soak assessment
Stream / manifest URLA/V sync, segment probing, origin vs. ad-stitched (server-side ad insertion) manifest comparison
▸ First time? I'll run a preflight check to see which tools you have.
How it was built

It started as a helper and snowballed

I didn't design this from the top down. I ran every experiment and investigation through AI anyway, to put my notes in order and write them up. Each new capability was checked against the ones already there.

  1. Experiment readouts

    I gave it data plus context. Higher quality score: good. Broken stream starts: bad. Playback failures: bad.

  2. The water-pump moment

    I asked it which fields in a raw session file matched the fields in the analytics warehouse. From then on it could connect them itself, and each correction I gave it stuck.

  3. Sources checking each other

    One copied log-search query taught it the whole log system. After that, the warehouse checked the logs and the logs checked the warehouse. A real-time analytics store came next.

  4. Stream probing

    A utility that takes a session, finds its manifest, and pulls and inspects segments over time: timing, sizes, MP4 box structure, gaps.

  5. Code introspection

    I connected the player, SDK and platform codebases, so every analytics event it saw had source code behind it. That let anyone read the code: read-only access, no local setup needed.

  6. Split into skills

    Dozens of focused skills (telemetry, field dictionary, streaming toolkit, experiment readouts, platform quirks…) loaded only when needed. That kept context small and cut down on hallucinations.

  7. A dossier of case studies

    Every real investigation became a worked example: the question, the method, what was found. By handoff there were dozens.

  8. Self-teaching

    At the end of a session: “Did we learn anything?” Good lessons got promoted into the skills. One-off details deliberately did not.

Architecture

Inputs, a small brain, lots of hands

Skip this part if you like. In short: you hand it a clue, it looks through the records, and it hands back a plain-English answer. Under the hood it’s an agent with written rules, small single-purpose tools, and a memory of past investigations that stays organized.

In

  • Error code or symptom
  • Session ID / downloaded session
  • Stream or manifest URL
  • Experiment + variants
  • Bug reports

Inspector Gadget

  • Instructionsinvestigation rules, code standards, report style, a polite and short voice
  • Dozens of skillstelemetry, field dictionary, streaming, platforms, experiments, code introspection
  • Python toolsstdlib only: segment probe, session analyzer, query builders, build checker
  • Local cachesdownloaded sessions and prior findings, so nothing is fetched twice

Out

  • Plain-English session readouts
  • Experiment readouts + queries
  • Repro steps & on-device tests
  • Release go / no-go verdicts
  • Markdown reports for the wiki
The loop: every investigation can become a case study, and every mistake it made became a written pushback rule. The next person to ask gets a smarter agent.
Case files

Things we couldn't see before

Anonymized, but real.

CASE · PLAYBACK ERROR

The error that started long before the error

One TV platform threw a playback error during paused live TV. Some viewers hit it far more than others. I asked Gadget to pull the full sessions of affected customers and find what they had in common.

Memory telemetry fell steadily, and every one of those sessions showed a burst of bitrate switches well before the error appeared. Because it knew the player's ABR internals (adaptive bitrate: how the player picks a video quality), it recommended doubling the minimum time between bitrate switches, so the player switches less often (in principle, up to about half as often).

CASE · MICRO-BUFFERING

Three little dots before every ad

A customer described “three dots before commercials.” Aggregate buffering metrics showed nothing.

Full session timelines showed sub-second stalls just before each ad break, and it pulled the buffer level going into every break from separate player events. The averages hid it; the sessions made it obvious.

CASE · CDN

Proving where the problem wasn’t

A late-night delivery issue on one stream type. We were fairly sure it was upstream, but we had to prove it.

Segment probing plus an origin vs. edge comparison gave us evidence to hand back. It took about two hours of my time, and most of that was the agent's time.

CASE · INCIDENT TRIAGE

An hour ahead of the room

During a live incident early on, I used Gadget in triage, asking "what does this do?" over and over.

It put me about an hour ahead of where I'd normally be. That was when I decided this couldn't stay my private tool. It had to be the team's.

What I learned

Rules for building an agent people can trust

90% right is the worst amount of right

It won't be right when you most need it to be. Parts that tend to guess, like config recommendations, carry explicit caution flags and "ask it three times" guidance.

If you can't find it in the data, stop

No session showing the bug means no theory about the bug. That one rule removed a whole class of confident nonsense.

Make the sources argue

Warehouse vs. logs vs. real-time metrics vs. the code itself. A finding that only one source supports isn't a finding yet.

Write tools for the next agent

Readable Python, no single-letter names, no magic numbers. Another AI, or a smaller model, has to be able to pick it up cold.

Skills over one giant prompt

Paste in everything and it "knows" things you never wanted it to use. Small skills, loaded on demand, stay cheaper and more honest.

Give it a voice, and take one away

Early on it promised deadlines and budgets it had no business promising. It now speaks politely and briefly, with no restating for effect.

Shared memory vs. personal memory

What's true for everyone goes in the dossier. One person's pet investigation stays in their own cache, so it doesn't clutter everyone else's.

You're a friend, not an adversary

When someone says the video looks blurry, start by trying to back them up. Quality metrics like VMAF (a picture-quality score) are there to support the complaint, not dismiss it.

The outcome

Built to be handed off

The point was never "look how clever Dave is." It was that this agent could be handed to anyone and help them as much as it helps me. I recorded walkthroughs, onboarding, a preflight checker, a maintenance routine, and an index of every case study, so a team could keep using it without me.

The pace in the demos was real. Investigations that used to swamp me ran while I made dinner. The agent let the team ask the questions they used to bring to me, and it keeps learning.