I Knew What AI Could Do. I Forgot to Think That Way.

What 48 hours of OCR tuning taught me about unlearning the old way of building.

I spent nearly 48 hours building a better way for software to read a movie or television title from a screenshot. I should have asked the model to look at the screenshot.

That is uncomfortable for me to admit. For more than a year, I have worked extensively with frontier AI tools. I have built products, workflows, and evaluation systems with them. I use LLMs across nearly every area of my work and many areas of my personal life. I think of myself as an AI-forward builder.

And still, when this particular problem became difficult, I didn't think of it in an AI-forward way. I reached for the architecture I already knew.

The task sounded straightforward: take a screenshot from a short-form video feed and identify the movie or television title shown on it.

In practice, the title might be stylized, split across lines, placed over a moving image, dimly lit, or partially obscured. The goal was not to recover a plausible string of text. It was to identify the correct title. A wrong answer could contaminate everything downstream, so “I don’t know” was better than a confident mistake.

My first approach was entirely reasonable. I tried multiple OCR methods. I generated different image crops and treatments. I cleaned the extracted text, removed interface noise, built candidate catalogs, added fuzzy matching, introduced confidence thresholds, validated candidates against external metadata, created a human-review queue, and began defining a golden set of known answers.

I kept iterating. Every failure produced another rule, another crop, another threshold, or another exception.

The system improved. In the slice I was reviewing, it could resolve roughly four out of five titles. That was good enough to feel encouraging and not good enough to trust.

The remaining fifth was not random. It contained the exact cases that were hardest to resolve safely: unusual lettering, partial words, ambiguous matches, and weak visual evidence.

I kept asking the same question: How do I make the OCR better? Then, after nearly two days, a more basic question finally arrived: Why am I asking OCR to do this job at all?

OCR is designed to turn pixels into characters. But a title treatment is not always ordinary text. It can be a logo, a composition, or a visual object. The shape of the letters, the spacing, the line breaks, and the relationship between partial characters can matter as much as the literal text.

OCR compresses that rich visual evidence into a fragile string. When it gets the string wrong, the rest of the pipeline is left trying to reconstruct meaning from damaged evidence.

A multimodal LLM can inspect the original screenshot without discarding its visual structure first. I could ask it for a visual reading, then use a constrained pass to compare that reading with a bounded set of candidates. If the evidence is insufficient or contradictory, the system leaves the item unresolved.

That sounds like a small change. It is not. It changes the level at which the problem is being solved.

The lesson is not to replace the entire pipeline with a prompt. It is to use each component where it is strongest. The model interprets the original image. Deterministic logic narrows the candidate set and applies hard checks. The system either records a supported match or leaves the item unresolved. The evidence and decision remain reviewable.

That is the experiment I have now built and am beginning to evaluate. If it works, the model could take on a narrow task I had reserved for a person: interpreting an ambiguous title treatment and proposing the best-supported candidate.

That is different from replacing human accountability.

The model has not earned automatic trust merely because the architecture feels better. Vision models can be confidently wrong. The new approach still has to be tested against an independent held-out set. It needs hard acceptance criteria, an explicit unresolved state, and a way to trace every answer back to the evidence that produced it.

The earlier work still matters. The candidate catalog bounds what the model may choose. OCR becomes an independent supporting signal. The review tool captures corrections. The golden set compares the methods honestly. Confidence gates determine what may be automated and what must remain unresolved.

The surprising part was not that a multimodal LLM might be able to read the title. I already knew these models could work with images. The surprising part was that this knowledge had not changed my first instinct.

Knowing how to use AI had not yet changed how I automatically framed the problem.

Expertise gives us familiar decompositions. When we see an image containing text, we think computer vision, OCR, cleanup, matching, and rules. Those techniques are concrete, inspectable, and controllable. Under pressure, they are comfortable.

So we often bolt an LLM onto the end of an inherited workflow instead of asking whether the workflow should still be decomposed that way at all.

The capabilities changed faster than my mental model.

I suspect this will happen repeatedly. We are not just learning new tools. We are learning when the architectures, workflows, and professional instincts that made us effective in the past may no longer be necessary.

Before building the intermediate machinery, we need a new reflex.

• What is the actual judgment the workflow is trying to produce?
• Can a current model perform that judgment directly from the raw evidence?
• Which steps exist only because earlier software could not reason?
• Which parts should remain deterministic because they create trust, control, and auditability?

The human defines what counts as truth, what evidence is acceptable, how much risk is tolerable, what must be tested, and when the system has to say “I don’t know.”

The model may perform more of the perceptual inference. The human still owns the standard.

This is not about forgetting everything we learned before. It is about letting go of the assumption that every technique still belongs in the same place.

OCR can remain useful. Golden sets become more important. Human review remains essential. But those components do not necessarily have to provide the primary intelligence anymore. They can constrain it, test it, and tell us when not to trust it.

I do not regret the 48 hours. I built much of the machinery I will need to determine whether the new approach is actually trustworthy.

But I had assigned the jobs incorrectly.

• The model should do more of the ambiguous visual reading.
• The surrounding system should constrain it, test it, preserve its evidence, and know when to abstain.
• The human should own the decision boundary.

AI-forward thinking does not mean abandoning rigor. It means moving rigor to the right place.

We are going to have to learn how to build again. Not because everything we knew is wrong, but because new capabilities change what should happen at each layer of the work.

That makes me wonder whether the next generation of software engineers will have an advantage. They will begin their careers assuming that models can see, reason, generate, and work alongside them. They may be less likely to decompose every problem according to the constraints of older systems because AI-forward thinking will not be a new behavior they have to adopt. It may simply be how they learned to build.

But experience remains an enormous advantage if we can separate enduring engineering principles from inherited implementation habits. Decades of software engineering taught us abstraction, testing, reliability, security, observability, and why plausible output is not the same as correct output. The leap is not to discard that knowledge. It is to preserve the rigor while becoming willing to redesign the path. The advantage may belong neither to the newest nor the most experienced builders, but to those who can combine AI-native instincts with hard-earned engineering judgment.

The most important new reflex may be the simplest: before spending another day improving an inherited step, ask whether that step still needs to exist.

---

This writing reflects my personal perspectives on product management, AI, and content discovery. It does not represent the official position of my employer or any affiliated organization.