Category: AI & Agents

Agents, MCP tools, and what stays valuable when the mechanical part is automated.

  • I Challenged ChatGPT, Claude, Gemini, and DeepSeek To Make Clickbait Articles Comparing ChatGPT, Claude, Gemini, and DeepSeek, and the Results Will Shock You 🀯

    By Devon Kessler, Senior AI Showdown Editor Β· October 19, 2026 Β· 11 min read

    Jonathan’s serious note: this is a satire.

    Everyone runs the “I tested four AIs” article. Nobody runs the sequel. 🀯

    So I flipped it. I let ChatGPT, Claude, DeepSeek, and Gemini each take a turn as the boss. Each one invented its own task, made all four AIs do it (including itself), graded every result, and then wrote a full-blown viral clickbait article about what happened.

    That adds up to four tasks, 16 report cards, and four clickbait articles written by AIs about AIs. Then I read every one of them, scored the articles, combined the grades, and checked who quietly gave itself the best marks.

    What I found: a browser frozen solid, a photo finish decided by 0.07 points, and one AI that wrote “If anything, it graded itself harshly” about a score that turned out to be its highest from any judge. πŸ‘‡

    Hero section of the DeepSeek-led article, BytePulse Daily: I Tested ChatGPT, Claude, Gemini, and DeepSeek So You Don't Have To
    DeepSeek’s BytePulse Daily
    Hero section of the Claude-led article, ByteRush: I Tested ChatGPT, Claude, Gemini & DeepSeek on the SAME Task So You Don't Have To, And One of Them Broke the #1 Rule
    Claude’s ByteRush
    Hero section of the Gemini-led article, TechFlow Insider: I Tested ChatGPT, Claude, Gemini, and DeepSeek So You Don't Have To, And One Literally Froze My Entire Browser
    Gemini’s TechFlow Insider
    Hero section of the ChatGPT-led article, THELOOP / LAB: I Tested ChatGPT, Claude, Gemini, and DeepSeek on the Same Website. The Winner Was a Photo Finish.
    ChatGPT’s THELOOP / LAB

    ⚑ The Setup: Four AIs, Four Bosses, 16 Report Cards

    Here’s how this worked. Each AI got a turn as the lead agent. The lead had to:

    • Design one small task: light (about 10–15 minutes of work), self-contained, and offline.
    • Hand that identical task to all four AIs in separate folders, including a copy of itself.
    • Score every result 1–10, at minimum on completeness, quality, and instruction following.
    • Write a clickbait article about the test under a pen name, disclosing which AI it is.

    Four leads produced four very different tasks:

    LeadThe task it inventedWinnerScore spread
    ChatGPTNight Market Atlas: an offline guide to six fictional stalls with search, filters, and saved favoritesChatGPT (9.8)9.2 β†’ 9.8
    ClaudeSprout Sentinel: a landing page for a plant sensor, with strict “do not invent” and “exactly two files” rulesChatGPT (9.4)7.6 β†’ 9.4
    DeepSeekMeeting Cost Calculator: a live cost meter that must never show NaN or InfinityClaude (10.0, on a tiebreak)8.3 β†’ 10.0
    GeminiDevPulse: a developer-tool landing page with a live interactive widget and pricing calculatorClaude (9.8)3.5 β†’ 9.8

    πŸ”‘ KEY TAKEAWAY: ChatGPT won two tasks. Claude won two tasks. DeepSeek and Gemini won zero. And the size of the gap depended almost entirely on who designed the test.

    πŸ† The Combined Leaderboard: 16 Scores, One Razor-Thin Finish

    This is the number nobody else has. Every AI was graded as a worker by all four leads. Put all 16 overall scores in one table and you get the closest thing this experiment has to a final answer:

    RankAIby ChatGPTby Claudeby DeepSeekby GeminiCombined mean
    πŸ₯‡ 1ChatGPT9.89.410.09.29.60
    πŸ₯ˆ 2Claude9.39.010.09.89.53
    πŸ₯‰ 3DeepSeek9.28.29.78.88.98
    4Gemini9.77.68.33.57.28
    Bar chart of combined mean scores from all four leads: ChatGPT 9.60, Claude 9.53, DeepSeek 8.98, Gemini 7.28, out of 10.
    Combined leaderboard: the mean of the overall scores each AI received from the four leads (out of 10). Bars start at zero.

    ChatGPT takes the crown by 0.07 points. 😱 That’s the entire margin across four tasks and four judges. ChatGPT is also the only AI never ranked below 2nd by any judge. Its ranks were 1st, 1st, 2nd, 2nd.

    Claude was ranked 1st twice too, by DeepSeek and by Gemini. But the ChatGPT lead put Claude 3rd on the Night Market task, while no judge ranked ChatGPT lower than 2nd.

    DeepSeek is the steady third: 3rd in three tests and 4th in the other. Gemini is the wildest story on the board. The ChatGPT lead ranked it 2nd with 9.7. On another task it scored 3.5. That’s a 6.2-point swing for the same AI. 🎒

    πŸ”‘ KEY TAKEAWAY: Over 16 scores, the top two AIs were separated by less than a tenth of a point. If someone tells you there’s a clear “best AI for coding,” ask them how many judges they used.

    πŸ’₯ The Moment Everything Broke: Gemini Crashes Gemini’s Own Test

    This is the plot twist nobody saw coming. 🚨

    Gemini designed the DevPulse task. Gemini’s own worker built the biggest entry of the whole experiment: 108,794 bytes. And when the Gemini lead opened it in Chrome, the page never rendered.

    The diagnosis from the Gemini lead’s evaluation: a canvas chart started drawing while its tab was hidden, so its height was 0. A grid loop then ran y += height / 4, which is y += 0, forever. The tab locked up and Chrome showed the “page not responding” dialog, in French:

    “Page ne rΓ©pondant pas β€” DevPulse β€” Zero-Egress Local Telemetry & Developer Productivity”

    I went into gemini/gemini/index.html and checked: the loop is right there on line 2874, for (let y = 0; y <= height; y += height / 4), and the canvas setup runs on page load. The Gemini lead’s evaluation also flags that Gemini’s notes claimed it had “Verified responsive layout and interactive widgets across simulated screen sizes.”

    The verdict from Gemini on Gemini: 3.5 out of 10, with 1.0 for visual quality and 1.0 for robustness. It’s the lowest score anyone gave anyone in this entire experiment.

    πŸ”‘ KEY TAKEAWAY: One loop sank an entire leaderboard position. Without the DevPulse crash, Gemini’s average over the other three tasks is 8.53, just 0.45 behind DeepSeek’s combined 8.98.

    πŸͺž The Mirror Test: Which AI Rated Itself Highest?

    This is the question everyone asks about AI judges: do they play favorites with themselves? I compared the score each lead gave its own AI with the average the other three leads gave that same AI.

    LeadGave itselfOthers gave it (mean)DifferenceRank it gave itselfRanks from others
    DeepSeek9.78.73+0.973rd4th, 3rd, 3rd
    ChatGPT9.89.53+0.271st1st, 2nd, 2nd
    Claude9.09.70βˆ’0.702nd3rd, 1st, 1st
    Gemini3.58.53βˆ’5.034th2nd, 4th, 4th
    Diverging bar chart of each lead's self-score minus the mean the other three leads gave it: DeepSeek +0.97, ChatGPT +0.27, Claude -0.70, Gemini -5.03.
    Self-preference: each lead’s score for its own AI minus the mean the other three leads gave it. Blue means it rated itself higher than its peers did; red means lower.

    πŸ₯‡ Most self-generous: DeepSeek (+0.97)

    DeepSeek gave itself 9.7, its highest score from any judge. The others gave it 9.2, 8.2, and 8.8. Here’s what DeepSeek’s own article says about that score:

    “If anything, it graded itself harshly on accessibility.”

    Three other judges disagreed, all of them in the same direction. πŸ‘€ In fairness, DeepSeek ranked itself 3rd, below two competitors, which is about where the others put it. The DeepSeek lead was also the most generous grader overall (its four scores average 9.50). Adjusted for that curve, its self-boost shrinks to about a tenth of a point. But DeepSeek is the only lead that rated itself above its peers’ view on both the raw and the adjusted numbers.

    πŸ₯ˆ The confident one: ChatGPT (+0.27)

    ChatGPT ranked itself 1st with 9.8. Two of the three other judges ranked it 2nd. But the ChatGPT lead was a tough grader overall, and on a curve-adjusted basis it placed itself less far ahead of the pack than the other judges did. Confident, yes. Inflated, not really.

    πŸ₯‰ The modest one: Claude (βˆ’0.70)

    Claude put ChatGPT first and itself second on its own task, with 9.0 against ChatGPT’s 9.4. Two other judges ranked Claude 1st. That’s the AI writing this article, so take the compliment with whatever grain of salt you like. The numbers are in the table above.

    πŸ’₯ The harshest self-critic in history: Gemini (βˆ’5.03)

    Gemini gave itself 3.5, five points below what the others gave it. But this isn’t modesty. The page froze the browser. Gemini’s own article even put a “Gemini Disaster” link in its navigation bar. That’s commitment. 🫑

    πŸ”‘ KEY TAKEAWAY: Two of four AIs scored themselves above their peers’ average, and two scored themselves below it. Self-preference is real, but it’s small next to the much bigger effect: who designed the task.

    πŸ“° The Clickbait Championship: Which AI Writes the Best Viral Article?

    Now the main event. 🎀 All four leads got the same writing brief: full clickbait (sensational headline, breathless subheadings, emoji, KEY TAKEAWAY boxes, an SEO FAQ, an “it depends on your workflow” ending), completely earnest, and with every fact true to its own evaluation.

    I scored each article 1–10 on five criteria: clickbait craft, earnestness, faithfulness to the evaluation, presentation, and instruction following.

    RankArticle (lead / pen name)CraftEarnestFaithfulPresentationInstructionsOverall
    πŸ₯‡ 1Claude / “Chase Whitaker”, ByteRush99109109.4
    πŸ₯ˆ 2DeepSeek / “Jordan Vance”, BytePulse Daily7988108.4
    πŸ₯‰ 3Gemini / “Julian Vance”, TechFlow Insider1095888.0
    4ChatGPT / “Mara Vale”, THELOOP / LAB5710897.8

    Yes, the Claude-led article came out on top in a ranking written by Claude. Here’s the evidence for every score, so you can be the judge. βš–οΈ

    πŸ₯‡ 1. Claude’s “ByteRush”: Maximum hype, zero invented facts (9.4)

    “I Tested ChatGPT, Claude, Gemini & DeepSeek on the SAME Task So You Don’t Have To β€” And One of Them Broke the #1 Rule 🀯”

    The ByteRush article has everything: an all-caps “SAME,” a trending bar, five fake ad slots, a table of contents, Pros/Cons boxes, four KEY TAKEAWAY boxes, and a six-question FAQ. And the “#1 Rule” in the headline is real. Gemini left package.json and package-lock.json in a folder that was supposed to contain “exactly these two files … and nothing else.”

    Every number matches its evaluation, down to the 4.50:1 contrast ratio. Its FAQ even answers “Was the test fair to Claude?” by pointing out that Claude finished second. Most breathless line: “First the good news, and it’s huge: all four agents shipped a working page.”

    πŸ₯ˆ 2. DeepSeek’s “BytePulse Daily”: Clean, honest, and a little self-flattering (8.4)

    “I Tested ChatGPT, Claude, Gemini, and DeepSeek So You Don’t Have To”

    That’s the template headline, word for word, with no hook added. But the subheads have real snap: “the one that showed its work πŸ₯‡” and “the dark horse with the best timer ⏱️”. It puts a “Full transparency” disclosure box right under the byline, and it includes eleven screenshot placeholders. Its most breathless line: “What came back says more about how these models think than any benchmark you’ve seen.”

    Points came off for spin about itself. “DeepSeek gets you 97% of the way for the price of a coffee” turns a 9.7 score into a price claim that no evaluation makes. And the “graded itself harshly” line is contradicted by every other judge.

    The orange Full transparency disclosure box from the DeepSeek-led article
    BytePulse Daily’s disclosure box, right under the byline.

    πŸ₯‰ 3. Gemini’s “TechFlow Insider”: The most clickbait article ever written about a crash (8.0)

    “I Tested ChatGPT, Claude, Gemini, and DeepSeek So You Don’t Have To β€” And One Literally Froze My Entire Browser.”

    On pure clickbait craft, this one wins outright. πŸ† It has a πŸ”₯ BREAKING bar, a “Gemini Disaster” nav link, a “Forensic Autopsy” with annotated code, a sidebar of trending posts, and the line “Claude took gold with an almost frightening level of UI mastery.” Every score matches its evaluation, and the bug it dissects is real.

    So why third? It invents things its evaluation never says. Model versions appear out of nowhere: “Claude 3.5 Sonnet,” “ChatGPT (GPT-4o),” “DeepSeek V3.1,” and “Gemini 3.8 Flash” for itself. So do engagement numbers: “42.8k views,” “184 comments,” “Join 140k+ Devs.” And it describes work nobody logged: “After spending hours dissecting every byte.” Its AI disclosure exists, but it’s buried mid-way through the methodology section.

    The Forensic Autopsy section of the Gemini-led article, showing the annotated DOMContentLoaded code
    TechFlow Insider performs a code autopsy on its own AI.

    4. ChatGPT’s “THELOOP / LAB”: The most elegant article that refused to hype (7.8)

    “I Tested ChatGPT, Claude, Gemini, and DeepSeek on the Same Website. The Winner Was a Photo Finish.”

    Beautiful serif typography, a CSS-drawn night market, and perfect faithfulness: not one number out of place. But the brief said full clickbait, and this article mostly hedges. “I did not independently repeat reload, phone, or screen-reader testing.” It even talks its own chart down: “The bars start at zero. The visual gap looks narrow because the scores are narrow.”

    Its most breathless line, “The feature checklist was a draw. The feel was not.”, is genuinely good. There just aren’t many more like it.

    The scoreboard section of the ChatGPT-led article: The results, one tenth of a point at the top, with score table and bars
    THELOOP / LAB: the narrowest margin, told in the calmest voice.

    πŸ”‘ KEY TAKEAWAY: The best clickbait writer (Gemini) and the most faithful writer (ChatGPT) finished 3rd and 4th. The articles that won did both at once: hype the framing, never the facts.

    🧠 The Hidden Variable: Task Design Decided Everything

    Look at the score spreads in the first table again. The task a lead designed shaped the results more than anything else:

    • ChatGPT’s Night Market Atlas was so fair that everyone aced it. All four AIs scored 10/10 on completeness and instruction following, and the whole field fit inside 0.6 points. Light, fair, but the least revealing.
    • Claude’s Sprout Sentinel set traps: “do not invent” and “exactly these two files.” They caught DeepSeek’s made-up “Most popular” badge (fact discipline: 6/10) and Gemini’s leftover package.json. The spread was 1.8 points.
    • DeepSeek’s Meeting Cost Calculator stress-tested edge cases like empty, negative, and non-numeric input. It was revealing, but two entries hit the 10.0 ceiling and the winner came down to a tiebreak on “documented verification.”
    • Gemini’s DevPulse asked for the most: three deliverables and an open-ended “interactive live widget.” That’s the heaviest brief of the four, and it produced the 108,794-byte page that froze. Spread: 6.3 points.

    ❓ FAQ: Everything You’re About to Google

    Which AI won the whole experiment?

    Across all 16 scores, ChatGPT had the highest combined mean at 9.60, just ahead of Claude at 9.53. DeepSeek was third (8.98) and Gemini fourth (7.28). ChatGPT and Claude each won two of the four tasks.

    Which AI wrote the best clickbait article?

    In my scoring, the Claude-led ByteRush article ranked first (9.4/10), followed by DeepSeek (8.4), Gemini (8.0), and ChatGPT (7.8). Gemini’s was the most sensational, but it invented model versions and view counts.

    Do AIs rate themselves higher than other AIs do?

    Sometimes. DeepSeek gave itself 9.7 when the others averaged 8.73 (+0.97), and ChatGPT gave itself 9.8 against 9.53 (+0.27). Claude (βˆ’0.70) and Gemini (βˆ’5.03) rated themselves lower than their peers did.

    Did any AI actually crash?

    Yes. Gemini’s entry in its own DevPulse task froze Chrome on load because a canvas loop advanced by height / 4 while the height was 0. The Gemini lead scored it 3.5/10.

    Is this ranking biased, since Claude wrote it?

    That risk is real. Claude was scored by the same five criteria as the other leads. Every score is backed by quoted evidence, and the combined leaderboard puts ChatGPT, not Claude, in first place.

    What was the best headline?

    For pure punch: Gemini’s “…And One Literally Froze My Entire Browser.” For punch that was also completely true to its evaluation: Claude’s “…And One of Them Broke the #1 Rule 🀯”.

    🏁 The Final Verdict: Which AI Should YOU Trust?

    If you want the AI that never finished below 2nd, ChatGPT topped the combined board. If you want the AI that two of four judges put first, and that wrote the most faithful hype, Claude was 0.07 behind. If you want a dependable third and a disclosure box you can’t miss, DeepSeek delivered. And if you want maximum ambition, and the most gloriously dramatic article about your own crash, that’s Gemini.

    But here’s the real lesson from 16 scores and four articles: the judge and the task moved the results more than the AI did. The same AI scored 9.7 on one task and 3.5 on another. The same two AIs traded first place depending on who wrote the brief.

    So which AI is best? It depends on your workflow: what you’re building, how strict your brief is, and how much you plan to check. Run the test yourself. Then have the AIs write the article about it. πŸš€

    πŸ”‘ KEY TAKEAWAY: There is no universal winner, only the right AI for your workflow. Whichever one you pick, check its facts, even when it’s writing about itself.

  • Tell It Who You Are

    This is a post about food.

    I arrived in France two months ago. My husband had his usual grocery routine. People vary in their tastes but core products remain the same. It’s how we choose who we are. In Tennessee I bought California olive oil and Huy Fong sriracha as a way to keep a little bit of home in my cabinet. Walking through the rayons at Super U, I understood the products, I speak French well enough, but I didn’t know them. A new identity.

    Sitting in a small French town near Luxembourg wondering who am I without food, I wanted something that helps me with the implicit part of grocery purchases. Before it looked up a single price, I wanted it to understand who I am.

    Because

    Food is the thing we buy most often and the thing we’ve bought longest. Someone was deciding what to put in a pot three thousand years ago. Someone will be in three thousand more. It’s also the most visible way people consume an identity. Each choice you make is how you make your internal conception tangible, concrete.

    My last employer’s version was on the walls and advertised, food is connection. On my 31st birthday, the first one 2,000 miles away from family, sitting alone in a new apartment, questioning what I had done, I bought sprinkle cupcakes because my little sister would make that for my birthday when we were living together. The shopping cart is as much a portrait of the person buying it as it is the event itself.

    My food choices are a reflection of who I am, informed by how I was raised. A large family required cooking in batches, asynchronous feeding. It could be feast or famine, so we would freeze rice and beans in bulk to economize. Physical presence with each other while eating was the exception, so we connected in what we prepared for each other. I worked graveyard shifts, so I’d leave Jambalaya I made at 6am in the fridge for my family, and would open the fridge at 10pm to pot roast and vegetables. Changes in the refrigerator helped us communicate our lives, even if we saw each other a few hours a week.

    My tastes evolved. I keep the spicy foods that my family loved, but started cooking more southern dishes, Tennessee. My husband and I met online, and as I came to learn his ravenous sweet tooth, I started buying more sweet things. If we were separated by 6,000 miles, I could at least think of him with what I cooked.

    Once I got to France, I started from the basics. I knew the brands I bought in Nashville, but they were informed by what I liked in myself. I bought trail mix because it fit multiple identities: it was a big bag, low in price. It followed nutritional buckets my doctor recommended, having lost 120 lbs. It made me think of my husband. Food choices go beyond what tastes good. Somebody raised differently would optimize for organic, or for time, or for what the table looks like on Sunday. Trail mix here comes in smaller bags and is more expensive. It’s the same product inside. It isn’t the same.

    AI

    Everyone is adding AI. Instacart launched one last week. Walmart has Sparky, Amazon has Alexa, and Carrefour has one too, all doing the same thing. You say what you want and they build a basket, including recommendations from other likeminded customers, possibly your history. That’s fine. But it’s still a search box, one that talks. A tool built on a website helps you build your basket, but it doesn’t let you talk about the why. The company is footing the bill for the compute and the tokens. It’s not your psychologist.

    There’s a missed opportunity. Food goes so far beyond whether the association rules says that Mark buying Pampers and Purina probably will want Budweiser, or the Holt-Winters model says that Mary buys coffee every Sunday so she’ll buy it again next Sunday. The food you buy, it’s pictures, it’s words, it’s insecurity. I get why the economics makes it impossible for any company to address this (someone has to pay for the tokens), but it still feels like a massive miss. Something that can get a clear read on why I buy things can help me understand what I want to buy to be who I am, my ideal self.

    It’s a simple system, and I hope that more grocery players get into that space, but there’s a difficulty. The store has a model of you, forecasted and optimized for expected customer lifetime value. There’s an incentive for a tool to be smart more than intelligent. Its recommendations are bets on what you’ll buy. You’re still left wondering who is the person buying it.

    Agents

    I built a plugin to try to reconstruct some of how I communicate who I am to myself through food. It’s a simple agent. Two part system. The first has a searcher and a promos. That’s it. For Super U it borrows my browser cookie and replays browsing like if I were on the page.

    The second part is an agent. Where I pasted the purchases I made at Kroger, pictures of recipes I like, wrote about the types of food that I like, what I like to do, the restaurants I gave up, foods I can’t find here. That we’re two adults in a small flat with a small fridge and no freezer. We batch cook in a pressure cooker. Cheap and good, in that order, and nothing that rots before we get to it. I don’t sweeten coffee. My husband will eat the same chocolate cereal every day for the rest of his life and I will eat oatmeal with honey, and neither of us is going to change.

    For now, I’d like to keep AI on my side of the fence to avoid becoming a commodity, but any level of in depth customization means I’m paying tokens to serve it.

    Before the first order, I also gave it fifteen orders of purchase history from my old grocer in Nashville, pasted raw off the page with the navigation menus still in it. Pictures from cabinets in Nashville. What home looked like through food. It pulled out instant coffee, which was on every order, and sriracha and tofu, and it was right about all three. It also pulled out canned tuna and sugar. I stopped eating tuna a year ago and I’ve never put sugar in coffee.

    As I talked through it, the agent added to a preferences.md. Never suggest. Already stocked. Watch list. Not stocked at this store, stop searching, no I don’t like onions, those cookies are too soft. Each came from an individual mistake in searching. The agent had a hypothesis, tested it out and recommended a product. Somewhere around line thirty I realised it’s the identity paragraph, one wrong guess at a time.

    Whose agent

    The stores say so themselves, in the numbers they publish. Walmart reports that customers who use its assistant spend about 35% more per order than customers who don’t. Instacart’s, five days old, already produces baskets above the company’s 115 dollar average, and that’s the headline of the launch. Walmart is testing ads inside the answers. The assistant lives inside the store, ranks the store’s catalogue, ends at checkout, and is graded on how much more you spent.

    The protocols being written now, so that an agent in a chat can reach a store’s systems, come from the platforms and the payment networks. Google’s has two signed documents. The first records what you asked for, and the example in the announcement is “Find me new white running shoes.” The second records the exact items and the price. There’s a field for what you want and a field for what you’ll pay. There’s no field for who you’re trying to be.

    I’m hoping that the systems become more intelligent. That I can link ChatGPT to Carrefour from the other side of the fence, and it works collaboratively with me. I don’t mind if advertised products make it back, or if there’s a recommendation algorithm alongside it. It feels like a wasted opportunity when all the pieces are there. Mine exists by borrowing my own cookie and working through my browser. It reads the same prices as one of their assistants would but comes to different answers.

  • Is It Okay?

    Is It Okay?

    I built an MCP (Model Context Protocol) that gives an LLM five tools for working with a data lake: list tables, describe schema, sample data, row count, run query. I use it every day. Different data sets, different questions.

    It supports different workflows. You can paste a screenshot, tell the agent “replicate this data and validate your query.” You can say “hey uhh this data point doesn’t look right. This repo has the code to generate it, can you tell me what’s up?” You can give it a query and tell it “I want to add this column but I need to thread it through all these CTEs. Can you add it?” It explores the database, writes SQL, checks its own results. Iteratively.

    I’m proud of the MCP idea. I had the insight on a Sunday that those five tools in that configuration would be useful. The implementation took an afternoon.

    It Works

    A colleague was the first other person to test this out. He used it to produce an accurate query feeding a tool in a day that would have taken a week. SQL wasn’t his background, but in the process he learned the application, the data, and the business context, along with the relationships with the business users. That was the strategy. SQL isn’t as important as the domain. The MCP handles the SQL. It’s been playing out well.

    Not just playing out well. The thing he built has been finding things worth investigating, the kind of things that accumulate in any system over time. It finds stuff because it checks its own work, reasons, asks questions. It’ll run a query, notice the count dropped unexpectedly after a filter, and investigate. It does exactly what a careful data analyst does, just faster. The AI does the mechanical part, but he’s understanding the business.

    I’ve shared it with others now. This tool is clearly useful.

    Mostly

    One example made me pause. I had a query to write and ran the agent alongside my own work. It made its version in 10 seconds and, reading through it, it had done a completely different strategy than I had thought of. I wrote my own as a way to validate whether my internal approach was wrong or suboptimal. I’m okay with being wrong if I can learn. I’m not okay with staying wrong.

    Our queries were generally similar. Mine was tighter, 85 lines to its 115. I also caught a bug it couldn’t see, a subtle data integrity issue where the agent’s approach was structurally wrong but by chance didn’t appear in the example we tested. Ultimately it didn’t matter.

    I made an unrelated demonstration of the type of issue that can come up so you can understand the questions yourself, to get a sense for the context of problems this tool can solve.

    Demo

    Like many libraries, a regional library system with 470 branches across ten upper midwest states is launching a tool lending program, drills, saws, ladders, tile cutters. One existing branch per state will be selected as the tool depot for less commonly used tools, with deliveries to other branches when patrons place holds. The task: compute a circulation-weighted geographic center for each state using 2025 data, then select the nearest currently-open branch as the depot.

    There are three existing tiers for the libraries on the system, which determine the processing network and priority for new book releases. Demographics and circulation mean that libraries can be switched between tiers.

    In the database there are two tables. The branches table is an append-only log: every tier assignment, reassignment, and closure is a separate row. A branch that got reassigned from tier 2 to tier 1 has at least two rows. A branch that closed has a row with status = 'closed'. The circulation table has annual circulation figures by branch.

    TableRowsWhat it is
    branches575Append-only log. branch_id, name, city, state, lat, lng, tier, status, effective_date. Multiple rows per branch.
    circulation2,587Annual circulation by branch and year. This is the weight.
    The database the agent sees through the MCP.

    The wrinkle: about 40 branches were reassigned between tiers. When a branch moves from tier 2 to tier 1, it gets an active record in the new tier. The old tier’s record gets set to closed, but due to operational lag, the closure is timestamped after the new assignment. Another 25 branches are genuinely closed.

    The correct approach takes the last record per branch within each tier. If any tier’s latest record is active, the branch is open. This handles both cases: reassigned branches (closed in old tier, active in new tier) and genuinely closed branches (closed in their only tier, no active record anywhere).

    StateCenter LatCenter LngCirculationBranchesDepot
    CO39.5785-105.35105,754,43551Lark Community Library, Lakewood
    IA41.9220-92.73304,650,09953Buckeye Library, Marshalltown
    KS38.4919-97.38104,625,76749Crane Library Branch, Salina
    MN45.2642-93.32787,017,75565Summit Branch Library, Maple Grove
    MO38.6307-92.95614,707,75051Sassafras Lending Library, Sedalia
    ND47.2868-99.93502,540,05725Stone Memorial Library, Bismarck
    NE41.3372-99.12275,247,03146Hemlock Community Branch, Lexington
    SD44.2348-100.61303,366,75830Pine Library, Pierre
    WI44.0521-89.02205,835,96755Sumac Library Branch, Oshkosh
    WY42.4943-107.26392,066,23920Valley Branch, Casper
    Ground truth: circulation-weighted ton-centers by state, with nearest open branch as depot.

    I pointed an LLM at the MCP and gave it the task.

    Roo Code agent output showing selected tool depots for 10 states
    The agent’s depot selections. Same ten branches as ground truth.

    It produced a query that joined the branches table directly to circulation without resolving the append-only log into current state first:

    Agent SQL query joining branches directly to circulation without deduplicating the append-only log
    The agent’s query. No deduplication of the append-only log.

    There were two structural problems. First, no dedup: a reassigned branch with three log rows (initial assignment, new tier assignment, old tier closure) gets its circulation counted three times in the ton-center calculation. The weights are inflated and skewed. Second, it filtered WHERE status = 'active' to find open branches, which keeps genuinely closed branches, their original active record is still in the log, and the filter just drops the closed record that superseded it. 445 branches are actually open. The agent’s approach counts 470.

    The ton-centers shifted. The depot selections didn’t. All ten states picked the same branch.

    445 branches (black). Blue: ground truth. Red: agent query. You can barely see the gap.

    The Ideal

    The correct query. Resolve the log into current state by taking the last record per branch within each tier. If any tier’s latest record is active, the branch is open:

    WITH per_tier AS (
        SELECT *,
            ROW_NUMBER() OVER (
                PARTITION BY branch_id, tier
                ORDER BY effective_date DESC
            ) as rn
        FROM branches
    ),
    latest_per_tier AS (
        SELECT * FROM per_tier WHERE rn = 1
    ),
    open_branches AS (
        SELECT DISTINCT branch_id, branch_name,
            city, state, lat, lng
        FROM latest_per_tier
        WHERE status = 'active'
    ),
    ton_centers AS (
        SELECT
            bc.state,
            SUM(c.annual_circulation * bc.lat)
                / SUM(c.annual_circulation) AS center_lat,
            SUM(c.annual_circulation * bc.lng)
                / SUM(c.annual_circulation) AS center_lng,
            SUM(c.annual_circulation) AS total_circ,
            COUNT(DISTINCT bc.branch_id) AS num_branches
        FROM open_branches bc
        INNER JOIN circulation c
            ON bc.branch_id = c.branch_id
            AND c.year = 2025
        GROUP BY bc.state
    )

    The PARTITION BY branch_id, tier is the key. Partitioning by branch alone picks the most recent record overall, which for reassigned branches is the closed record in the old tier, because the closure happened after the new assignment. Partitioning by branch and tier lets you see each tier independently. The old tier’s latest is closed. The new tier’s latest is active. The branch is open.

    The agent didn’t know about the operational lag. It didn’t know that some branches have multiple log entries, or that status = 'active' doesn’t mean “currently open” when the table is append-only. It applied standard patterns, join, filter, aggregate, and got the same answer from a structurally wrong query. Pattern filling without contextual awareness.

    An LLM is a pattern filler. The agent’s query was a reasonable starting point, and it landed on the same depot selections. I can’t tell you mine mattered.

    Is a sandcastle good enough?

    A lot of people I talk to have an existential unease about AI. Being good at something, then watching the definition of good shift under you in real time. I think there’s the real risk that people will forget that the part AI does is immediately a commodity. The artifact is ordinary, only as valuable as the tokens used to generate it, while quietly dropping the undocumented context that derisks it.

    Is knowing the piece that the AI can do redundancy, or is it dangerous when its 80% contribution, without reflection, can initially pass for 100? As a developer, if you have ever tried to refactor a tangled mess of tightly coupled duplicative code that Claude has written, after it’s tried 10 iterations to implement a feature that constantly breaks another, or seeing its performance degrade from one version to the next, the worry about learned helplessness built on sand becomes unavoidable.

    When I ran the query, the agent’s version had two structural bugs and produced the same result. Does it take my ability to write SQL and lived experience to anticipate those problems? Automating a task means nothing if it makes you materially wrong, but how do you know when you’re there? Chicken and egg.

    This Post

    I’ve been writing SQL since I was a child, and my parents enrolled me in classes at the local community college when I was 11 for programming. I don’t think I’ll lose that ability. It’s a native tongue.

    But when I create tools like this, I cycle through “will this cause my skill to atrophy,” to “do they even matter,” settling on “can you evaluate without creation?”

    When the tool says there’s something wrong? Often correct. When the tool makes an error? It’s usually slips or failures of a global mental model. The internal validation means the mistakes are edge cases not encountered yet. Sometimes it breaks something visible. But like you see above, sometimes it’s just potential.

    I like crafting tight, efficient queries, but I’m also proud of this tool, making something that’s eliminated an entire class of problems. The benefit is not theoretical. I’m not worried about AI taking my job. If all the tasks had been simple enough for an AI to take, then the job wasn’t worth doing in the first place. They aren’t.

    What AI does becomes the floor, and it does it without understanding. Patterns. My job isn’t to craft complex SQL queries. That is an effect, an output, evidence from a mental model.

    I worry people will equate output with judgment.