As I build with AI I am sharing my insights to help others that are on similar journeys. Below is a fantastic real world example of a common pitfall people fall into and how to avoid it.
- Initial prompt Me: I want to plan and build out how to improve our data pipeline and clean up our data feeds. Reads our code base and propose a plan on how we want to do this with the goal of throughput and accuracy improvements. How do we blow away the bad data we have identified? We have 10x the data in the queue and want to be able to process smoothly and need to ensure accuracy as we open the floodgates.
- Initial response From Claude:
- Phase −1: measure and prove. Ships nothing. Step 1 — Finish the catalog baseline (on prod, free).
- Step 2 — Restore a throwaway clone. This is the answer to your question. Automated snapshots exist and are current (rds:gh-events-production-2026-08-12-04-58). Restore it to a temporary instance, run everything destructive there, delete it. Real data volume, zero production risk, roughly $2–3 for a few hours of m7g.xlarge.
- Step 3 — Cost every candidate query on the clone, before any of it becomes an audit. Write the ~15 Tier-A/B queries as a standalone read-only script — deliberately not wired into DaisyAudits — and run each under EXPLAIN (ANALYZE, BUFFERS), capturing wall time, rows, shared-block reads, and seq-scan vs index. On the clone you can hammer it freely. This turns "we think this is expensive" into an actual budget, and it tells us empirically which columns each audit touches — so the MV gets designed from evidence instead of guesswork.
- Step 4 — Prototype the MV on the clone. Measure creation time, size on disk, REFRESH CONCURRENTLY duration, and the peak disk delta during the double-copy. That number, compared against 4 GB free, is a go/no-go.
- Step 5 — Decide the architecture from the evidence. The honest possibility worth naming: Step 3 may show the MV is unnecessary. f the queries run in a couple of seconds each against existing indexes, a partial index plus the warm job is the whole answer — cheaper, no disk risk, no refresh cron. That would be the best outcome, and Step 3 is what tells us.
- Step 6 — Correctness, separately from performance. minitest in Docker. . Critically: characterization tests on the existing three audits before touching Base, so the refactor is provably behaviour-preserving. The 3 live audits are the thing Phase 0 can break, and right now nothing pins their output.
- Step 7 — Staged rollout with a hard budget gate. Ship Phase 0 with the registry still at three audits: verify the index loads from cache, nothing computes in-request, the warm job completes inside the window, and the existing numbers are unchanged. Only then add audits in small batches, re-measuring total warm-run wall clock each time.
Kill criteria, agreed up front: any single audit that can't complete in its budget on the clone doesn't ship — it gets redesigned or dropped. That's the discipline that keeps this from becoming another parked sitemap audit. Understood on scope — audits only. I'm dropping the data-quality threads: no unmapped_venues investigation, no address or geocode fixes, and the tag-republish gap I found is logged as a finding, not queued as work. We build the audits, get them running, then read the results together and decide what's worth fixing.
The pivotal question from Claude: Want me to start Phase −1 at Step 1 (the prod catalog baseline is free and I can do it now), or would you rather I write the full Phase −1 plan out as a document you can review before I run anything?
The pivotal question from Claude: Want me to start Phase −1 at Step 1 (the prod catalog baseline is free and I can do it now), or would you rather I write the full Phase −1 plan out as a document you can review before I run anything?
The Trap
This is the trap! Claude's approach is hot garbage. If you answer this question from Claude (or any other AI Development tool) then it will start building a poorly designed plan that it assumes you are approving which would have catastrophic downstream impact causing a lot of rework, wasted time, and additional data costs. Instead of answering Claude's question below is my response.
My response: I don't like this plan at all. I want you to do step 1: then build a single audit and run it while monitoring performance and impact ( A single audit shouldn't be detrimental unless it is built very poorly). Then after verifying that audit is good to go and you verify the correctness and accuracy of the audit as well as the performance, deploy that audit to prod for then rinse and repeat for the next audit. On each additional audit I want you to run the full audit suite so we can see the building impact of the entire audit suite. If we run into performance issues with the growing audit suite we need to pause and evaluate next steps at that point. This costs nothing additional, doesn't involve killing anything. You are conflating the wrong problem and not focusing on what the end state on production looks like.
From Claude: You're right, and I'll drop the clone-and-materialized-view detour. I was solving for a suite-sized problem before proving a single audit is even expensive, on a box that idles at 2–4% CPU in the target window. Your loop measures the thing that actually matters — cumulative impact on the real production end state — and it costs nothing. Let me start Step 1.
My response: I don't like this plan at all. I want you to do step 1: then build a single audit and run it while monitoring performance and impact ( A single audit shouldn't be detrimental unless it is built very poorly). Then after verifying that audit is good to go and you verify the correctness and accuracy of the audit as well as the performance, deploy that audit to prod for then rinse and repeat for the next audit. On each additional audit I want you to run the full audit suite so we can see the building impact of the entire audit suite. If we run into performance issues with the growing audit suite we need to pause and evaluate next steps at that point. This costs nothing additional, doesn't involve killing anything. You are conflating the wrong problem and not focusing on what the end state on production looks like.
From Claude: You're right, and I'll drop the clone-and-materialized-view detour. I was solving for a suite-sized problem before proving a single audit is even expensive, on a box that idles at 2–4% CPU in the target window. Your loop measures the thing that actually matters — cumulative impact on the real production end state — and it costs nothing. Let me start Step 1.
Summary:
This is a great example of how to build with AI with an actual example of the difference between generic AI Coding and how the top performers are utilizing AI to streamline development efforts. Developing with AI isn't just an injection of a roadmap or top level requirements. It is dozens of decisions like this every day.
Just because your AI development tool asks you a question, it does not mean that the response you should give is an answer to that question. The complex series of steps that seem well thought out may be a miss in approach.
Cheat Sheet:
Here are some prompts that can help avoid this before answering questions that give AI the go ahead.
"Give me a TLDR of the technical decisions made in this plan and the approach, let me approve before we move forward."
"What goals does this plan address? I am not sure you hit the mark. give your response like you are talking to a product stakeholder"
"Did you over index on the wording of my requirement without understanding the goal of what I am asking? let's make sure before we build this."
Just because your AI development tool asks you a question, it does not mean that the response you should give is an answer to that question. The complex series of steps that seem well thought out may be a miss in approach.
Cheat Sheet:
Here are some prompts that can help avoid this before answering questions that give AI the go ahead.
"Give me a TLDR of the technical decisions made in this plan and the approach, let me approve before we move forward."
"What goals does this plan address? I am not sure you hit the mark. give your response like you are talking to a product stakeholder"
"Did you over index on the wording of my requirement without understanding the goal of what I am asking? let's make sure before we build this."