How We Review Code at Videoselz: Three Reviews, Risk Tiers, and Guardrails Against Data Loss

Merchants trust Videoselz with the videos on their live storefronts. When one of those videos stops working, a shopper sees a broken page and the merchant loses a sale. So we treat code quality as part of the product, not as engineering housekeeping.

This post covers how we review code before it reaches merchants: who reviews each change, how our AI review command /do-review works, and the extra checks we run on changes that could delete or overwrite data.

In this post

• Why we hold a high bar
• Three reviews on every change
• Inside /do-review
• High-risk and low-risk changes
• Guardrails against accidental data deletion
• Guardrails against regressions
• What review does not replace
• FAQ

Why we hold a high bar

A regression that reaches a merchant is a broken promise. A merchant who set up a video carousel last month expects it to work today, whether we shipped one change since then or fifty.

We learned how high that bar has to be earlier this year. A change shipped by one of our engineers had adverse consequences for some merchants, and the cleanup that followed caused problems for them and for our team. The change had been reviewed, and nothing in its diff looked wrong. We concluded that our review process had to catch problems that a careful reader of the diff would miss.

That incident is why the system below exists. It is not why we care about quality, which we always did. It showed us where care alone was not enough.

Three reviews on every change

No change reaches production on one person's judgment. Every pull request goes through three reviews, and each one looks for something different.

The author knows the intent best, and the self-review is where they record it, including the expected impact of the change. The AI reviewer does the slow, mechanical tracing that a person can skip at the end of a long day. The human reviewer brings judgment and accountability, and has the final say.

The layers overlap on purpose. Any single reviewer can miss something, and a miss has to get past all three to reach a merchant.

Inside /do-review

/do-review is a custom Claude Code command we built and maintain alongside our codebase. An engineer gives it a pull request link. It collects the diff, reads every changed file in full, loads our engineering conventions, and hands all of it to an AI reviewer.

Three rules shape how that reviewer works.

It assumes the change is broken

The reviewer is told to act as an adversarial senior engineer and to prove the change fails. It constructs the input, state, or timing that would break the change. "Looks fine" is not an accepted result. If it finds nothing, it has to say what it tried to break and why the code held.

It reads outside the diff

A diff shows where a change is, not where the bug is. Most production bugs come from code the pull request never touched: a caller, another code path that writes the same data, or a screen that reads a response whose shape changed.

So the reviewer must trace every call site of the changed code, every place that creates or updates the affected data, and every consumer of a changed response. For any field used to select records, it checks the field's history, because records created before a field existed may not have a value for it. Every review reports which files outside the diff it read, so the human reviewer can see what was covered.

It verifies claims instead of trusting them

A code comment saying "every path sets this field" is a claim, not a fact. The reviewer treats any statement about code outside the diff as unproven until it has read that code. Each finding must cite a file and line, or be marked unverified for the author to check. The rule applies to the reviewer's own output too.

What it focuses on

Style is out of scope. The review effort goes to five risks that can hurt merchants:

1. Destructive operations
2. Resource leaks
3. Unhandled failures and missing data
4. Data operations that slow down as a merchant's catalog grows
5. Breaking changes to APIs and data models

Findings are rated by consequence, not by how hard they were to spot. A one-line missing error handler that silently drops an upload is critical. A large refactor that carries no risk is not a finding at all. Convention checks still run, as a final pass that can never block a change on its own.

High-risk and low-risk changes

Not every change deserves the same scrutiny. A copy fix and a background job that deletes files carry very different risks, so /do-review sorts every pull request into one of two tiers before the review starts.

This decision is mechanical, not a judgment call. The command scans the diff for deletes, bulk updates, scheduled jobs, and the filters that feed them. Any match makes the change high risk. A delete that "looks guarded" gets no exemption, because the classification is meant to work even when the reviewer is wrong about how safe the code is.

For a high-risk change, the AI reviewer cannot give a plain approval. Its best possible verdict is conditional on human sign-off, and it states the number a person should check the job's dry-run output against. A model's opinion is not enough to ship something that cannot be undone.

Guardrails against accidental data deletion

Deleting a merchant's file from Shopify is permanent. There is no undo, so the checks for high-risk changes are designed to catch mistakes before any delete runs. The AI reviewer flags each of these as a critical finding:

• Selecting by absence. A job must delete records because something marked them for deletion, never because a match is missing. A missing value means "unknown", not "safe to delete".
• Unverified flags. If a job selects on a status or flag, the reviewer checks every code path that creates the record to confirm the flag is set correctly everywhere, including on older records.
• Unsafe empty inputs. If an input is empty or missing, the job must stop. It must never widen the query and match everything.
• No dry run. Jobs default to dry-run mode, where they log what they would delete and delete nothing.
• No abort cap. Each run has a hard limit on how much it can delete. Hitting the limit stops the whole run instead of trimming the list.
• Delete in the first release. A new destructive job ships with no delete call at all. Deletion is turned on in a later change, once dry-run counts match what we expect from production data.
• No grace period. Permanent deletes sit behind a quarantine. A record is marked first and removed only if it stays marked through the grace period.
• Hidden side effects. The reviewer lists everything a deletion removes along with it, because a silent side effect can break a live carousel.
• Wrong store. Background jobs run without a merchant session, so every query must be scoped to the store being processed, and the job confirms ownership before each delete.

// The pattern we require (illustrative): positive signal, cap, dry run
      const candidates = videos.filter(
        (v) => v.status === "removed_by_merchant" && v.removedAt < graceCutoff
      );
      if (candidates.length > MAX_PER_RUN) throw new Error("Cap exceeded, aborting run");
      if (dryRun) return log("Would delete", candidates.length);
      

Guardrails against regressions

Data safety is the highest-stakes check, but most of what reaches merchants is everyday product work. /do-review also checks that changes leave existing features working:

• Breaking changes across our apps. Our merchant app, storefront widget, and tracking pixel deploy on different schedules. When a change alters an API, the reviewer checks every consumer, including older versions of the widget still running on storefronts.
• Known fragile flows. Upload, cancel, retry, and concurrent-upload behavior is checked on every change that touches those paths, because those are the flows merchants use most.
• Missing tests. A change in behavior with no matching test change is flagged, because the path is either untested or its test is no longer checking anything.
• Repeated bugs. When a change fixes a bug, the reviewer searches for the same pattern elsewhere, so a fix in one place is not left undone in another.
• Store isolation. Every data read must be limited to the merchant's own store, at both the code level and the permission level.

What review does not replace

• People keep the final call. AI review lowers the chance of a miss without removing it. That is why high-risk changes need human sign-off and every change gets a human reviewer.
• Production data still needs a person. The reviewer cannot see live data, so a person compares dry-run counts against production before any delete is turned on.
• We hold our tools to the same bar. Our other AI-assisted engineering workflows stay paused until they carry the same guardrails as /do-review.

FAQ

Does every change really get three reviews?

Yes. The author reviews their own change, /do-review reviews it, and a second engineer approves it before it can merge.

Does AI review replace human review?

No. It does the slow tracing work, such as following every caller and every code path that writes a field. People make the final decision.

What makes a change high risk?

Anything that deletes or bulk-updates data, schedules a background job, or changes what such a job selects. The classification is automatic, so it does not depend on anyone's judgment about whether the code looks safe.

Can I apply this to my own app?

Yes. The data-deletion rules work with any stack. Put them in your review checklist, your CI, or your own AI review prompt.

Questions about Videoselz or how we protect your store? Our support team is happy to help.

Open your Videoselz dashboard · Contact support · Book a demo