Your software is only as good as the team that builds it. So we vet the team.
Agencies go through a four-stage gauntlet before they get to bid on your brief. The bar is the bar — we re-test every 90 days. Failure rate is high, and that’s the point.
The four-stage gauntlet
We don’t care about portfolios, awards, or how long an agency has been around. We care that they ship working AI-native software, fast, and that their workflows are actually built around modern tools. So we test for that. Four stages, all real work.
Build test
We send the agency a real brief — a problem from a past project, with anonymized data. They have 5 to 7 days to ship working code. Not slides, not a Figma file. A repo we can clone and run.
Output: a deployable prototypeCode review
A senior reviewer reads every commit. Architecture choices, security boundaries, prompt design, data flow, error handling, test coverage. They sign off — or the agency does not pass to stage 3.
Output: a written review · pass/failAI workflow verification
Agencies screen-share their daily workflow. Claude, Codex, Gemini, Cursor in active use — not just installed, used. If AI tooling is on the marketing page but not in the daily commits, the agency is out.
Output: a 30-minute screenshare reviewQuarterly re-review
Every 90 days we re-test. The bar moves with the tools — an agency that was AI-first in Q1 may not be by Q3. Stale workflows, missed updates, slipping shipped quality — back to stage 1, or out.
Output: continued network access · or removalTools we expect to see in daily use
In 2026, an agency that builds without modern AI tooling is shipping at 1× speed while the rest of the world ships at 4×. That gap shows up in your timeline, your bill, and the polish of the final product. So this is non-negotiable: agencies in our network use these tools every day, and we verify it.
What gets an agency rejected
- Portfolio without code. Polished case studies are easy to make. Working repos aren’t.
- Legacy stack with AI sprinkled on top. A WordPress shop that just discovered Claude isn’t AI-first — they’re a WordPress shop.
- No senior engineer who can lead architecture. AI tools amplify good engineers; they don’t replace them.
- No shipped products. Hackathon demos and side projects don’t count. We need to see software running in production with real users.
- Slow on the build test. If an agency takes 14 days for what should be a 7-day build, every project they touch will overrun.
- Defensive about code review. Senior engineers welcome scrutiny. Junior teams pretending to be senior get defensive. Tells.
What this means for your project
- Three agencies bid on every brief, all of whom have proven they can ship working code, not just promise it.
- The agency you pick has been senior-code-reviewed in the last 90 days — their architecture isn’t stale.
- You’re never matched with an agency that has a portfolio but no production track record.
- If an agency drops in quality, they’re removed from the network within 90 days. Your project team is current, not legacy.
- You don’t do the vetting work. You don’t need to know what “AI-first” actually means under the hood. We’ve already done it.