TL;DR. Small and medium sized innovation leaders evaluating a validation-first build partner should score candidates across five dimensions.
Validation methodology (productized or improvised).
Kill-rate transparency (do they publish how often they say no).
Team integration (one room or handoffs).
Pricing model (does the agency carry risk).
Exit clauses (can you stop after validation).
This framework can give a 25-point scorecard, 15 RFP-grade questions, and a decision matrix for matching the agency profile to your bet type. The right partner is the one who will tell you when and what not to build.
This is the framework we wish someone had handed us before we ever signed an agency contract.
It is for innovation directors, heads of new ventures, CTOs at growth-stage companies, and heads of digital at established incumbents.
The decision in front of you is which external partner to spend €100,000 to €1,000,000 with on a bet that may or may not work.
The cost of picking the wrong partner is not just the engagement fee. It is the bet underneath, the team time around it, and the political cost of explaining to the board why the launch missed.
The good news is that there is a small number of things that actually predict the outcome. Most evaluation processes do not measure them, because they are uncomfortable for both sides. I will share my framework and why it measures them.
When this framework applies
Use this framework when you are evaluating an external partner for:
- → A new digital product launch from inside an established company.
- → A spin-off or internal venture that needs to validate before scaling.
- → A first-of-kind regulated product (MedTech, FinTech, energy) where the regulatory and the bet risk compound.
- → A platform consolidation or modernization where the strategic question is bigger than the engineering question.
Do not use this framework when you are buying staff augmentation, when you already know what to build and just need execution, or when you are doing pure marketing or brand work. Different buy, different signals.
The five dimensions
The framework scores candidate agencies across five dimensions. Each is scored 1 to 5. The total is out of 25. The framework is not about hitting 25. It is about diagnosing which dimensions an agency is strong on, and whether those match the dimensions your bet needs.
1. Validation methodology
The question is whether the agency has a named, productized, repeatable validation engagement. Or whether discovery is improvised per project.
A productized methodology has a name. It has a published structure. It has a fixed price band. It produces a named artifact (an Evidence Pack, a discovery report, a build-or-kill verdict). It can be sold to multiple clients with the same shape, because the agency has run it enough times to standardize it!
An improvised methodology is custom every time. The agency will tell you “we tailor the discovery to your specific needs.” This sounds nice. It usually means the agency has not run discovery enough times to know what the shape should be.
- → 1 point. Discovery is improvised, no named methodology.
- → 3 points. Discovery has a documented structure but each engagement runs differently.
- → 5 points. Discovery is productized, named, with a fixed price band and a repeatable artifact.
Ask: “What is the name of your discovery engagement, and can you send me the structure and the price band?” If the answer is “we customize it for each client,” score 1. If the answer is a URL with a name, a structure, and a price, score 5.
2. Kill-rate transparency
The question is whether the agency tracks and discloses how often they recommend not building.
This is the most diagnostic single question in the framework. An agency that tracks kill rate has internalized that the killing is the work. An agency that does not track it is either an agency that never kills or an agency that does not want to admit how often it does. Both are bad signs.
The number matters less than the existence of a number. An agency with a 10% kill rate that knows the number is better than an agency with no number at all.
- → 1 point. Will not share, or claims they have never recommended against a build.
- → 3 points. Will share if asked, with a number that varies by year.
- → 5 points. Publishes kill rate publicly. Can name recent cases where they killed a bet.
Ask: “What is your kill rate, and can you name a recent engagement where you recommended against the build?” The follow-up matters. If they cannot name a recent case, the number is invented.
3. Team integration
The question is who will actually be on your engagement, and whether they sit in one place or hand off to other places.
The two failure patterns are well known.
- Pattern one: the agency sells you the senior partners in the pitch, and the work is delivered by junior consultants who join the call once a month.
- Pattern two: the agency presents an integrated team, but the design happens in one office, the development happens in another, and the integration happens in a Jira board that nobody owns.
The good pattern is a single co-located or virtually co-located team, including the senior who pitched, working end-to-end through validation and into build. With the same senior owning the validation decision.
- → 1 point. Account manager plus offshore delivery team. Pitching senior not on the project.
- → 3 points. Mixed onshore and offshore single team. Pitching senior present occasionally.
- → 5 points. Single co-located team end-to-end. Pitching senior is the named lead on your engagement.
Ask: “Who specifically will be on my engagement, and will the people in this room today be on the project?” Get names. Verify them on LinkedIn after the call. Check their tenure at the agency.
4. Pricing model
The question is who carries which risk.
Three models matter.
Time and materials means you carry all the risk. Every hour, every overrun, every revision. The agency is incentivized to extend the engagement. This is fine for staff augmentation. It is poor for a bet.
Fixed scope means the agency carries delivery risk. They have to ship the scope inside the budget. The incentive is to fight scope creep. The problem is that the scope has to be right to begin with. If validation has not happened yet, the scope is wrong.
Shared outcome means the agency carries some of the bet risk. A reduced fee plus a milestone payment tied to a validated outcome. Or equity. Or revenue share. These are uncomfortable for the agency. The ones who offer them are the ones who believe in the methodology enough to bet on it.
A mature partner offers different models for different phases. Fixed-scope productized validation up front, time-and-materials or shared-outcome build behind it.
- → 1 point. Time and materials only.
- → 3 points. Fixed scope available for some phases.
- → 5 points. Shared outcome or success-fee structures available for the right engagements.
Ask: “Have you ever taken on a project on a shared-outcome basis? Can you describe one?” Real examples score 5. Theoretical answers score 1.
5. Exit clauses
The question is whether the contract has a contractual stop point.
The strongest validation-first partners build a stop point into the contract. Validation engagement runs four to twelve weeks. At the end, the agency presents a verdict. You decide whether to continue into build. If you decide not to continue, the engagement ends cleanly. No build commitment. No make-up fees.
Most agency contracts do not work this way. The validation phase is sold as a discovery sprint that flows into a full-build engagement. The agency is incentivized to make the discovery surface a build-shaped problem.
An exit clause is the single best defense against discovery theater. If the agency knows you can walk after validation, they are less likely to invent a build for you. They will validate the bet honestly because the alternative is to surface a real verdict. The shape of a real validation experiment, including the 4-8 week milestone, is well-documented and should match what the agency proposes.
- → 1 point. No exit clauses. Validation flows into full build with no contractual stop.
- → 3 points. Phase gates exist but exiting is difficult or expensive.
- → 5 points. Contractual stop point after validation. No penalty for walking.
Ask: “Can we contractually stop after the validation phase if the verdict is to not build?” If the answer is “yes, here is how it works,” score 5. If the answer is “well, we would have to discuss,” score 1.
The 25-point scorecard
Score each candidate agency 1, 3, or 5 on the five dimensions. Total out of 25.
| Dimension | 1 point | 3 points | 5 points |
|---|---|---|---|
| Validation methodology | Improvised per project | Documented but custom to project | Productized, named, repeatable |
| Kill-rate transparency | Will not share | Will share if asked | Publishes publicly |
| Team integration | Account manager plus offshore | Mixed onshore and offshore single team | Single co-located team end-to-end |
| Pricing model | Time and materials only | Fixed scope offered | Shared outcome available |
| Exit clauses | None, full-build commit | Phase gates, hard to exit | Contractual stop after validation |
A score above 18 is a strong validation-first partner. A score of 11 to 17 is a strong build partner who can also do validation if you ask. A score below 11 is a build partner who will sell you discovery theater.
None of these are wrong agencies. They are wrong agencies for different bets. Match the agency to the bet.
Matching agency profile to bet type
Not every bet needs a 25-out-of-25 agency. Some bets need depth in a specific dimension. The three patterns that I see:
New category bet
You are launching a product in a category your company has not played in. The validation question is the biggest question. Demand and willingness-to-pay are unknown. Engineering complexity is solvable but secondary.
You need a high score on validation methodology (5), kill-rate transparency (5), and exit clauses (5). Build capability and pricing model matter less. The bet may not survive validation, and you want a partner who will tell you that.
Adjacent expansion bet
You are extending an existing product into an adjacent segment or geography. The validation question is real but partial. You know the product works. You do not know if it works here.
You need a high score on team integration (5), with sector or geographic depth in the team. Validation methodology matters (3+) but does not need to be the deepest. Pricing model and exit clauses can be more flexible because the bet is smaller.
Platform consolidation bet
You are modernizing a legacy platform or merging multiple acquired platforms. The validation question is small. The engineering and migration questions are huge.
You need a high score on team integration (5) and build capability.
Validation methodology and kill-rate transparency matter less because the question is not “should we build this” but “how do we land this without breaking the company.” Match the agency profile to the engineering and change-management complexity.
Why do we talk about validation methodology? Basically we have 4 layers of ROI going from killing features, sharper delivery, unlocking new incremental revenue and landing at new business models for creating a different tier of business.
Where to start your shortlist
The framework above runs on candidate agencies. To get candidates, start with our 2026 ranking of European product validation agencies. Pull a starting list of five to eight names. Run them through the scorecard. Take the survivors to a first call.
Score Camplight against this framework.
We score 5 on validation methodology (The First Bet), 5 on kill-rate transparency (around 25% kill rate), 5 on team integration (single embedded team, no offshore handoffs), 4 on pricing (value-priced tiers, occasional shared outcome), and 5 on exit clauses (contractual stop after validation).
Book a call and pressure-test us against your bet.
Frequently asked questions
How do I choose a product validation agency for an enterprise project?
Score candidates across five dimensions: validation methodology (productized vs improvised), kill-rate transparency, team integration, pricing model, and exit clauses. Each is scored 1, 3, or 5. The total is out of 25. A score above 18 is a strong validation-first partner. Below 11 is a build partner who will sell you discovery theater. Match the agency profile to your bet type. New category bets need deep validation rigor. Platform consolidation bets need deep engineering. Adjacent expansion bets need sector depth.
What is a productized validation engagement?
A productized validation engagement has a name, a published structure, a fixed price band, and produces a repeatable artifact (an Evidence Pack, a build-or-kill verdict). The agency has run it enough times to standardize it. Examples include Camplight’s First Bet, Netguru’s Product Validation Sprint, and Boldare’s Product Discovery Workshop. The opposite is improvised discovery, which the agency customizes per project and signals that they have not run it enough times to know what the shape should be.
What is a typical kill rate for a good product validation partner?
Good validation partners report kill rates between 15% and 35%. Anything below 10% suggests the agency rarely says no, which means they have not built a real validation methodology. Anything above 50% suggests the agency is killing bets that should have been killed before the engagement started, which means their qualification process is weak. The number matters less than the existence of a number. An agency that tracks kill rate at all has internalized that killing is the work.
Should an enterprise agency offer a fixed-scope or time-and-materials contract?
Well, I think it depends on the phase. Fixed-scope productized validation engagements are stronger than time-and-materials for the discovery phase, because they force the agency to ship a verdict inside a budget. Time-and-materials is acceptable for the build phase if the scope was rigorously validated. Shared-outcome structures (success fees, milestone payments tied to validated outcomes) are rare but signal the strongest partners. A mature agency offers different pricing models for different phases.
How long should a validation engagement last before committing to build?
My suggestion: four to twelve weeks. Of course everyone has their own vibes. Four weeks is suitable for narrow, single-question bets. Eight to twelve weeks is typical for enterprise bets where the Evidence Square has four corners to light up. Engagements longer than twelve weeks are usually validation theater dressed up as rigor. The right partner ends the engagement when the bet either survives or dies, not when the calendar runs out. A contractual stop point at the end of validation is the single best defense against discovery theater.
Camplight is a worker-owned software cooperative founded in 2012. We help B2B teams validate, build, and scale digital products and ventures from zero to $10M+ ARR. Our validation-first approach has driven a 95% client satisfaction rate across 300+ delivered projects in FinTech, HealthTech, EdTech, and beyond.
