Buyer guide

How to choose a CRO agency

By CRO Agency Index research deskPublished

Summary: Shortlist agencies whose published work matches your business type and vertical, check that they own research, build and QA end to end, ask how they decide what to test and how they use AI and your data, and compare on cost per shipped experiment rather than headline fees.

1. Start with fit, not fame

The best agency for a global retailer is rarely the best agency for a Series B SaaS company. Filter by:

  • Business type: SaaS, B2B, ecommerce, marketplace or retail. See SaaS, ecommerce and B2B rankings.
  • Vertical: health, finance and other regulated categories need extra care. See regulated industries.
  • Specialization: dedicated CRO, CRO-led, or multi-service. Our directory lets you filter on this.
  • Budget: only a few agencies publish prices, but asking for a range early saves time.

2. Check who actually does the work

Strong agencies own the whole loop: research, prioritisation, copy, design, development, QA, analysis and rollout. Ask for the roles on your account and how much of their time you get. A senior strategist who disappears after the pitch is a common complaint.

3. Ask how they decide what to test

Good answers mention customer research, analytics, a prioritisation framework and a clear link to commercial goals. Weak answers are a list of “best practice” tests used for every client.

4. Ask about statistics and stopping rules

You should hear about sample size, minimum detectable effect, test duration and what counts as a win, decided before launch. Free tools such as sample size calculators are a good sign the agency shows its working.

5. Ask how they use AI and your data

Most agencies now use AI somewhere. Ask where (pattern finding, variant drafting, analysis), who signs off decisions, and how your data is accessed and stored. Our AI-ready ranking favours agencies that publish this.

6. Look for evidence you can check

Case studies that describe the change, the metric and the outcome are worth more than logo walls. Anonymised case studies are common under NDA, but they should still describe the experiment in detail. Our profiles mark every fact with its verification status.

Questions for the first call

  1. What would the first 30 days look like for us?
  2. Do we have enough traffic to test, and where would you test first?
  3. Who is on our team, and how much of their time do we get?
  4. Do you build and QA tests in our stack, or hand over designs?
  5. How do you report results, and in what commercial terms?
  6. What happens to scope when tests are inconclusive?
  7. Where do you use AI, and what data does it see?

Red flags

  • Guaranteed results with no definitions
  • Win rates with no definition of a win
  • No measurement audit before testing
  • Recommendations without implementation support
  • Reporting only on conversion rate when revenue per visitor or pipeline is what matters

Topics: A/B testing, CRO agency pricing, SaaS and B2B CRO

Related rankings

Common questions

Should I choose a CRO-only agency or a full-service agency?

Choose a dedicated CRO agency if conversion is your main constraint and you want the deepest testing bench. Choose a multi-service agency if you also need traffic and want one team accountable for both.

How many agencies should I shortlist?

Three is usually enough. Use our rankings to narrow by business type, vertical and budget, then compare the finalists on our comparison pages.

More buyer guides