How to Run an Agency Review That Tests the Work, Not the Pitch

How to Run an Agency Review That Tests the Work, Not the Pitch

Posted 11/18/26
6 min read

Most agency reviews reward the best pitch performance, not the best working relationship. Here is a client-side process that briefs each phase, tests real work for pay, and scores agencies on weighted criteria, including how they actually use AI.

  • Why pitches reveal presentation skill more than working method
  • Brief each phase, pay for the test, and weigh the criteria
  • What the client owes the process, and what happens after the decision

Why Reviews Reward the Pitch, Not the Partnership

Agency reviews have a reputation problem on both sides. The ANA and 4A's agency search simplification report opens by noting that the search and pitch process has become fraught with concerns for agencies and clients alike. It covers a short list of recurring issues: the participation of key client decision-makers, chemistry meetings, agency ownership of work produced during the review, and timely, actionable client feedback.

The structural flaw is that a pitch is a performance with a few days of preparation, while the relationship is months of briefs, revisions and trade-offs. A review that only samples the performance selects for the performance. The fix is to design the process so that what you observe resembles what you will buy.

Decide What You Are Buying First

Before writing a brief, settle the shape of the decision. The ANA and 4A's maintain separate guidance for agency reviews for project work, noting that best practices for an agency-of-record search were well documented while project reviews had no equivalent. A retainer partner, a project specialist and an in-house build are different purchases with different tests.

The in-house option is real. In its 2026 benchmark, the ANA found jurors five times more likely to say marketers are in-housing more than ever than to say they are pulling back. Our guide to building a service catalog for an in-house team describes what an internal team has to define before it can compete with an agency, and our analysis of the agency reset covers why pricing models are shifting.

Brief Every Phase, With Escalating Detail

The ANA and 4A's agency selection briefing guidance holds that every phase of a review needs a thoughtful briefing, and that the information shared with agencies should escalate as the process moves from RFI, RFP and credentials to strategic or speculative work and final presentations. A workable sequence:

  • RFI and credentials. Business context, objectives and the shape of the work. Ask how they work, not only what they have won.
  • Shortlist and chemistry. Share the real team structure, ask who will run the account day to day, and ask to meet that person.
  • Test. A scoped piece of real work, briefed with the same completeness you expect to give the winner. Our guide to creating an effective creative brief lists what belongs in it.
  • Finals. Open access to decision-makers and a Q&A that tests how the agency handles disagreement.

Test the Work, and Pay for It

If speculative work is part of the review, 4A's guidance is to give finalists enough information, access to the client and time to do it properly. And the ANA and 4A's consider it best practice that an agency keeps ownership of the IP in work presented during a review unless the marketer pays for it at an agreed level and receives the rights.

A paid, narrow test beats a free, broad one. AirOps's guide to evaluating B2B agencies recommends running a paid pilot before a retainer. Scope the test to something you will actually need: a campaign adaptation, a brief-to-first-draft exercise, or a revision round with deliberately contradictory feedback. Fix the budget, the timeline and the rights up front, and give every finalist the same package.

Ask How They Use AI, and Look for Evidence

Every agency now claims AI-driven efficiency. Ask to see the workflow, not the slide: which tasks are automated, where a human reviews, how outputs are checked against brand rules, and what happens to your data. Request a sample of a real deliverable with the steps behind it. In the AirOps scorecard for B2B content agencies, an AI-search capability criterion carries a 20% weight, and the evidence requested is concrete: cited URLs for a sample of your prompts. That weight is specific to that guide. The principle of asking for evidence is general.

Link the AI questions to commercial terms. If an agency says AI makes production faster, ask how that shows up in fees and timelines, and what you will see in reporting.

Score With Weights You Set Before You Meet Anyone

Agree the criteria and weights before the first meeting, so that charisma cannot rewrite them afterwards. AirOps shows the format: a table of criteria with percentage weights and the evidence each requires. Build yours from five to seven criteria, for example strategic fit, quality on the test, process and AI practice, team and continuity, commercial model, and cultural fit. Score independently, then compare, and record why scores differ.

Include the people who will work with the agency. The ANA and 4A's list participation of key decision-makers among the elements of a good search. Add the day-to-day users to that list.

Owe the Process What You Ask of It

The simplification report names timely and actionable client feedback among the issues that need attention. Commit to a timeline, answer questions within a day, and give feedback that tells the agency what to change. Then close the loop with the agencies you did not choose, in writing, with reasons.

The decision is the start of the work. Plan the first thirty days before you sign: access, brand knowledge, campaign history and who decides what. Our 30-day agency transition playbook is written for the agency, and the same four categories tell a client what to prepare. The agency-side view of the same process is in our new business checklist. Once the engagement starts, one shared project record for briefs, feedback and approvals, the kind of infrastructure MTM for Brands and MTM for Agencies offer, keeps client and agency working from the same page.

What to Do Before the Next Review

Write the decision first: what you are buying, for how long, and what a good first year looks like. Draft the phased briefs, pick one paid test, set five to seven weighted criteria and share them with your internal panel. Then tell every agency the process, the timeline and the rights position on day one.

FAQ

Should we pay agencies for pitch work? For anything beyond credentials and a conversation, yes. The ANA and 4A's treat work presented in a review as the agency's property unless the marketer pays at an agreed level and receives the rights, so a paid, scoped test is cleaner for everyone.

How many agencies should be in a review? Keep the finalist round small enough that each agency can be briefed properly and given real access. Three finalists is a workable size, and larger fields spread the same attention thinner.

Do we need a search consultant? Not always. The ANA and 4A's cover the use of an agency search consultant among the topics of their simplification report, and consultants can add process and benchmarks. For a project review, a clear internal process is often enough.

How do we judge AI claims fairly? Ask for workflows and sample deliverables with the steps behind them, and tie any efficiency claim to fees and timelines. Weight the criterion, but do not let it outweigh strategic fit and creative quality.

What if we cannot decide between the last two? Run a second, smaller paid test with the same brief and a deliberately difficult feedback round. Seeing how each agency handles disagreement usually separates them.

Sources