The most reliable approach to vetting an agency combines a short, theme-based question set with a consistent internal scorecard and a small paid test project before full commitment. Three priorities matter more than the rest: fit to your specific objectives, a clear measurement approach, and fee transparency. Get those three right and most other risks become manageable.
TL;DR:
- Agencies should be evaluated based on clear KPIs, transparent fee structures, and demonstrated team continuity before any commitments are made.
- Asking for comparable case studies, a tailored strategy, and detailed measurement approaches helps identify genuine capability over polished pitches.
- Conducting a paid test project, scoped to real deliverables, offers a more reliable assessment than free, open-ended pitches.
- Watch for vague deliverables, undisclosed third-party fees, and teams that differ from those who pitch to avoid potential relationship pitfalls.
- Using a structured, scorecard-based interview process and verifying claims through source-backed profiles reduces research time and decision risk.
Table of Contents
- Agree internal priorities and evaluation criteria before you ask questions
- The question bank: grouped and copyable questions for each theme
- What to ask about fees, transparency, and contract terms
- How to validate claimed results and what to probe in reference checks
- Warning signs and quick contractual or process fixes to reduce risk
- A 60-90 minute interview agenda with scoring guidance and next steps
- How Agency Review Guide’s 100-point methodology maps to interview scorecards
- Author perspective: choosing between closely matched finalists
- How Agency Review Guide helps you shortlist faster and with less risk
- FAQ
- Sources
Agree internal priorities and evaluation criteria before you ask questions
Before any agency sits across from you, your team needs to agree on what a good answer actually looks like. Without that alignment, interviewers score the same pitch differently, and the selection process turns into a popularity contest rather than an evaluation.
Start by clarifying your business goals and the KPIs that matter most, whether that’s brand awareness, lead volume, or revenue growth. An agency that cannot connect its proposed tactics to your specific KPI is optimizing for its own convenience, not your outcome. Google’s guidance on agency partner questions recommends opening conversations with measurement precisely because it separates strategic partners from execution-only vendors early.
Next, decide the scope and depth of partnership you actually want. A single campaign and an ongoing retainer require different skill sets, different staffing, and different contract protections, and conflating the two during interviews produces vague, hedged answers from agencies trying to cover both bases.
Set budget bands and decide which remuneration models you’re willing to consider (hourly, retainer, performance-based, or a hybrid) before you ask the agency to name a number. Finally, write down your non-negotiables: data ownership, access to raw platform accounts, intellectual property terms, and any service-level requirements. Reviewing how agency specialties map to different scopes helps confirm whether the agencies on your list actually match what you need before you spend time interviewing them.
- Define your top one to three KPIs and rank them by priority.
- Decide project-based versus retainer-based partnership before outreach.
- Set a budget range and acceptable remuneration models in advance.
- List non-negotiables: data access, IP ownership, SLA terms.
Pro Tip: Write your evaluation criteria down and share them with every interviewer on your team so scores are comparable across candidates.
The question bank: grouped and copyable questions for each theme
Once your criteria are set, the questions themselves do the work of surfacing real capability versus polished pitch material. Organizing them by theme keeps the conversation structured and makes answers easier to score against your rubric later.
Experience and fit
- Can you show us case studies from clients with a comparable objective, size, or industry, including what changed before and after?
- What is your team’s client retention rate, and how long has your average account stayed with the agency?
- Will the same team that pitches us also work on our account, and how often does staff turnover on active accounts?
These questions matter because a polished pitch deck often features work from a senior team that never touches day-to-day execution. Asking directly about team continuity exposes that gap before you sign anything.
Strategy and approach
- Ask for a short strategic outline tailored to your stated objective, not a generic capabilities deck.
- Ask how the agency avoids applying the same playbook to every client regardless of industry.
- Ask what they would do differently if your budget were cut by a third.
A capable strategist can sketch a rough point of view on your specific problem within the pitch process itself. An agency that stalls here, offering only broad frameworks and no specifics, is usually signaling that strategy is not where it invests its best talent.
Measurement and data
- What KPIs would you track for our specific goal, and how do they map to revenue or pipeline?
- How do you attribute results across channels, and what happens when platforms disagree?
- How often will we receive reports, and what format will they take?
- Do you test for incrementality, or do you rely solely on platform-reported conversions?
Google’s own guidance to advertisers frames measurement as the question that should come first in any agency conversation, since it reveals whether a prospective partner actually has the managed IT services data infrastructure to back its promises. The AMA’s research on agency metrics notes that agencies themselves track internal figures like utilization, which averages near 73%, and project margin, which runs close to 36%, so a mature agency should be comfortable discussing its own operational metrics as readily as yours.
Creative and production
- How many creative concepts do you typically present, and how many revision rounds are included before extra fees apply?
- Who owns the final assets: copy, design files, video, and code, once the engagement ends?
- What is your typical turnaround time from brief to first draft?
Process, resourcing and timelines
Ask who specifically will work on your account day to day, including their availability and whether they’re shared across multiple clients. Ask what onboarding looks like in the first 30 days and what the escalation path is when something goes wrong outside business hours. These answers tell you more about day-to-day reality than any case study, because they describe the operating model you’ll actually live inside.
What to ask about fees, transparency, and contract terms
Fee structures are where ambiguity does the most damage, often long after the contract is signed. Ask for a full breakdown of labor costs, media spend, and any third-party or platform fees, and ask directly whether the agency receives rebates, discounts, or commissions from media vendors that aren’t passed through to you. The WFA and MediaSense research on agency remuneration found that many advertisers report limited visibility into how agencies actually make their money, which is exactly the gap a direct question about rebates is designed to close.
Ask which remuneration model the agency prefers for work like yours and why. A retainer-only agency may resist performance-based terms simply because they shift risk onto the agency, not because the model doesn’t fit your goals.
- Request an itemized fee breakdown covering labor, media, and pass-through costs.
- Ask directly whether any vendor rebates or undisclosed revenue apply.
- Clarify how scope changes are priced and whether change orders require written approval.
- Confirm termination notice periods, IP ownership terms, audit rights, and SLA remedies.
Scope creep is one of the most common sources of billing disputes, so ask exactly how additional requests outside the original scope get priced and approved before work starts, not after an invoice arrives. Finally, confirm termination rights and notice periods in writing, along with who owns intellectual property created during the engagement and whether you retain audit rights over billing and media spend.
Pro Tip: Ask for the termination clause and IP ownership language in writing before you sign, not as a verbal assurance during the pitch.
How to validate claimed results and what to probe in reference checks
Case studies look persuasive in a deck, but the details behind them matter more than the headline number. Ask for the baseline the agency measured against, the attribution method used, and whether the reported lift accounts for seasonality or other campaigns running at the same time. A result with no stated baseline is not a verifiable result.
Request references from clients in a similar industry or with a similar campaign type, and prepare a short, consistent questionnaire so every reference answers the same questions. The ANA’s survey on formal agency evaluations found that 82% of marketers already run formal performance evaluations internally, and that best practice assigns a neutral point person to run them using a consistent format, the same discipline worth applying to your own reference calls.
- Ask the reference about staffing changes and whether the promised team actually delivered the work.
- Ask how the agency communicated when something went wrong or missed a deadline.
- Ask whether results matched what was promised during the agency’s own pitch.
Honest references will mention friction points as well as wins. An agency with no references willing to discuss a rough patch is either new to the kind of work you need or selectively curating who you’re allowed to call.
Warning signs and quick contractual or process fixes to reduce risk
Certain patterns show up repeatedly in agency relationships that later sour, and most of them are visible before a contract is ever signed. Watch for an agency that can’t articulate a measurement plan beyond vague references to “growth,” deliverables that stay undefined even after you ask for specifics, third-party fees that remain unclear despite direct questions, and a pitch team that’s noticeably different from the account team you’re introduced to post-signing.
- No clear measurement plan tied to your stated KPI.
- Deliverables described only in vague terms like “ongoing support.”
- Reluctance to disclose third-party fees or media rebates.
- High staff turnover on similar accounts or a pitch team that disappears post-sale.
When you spot one of these, you have leverage, not just a reason to walk away. Ask for a smaller trial project before committing to a full retainer, limit the initial scope to a single campaign or deliverable, negotiate a short termination notice period, and request audit rights over billing. Tying payment to milestones rather than a flat monthly fee also reduces your exposure if the relationship doesn’t work out.
On the pitch process itself, the ANA and 4A’s joint research on the cost of the pitch found that agency review processes carry real costs for clients too, with participation running into the hundreds of thousands of dollars for incumbents in some cases. Rather than running an open, unpaid pitch across five or six agencies, request a short paid test or a realistic, narrowly scoped brief from your top two or three finalists instead.
Pro Tip: A paid test scoped to mirror the first 30 to 60 days of real work reveals more about fit than any open-ended pitch presentation.
A 60-90 minute interview agenda with scoring guidance and next steps
A structured agenda keeps the conversation focused and makes scoring consistent across every agency you interview. Before the meeting, request case studies, a sample media plan or strategic outline, and a reference list, and make sure your scorecard headers are agreed internally so every interviewer is rating the same categories.
- Introductions and context (10-15 minutes): confirm objectives, budget range, and timeline expectations.
- Strategy discussion (20-30 minutes): walk through their proposed approach to your specific objective.
- Measurement and reporting (15-20 minutes): cover KPIs, attribution, and reporting cadence.
- Commercial terms and next steps (10-15 minutes): fees, contract terms, and what happens after this call.
Score each criterion on a simple 1 to 5 scale immediately after the meeting, while details are fresh, rather than days later when impressions have blurred.
| Score range | Decision |
|---|---|
| — | Shortlist for final round or contract |
| — | Request a paid test project before deciding |
| — | Pass |
This rubric works across agency types, from creative shops to performance marketing specialists, because the categories (experience, strategy, measurement, process, commercial terms) apply regardless of discipline.
How Agency Review Guide’s 100-point methodology maps to interview scorecards
We built our 100-point evaluation methodology around the same categories that matter in a live interview, because the questions you ask an agency in a pitch meeting should map directly to the criteria used to verify its profile. Our pillars cover expertise, transparency, measurable outcomes, and client fit, and each one translates into a scorecard line you can reuse.
- Expertise pillar: do they show directly comparable case studies with named clients and specific results?
- Transparency pillar: do they disclose fee structure, media rebates, and any conflicts of interest unprompted?
- Outcomes pillar: can they explain baseline, attribution, and incrementality rather than just a headline figure?
- Fit pillar: does their proposed team, process, and communication style match how your organization actually works?
Every profile on our platform carries a verification status, so claims that can’t be substantiated are flagged rather than presented as fact. That flag alone often shortens shortlisting time, since it tells you in advance which agencies are willing to back their claims with evidence and which aren’t.
Author perspective: choosing between closely matched finalists
When two finalists score nearly the same on your scorecard, resist the urge to let chemistry decide first. Run a small paid test project, scoped to a real deliverable, before committing to a longer contract. Let the objective score from that test carry the most weight, and treat chemistry as the tiebreaker only after the numbers are in.
Once you’ve signed, schedule mid-term reviews rather than waiting for a renewal date to surface problems. Build two-way feedback into the relationship from the start: ask the agency what they need from you, not just what they’ll deliver to you. Agencies that resist candid, structured feedback early are rarely the ones that improve under pressure later.
— Quincy
How Agency Review Guide helps you shortlist faster and with less risk
Running the interview process above takes real effort, and we built a platform to help reduce the research time before you even reach the pitch stage. Every profile in our agency directory includes source-backed claims and a verification status, so you walk into your first meeting already knowing which agencies have evidence behind their case studies and which don’t.
- Search and filter by specialty, budget, and market fit before you schedule a single call.
- Review profiles scored against a published methodology.
- Use our local agency finder when you need options filtered by city or service.
Start building your shortlist on our agency directory, or submit a review if you’ve worked with an agency and want to help the next buyer ask sharper questions.
FAQ
What are the most important questions to ask a marketing agency?
Prioritize questions about measurement approach, fee transparency, and team continuity, since those three areas surface the most common sources of disappointment later. Ask for comparable case studies, a clear KPI framework, and a full fee breakdown including any vendor rebates, as recommended in Google’s agency partner guidance.
How do I evaluate an agency during a pitch meeting?
Use a consistent internal scorecard across every candidate so scores are comparable, and assign a neutral point person to run the process. The ANA’s research on agency evaluations found that 82% of marketers already use formal evaluation processes, often with a dedicated point person managing the scorecard.
What are red flags when interviewing a potential agency partner?
Vague deliverables, no clear measurement plan, undisclosed third-party fees, and a pitch team that differs from the actual account team are the most common warning signs. Any hesitation to put termination terms or IP ownership in writing is also worth treating as a reason to pause.
Should I ask agencies to complete a free pitch or a paid test?
A narrowly scoped paid test that mirrors real early-stage work reveals more about fit and delivery quality than an open, unpaid pitch, and it costs you less in review time overall. Research on agency partner selection recommends this approach specifically to validate capability before a longer commitment.
How many agencies should I interview before deciding?
There’s no fixed number, but narrowing to two or three finalists for a paid test tends to balance thorough comparison against the real cost of running extended reviews. The ANA and 4A’s cost of the pitch research found that full agency review processes carry substantial costs for clients, which is a strong argument for a tight shortlist rather than a wide one.
Sources
- Majority of Marketers Conduct Formal Agency Performance Evaluations According to New ANA Survey | ANA
- 6 Key Metrics for Marketing Agencies | AMA
- Questions to ask your agency partner - Think with Google