Read the case study as a claim, not a credential
An agency case study is a sales document. It can show relevant experience, but an attractive presentation does not establish who did the work, whether performance improved or what caused the change. Your job is to separate the story from the evidence.
Start with the assignment you need delivered. A visually impressive rebrand may demonstrate design craft without demonstrating customer research. A successful advertising campaign may tell you little about the agency’s ability to rebuild your measurement setup.
As you compare marketing and digital agencies, use case studies to generate verification questions rather than treating client logos as proof of fit.
The headline result has no traceable evidence
“Transformed performance” is positioning. A specific improvement is a factual claim that should have a supporting record. The warning sign is not brevity; it is an agency’s inability to explain where its headline came from.
The US Federal Trade Commission’s advertising substantiation policy says advertisers must have a reasonable basis for objective claims before disseminating them. That provides a useful buying principle: ask what evidence existed when the agency published the result, not merely what explanation it can offer now.
Ask for:
- The underlying reporting system or research source.
- The metric definition and reporting period.
- A redacted export, research summary or other inspectable record.
- Confirmation of whether the client reviewed the claim.
A screenshot can help, but check whether it identifies the account context, dates and filters. Cropped dashboard tiles are difficult to interpret. Client approval confirms permission or agreement; it does not independently validate the measurement method.
Record unknowns without filling them in
When a detail cannot be verified, write “not published” and identify the source you checked. For example, if the public case study omits its reporting period, record “Measurement period: not published in the agency’s public case study.”
Do not substitute an assumed timeframe or budget. Keep public omissions separate from information the agency later supplies privately.
The improvement has no baseline
A result needs a starting point. “More leads” leaves open whether the previous period was unusually weak, whether lead definitions changed or whether advertising spend increased.
Ask the agency to describe the before-and-after position using consistent definitions:
- **Starting condition:** What was happening before the work began?
- **Comparison period:** Were the periods reasonably comparable?
- **Inputs:** Did budget, staffing, distribution or promotional activity change?
- **Population:** Were the same markets, products and customer groups included?
- **Tracking:** Did consent settings, event definitions or reporting tools change?
An improvement after repairing broken tracking is not necessarily an improvement in underlying demand. Similarly, revenue growth during a major promotion does not isolate the effect of new creative.
For conversion optimization agencies, ask how conversions were defined and whether the comparison used an experiment or an observational before-and-after analysis. Both can inform a decision, but they support different levels of confidence.
The metric does not match the business objective
A weak case study may substitute an easy-to-display activity measure for the outcome the client wanted. More content, impressions or visits might indicate delivery or reach, but those measures do not by themselves establish commercial value.
Ask the agency to connect the reporting chain: what it delivered, how audiences responded and what business outcome followed. Where that connection stops, the claim should stop too.
In SEO, distinguish search visibility from qualified enquiries and sales. Our guide to choosing an SEO agency can help you turn that distinction into selection questions.
Not every engagement needs a revenue claim. A branding assignment may legitimately be evaluated through research quality, strategic clarity, implementation and usability. When evaluating a branding agency, ask for evidence appropriate to the assignment rather than encouraging a speculative sales attribution.
The agency claims credit for everything that happened
“Following our launch, sales grew” describes sequence. “Our launch caused the growth” makes a stronger claim.
The UK government’s Magenta Book evaluation guidance explains approaches to assessing impact, including counterfactual reasoning and theory-based evaluation. For buyers, the practical lesson is to ask how the agency considered what might have happened without its work.
That does not mean every case study requires a controlled experiment. It means the explanation should acknowledge competing causes and match its confidence to the evidence.
Ask for a contribution map
Have the agency identify who owned strategy, creative, development, media buying, analytics and approvals. Then ask which activities were handled by the client, another agency or a specialist contractor.
Watch for borrowed credit:
- A subcontracted deliverable presented as ownership of the whole engagement.
- A founder’s former-employer project presented without that context.
- A client’s overall growth attributed to a narrow campaign.
- Work by a previous team presented without explaining who remains available.
These situations are not automatically disqualifying. Contractors and prior experience can be relevant. The problem is ambiguity about whose capabilities you are buying now. Ask the agency to label its role and connect that role to your proposed delivery team.
Caveats disappear from the headline
Check whether the body quietly narrows the result advertised at the top. Was the outcome limited to a product, market, campaign or selected period? Were cancellations, returns or low-quality leads excluded?
The UK Advertising Standards Authority’s CAP Code section on misleading advertising addresses substantiation and significant qualifications. As a buyer, look for limitations alongside the claim, not only in a sales call after you challenge it.
For paid media, distinguish platform-attributed revenue from business-level revenue and ask how overlapping channel credit was handled. Put those definitions into your paid media agency brief so competing agencies respond on comparable terms.
Missing public detail is not proof of weak work
Confidentiality can prevent an agency from naming a client, exposing account data or publishing commercial terms. An anonymized case study is not inherently unreliable.
Ask what can be shared with permission: a redacted report, a walkthrough of the method, a sample deliverable or a client reference. Do not demand confidential client information or assume refusal proves dishonesty.
The useful distinction is between restricted evidence and unexplained claims. An agency should be able to describe its process, contribution and measurement limitations even when sensitive records remain private.
Turn the case study into a buying decision
Create a short evidence note for each shortlisted agency. Record the assignment, baseline, result, source, contribution and unresolved questions. Keep documented facts separate from sales-call assertions.
Then ask the proposed team to walk through the case, including what failed, what changed and what they would do differently. Specific explanations are more useful than a polished retelling.
A strong case study need not prove that every engagement will succeed. It should give you enough traceable evidence to judge relevant capability, understand uncertainty and decide what to verify next.