Can we talk about how useless most agency case studies really are

I was digging through a local agency's site here in Portland last week and noticed they claimed '500% ROI increase' for a client. Sounded great until I dug into the actual data and realized they were measuring from a single bad month. The baseline was conveniently the worst month the client ever had. I checked five more agencies and four of them did the same trick with the timeline. Nobody talks about how easy it is to fake a case study when you control the starting point. Makes me wonder how many people sign contracts based on numbers that are basically nonsense. Has anyone else actually looked behind those shiny case study pages and found the same thing?

2 comments

2 Comments

schmidt.eva10d ago

I once wrote a case study that made a 2% bump look like a revolution.

dianagreen10d ago

The best trick is to ask them for the raw numbers from six months before and six months after their campaign... watch how fast they dance around that request. Most agencies rely on you just glancing at the headline number and moving on. That timeline cherry-picking is the oldest move in the book, always ask for the full date range.