Every Cold Email Agency Sounds the Same. 7 Questions That Tell Them Apart

If you've spoken to a few outbound agencies, you'll have heard the same pitch more than once. Targeted lists, personalised copy, strong deliverability, booked meetings. We say some of those things too. From the outside, it's hard to tell agencies apart, and if you've been burned before, every claim sounds like the last one.

So don't judge agencies on their claims. Judge them on how they work. This post gives you seven questions to ask every agency on your shortlist, including us, and what a good answer sounds like. At the end there's a simple scorecard, the red flags I'd walk away from, and answers to the questions founders ask me most about choosing.

Why so many agencies end up running one campaign

Most retainer agencies sell a set-up. They write one sequence for a broad list, launch it, and spend the retainer keeping it running. When the first campaign misses, there's often no plan B beyond "give it more time".

That model isn't dishonest, but it only works if the first guess is right. Most first guesses aren't. What you're looking for is an agency that runs a search for the right message, with a plan for what happens when an idea fails.

Question 1: What do you do before writing any copy?

A good answer involves research into your market: your customer calls, your CRM, the competitors your buyers compare you with, and the places your buyers talk about their problems, such as Reddit and LinkedIn.

A weak answer goes straight to "we'll build a list and write a sequence".

Question 2: How do you split my market?

Listen for segments based on a shared situation, such as companies that have recently changed a tool, entered a new market or hired for a role. Company size and industry alone usually make a poor segment, because companies of the same size can have very different problems.

Ask whether they use signals, meaning a sign that a company is in that situation right now.

Question 3: How many combinations will you test at once?

This is the most useful question on the list. A combination is a segment, a signal and a message. Testing one at a time means each miss costs you a month. We run 10 to 15 combinations at once.

The space to test is bigger than most people expect. Take 12 segments, 9 signals, 7 messages and 4 offers, and you have 3,024 possible plays. Those numbers are only an illustration, but they show why one sequence to one list is unlikely to find the best one.

For scale, one AI customer support client of ours ran more than 320 experiments with us. They went from pre-revenue to 435 demos booked.

Question 4: What happens when a combination fails?

Ask what they changed in week three of their last campaign that wasn't working. A good answer is specific: the segment was cut, the signal was swapped, the message was rewritten from the replies. A vague answer ("we optimise the copy") usually means they keep sending the same thing.

We work on a weekly cycle. Combinations go live on a Monday and get read on a Friday, and each one gets a decision: keep, change or cut.

Question 5: How do you measure success, and what counts as a winner?

Open rates are a poor measure. Ask for positive replies per 10,000 emails, broken down by combination, plus how many of those replies became meetings.

Then ask what they call a winner. Ours is a combination where we can send 10,000 emails to a segment and get predictable positive replies. Depending on the offer, the target might be 1 positive reply in every 200 leads, or 1 in every 1,500. An agency that can't define a winner will struggle to tell you when it's found one.

Question 6: What will I see each week?

Ask to see a real weekly report, with client details removed. It should show which combinations ran, what came back, and what was decided for each. If the reporting is a dashboard of sends and opens, you won't be able to tell whether the agency is learning anything.

Also ask how replies are handled. A positive reply is worth most when it's answered quickly. Who reads replies, how fast, and how are meetings booked?

Question 7: What do I keep if we stop?

This one tells you how confident an agency is in its work. Ask who owns the lead lists, the sequences, the sending domains, the reply history and the test results.

With us, everything runs inside your own accounts, and lead lists are yours once paid for. If we stop working together, you keep the lists, the sequences, the domains and every reply. Your main domain is never used for cold email.

How to use the questions: a simple scorecard

Asking the questions is half the job. Comparing the answers is the other half. This is how I'd do it.

1. Ask every agency the same seven, in the same order. It's the only way to compare like with like.
2. Write the answer down in their words. Paraphrasing makes weak answers sound better than they were.
3. Score each answer 0, 1 or 2. Zero for vague, one for a reasonable answer with no example, two for a specific answer backed by something they can show you.
4. Ask for the evidence. A redacted weekly report, a sample segment list, an example of a message rewritten from replies.
5. Check the contract against the call. Ownership, guarantee wording and term should match what you were told.
6. Talk to a client if you can. Ask what changed in their first month and whether it paid back.

An agency that scores well on questions 3, 4 and 5 is running a search. One that scores well only on the pitch is selling a set-up.

Red flags I'd walk away from

• They can't tell you how many combinations they'll test.
• They want to send from your main domain.
• The weekly report is only sends, opens and clicks.
• Nobody can say what counts as a winner.
• They own the lists, the domains or the reply history.
• The guarantee on the sales page isn't in the contract.

A worked example: two answers to question 4

This is what the difference sounds like in practice. Both answers are made up for illustration.

Agency A: "If something isn't performing, we A/B test subject lines and tweak the copy until the numbers improve. We're always optimising."

Agency B: "On our last campaign, the segment of companies that had just hired a head of finance was getting replies, but mostly 'not my area'. We realised the new hire wasn't the buyer for that problem in their first month. We kept the signal, switched the target to the CFO, and rewrote the opening around the hire. Positive replies in that combination went up the following week, so we kept it and cut two others."

Agency A has described an activity. Agency B has described a decision, the evidence for it and what happened next. You want the second kind of answer to all seven questions.

Questions to ask a reference client

If an agency puts you in touch with a client, use the time well:

• What did the agency change in the first month, and why?
• How quickly were positive replies answered?
• Did the weekly reports help you make decisions, or were they only numbers?
• Did it pay back, and how long did it take?
• What would you do differently if you started again?

Two questions about the deal itself

Once the process questions are answered, two more about the commercial side are worth asking.

What is guaranteed? Meeting guarantees often hide the catch in the definition. A guarantee tied to the work an agency controls tends to be more useful. I've covered this in why we don't guarantee meetings.

What happens if it doesn't work? Ask what you'd be left with after three months of no result, in money and in learning.

Proof worth asking for

Ask every agency for return on investment, not activity. Reply-rate screenshots are easy to produce. A client willing to say the work paid back is harder to find.

One cold outbound infrastructure client had hired at least two dozen agencies before us. Their founder said we were one of two that had generated a positive ROI. That tells you how wide the gap between agencies can be, and why the process questions above matter more than the pitch.

Common mistakes when choosing

Choosing on price per month. The fee only means something next to what it has to pay back and what you keep. I've set out the sum in is a $5K a month cold email agency worth it.

Choosing on the case study logo. A famous client tells you who they sold to, not how they'll run your market. Ask how the result was found.

Choosing the agency that agrees with everything. If an agency never pushes back on your segment or your offer, it isn't going to tell you when something isn't working either.

Skipping the fit question. A good agency will tell you when cold email isn't right for you, for example when the market is too small or the ACV too low.

FAQ

How many agencies should I speak to? Enough to compare answers to the same questions. Three is usually plenty to see the difference.

Should I ask for a trial? You can ask any agency what a low-risk first step looks like. Whatever the answer, judge it on the seven questions above, because a small first step run on a weak process teaches you very little.

Is a specialist in my niche better? Niche experience helps, but the process matters more. I've covered this in does your agency understand your market.

What should the first month look like? Research, segments, copy approval and the first combinations live once the new domains have had about 14 days of warm-up, then weekly reports.

Use these questions on us too

If you're comparing agencies, bring this list to our call and ask every question. We'll show you the process step by step, the weekly report format and how we decide what to change. Book at thegtmcompany.com.