The case against testing headlines

Recently I shared the results of an interesting study.

A group of researchers looked at hundreds of A/B tests for ecommerce stores, and basically validated what experienced copywriters have said for generations:

Don’t waste your time fiddling with cosmetics.

Test big things like your guarantee or adding customer testimonials.

In response, reader Karl points out that the study has a glaring omission:

They forgot to test headlines.

Now I can forgive them on this one.

These tests focused on ecommerce stores, which usually have product names vs. real headlines.

(By the way, testing product names is also a good idea.)

That raises a broader question though that I want to tackle today:

Is it even worth testing headlines in the first place?

I’ll go out on a limb here and say that the way many people test headlines, they’re wasting their time.

What?

Heresy!

Allow me to explain:

When I started working with John at Simple Programmer, his main lead magnet was an email course about how to start a programming blog.

Most of the signups for this lead magnet came from a popup on his site.

The blogging lead magnet converted at just under 1%.

I got excited when I saw that, because I knew a decent popup can do 2-3X better.

I figured I’d run a few A/B tests and score a quick win.

Nothing doing.

I must’ve run a dozen different tests with different headlines, and no matter what I tried, I couldn’t seem to move the needle on that turkey.

A headline has two basic purposes:

1. To “flag down” the people who are most likely to be interested in your offer.

2. To get them to read whatever comes next.

The problem with most headline tests is that you’re usually just expressing the same idea with slightly different words.

In my case, the idea boiled down to:

“Blogging will help you get a better software development job.”

If you’re just shuffling words around and the core idea stays the same, that means the group of people that you’re “flagging down” will stay pretty constant—and so will your results.

Instead of testing headlines, I aim to test different “appeals” or benefits.

For example, I could have pitched the same blogging lead magnet by saying:

“Blogging will help you learn new technologies and programming languages faster and more efficiently.”

Or:

“Blogging will help you earn more respect at work.”

In other words, test concepts, not phrasing.

Ask yourself:

What is a radically different way of looking at my offer that might reach a broader segment of my audience?

That’s where you’re likely to find those elusive 200-500% response bumps.

Related posts:

  1. Case Study: What Does ‘Homework’ In Email Courses Do For Sales? My short take on giving “homework” in email courses is: DON’T DO IT! Almost no one actually follows through on your assignments, and you’re just adding guilt and stress to your subscribers’ lives. Subscriber Ed calls me out though: Interesting. So you’re suggesting a “pure info” course with no homework?...
  2. Case Study: How I Increased Email Signups 372% with Thrive Leads and the 80/20 ‘Peel and Stick’ Technique Lately I’ve been on a tear to boost the number of email signups we get at Simple Programmer. This all started about a month ago. I was feeling pretty frustrated with the slow pace of our email list growth at Simple Programmer. Most of our signups come from this full-screen...
  3. Why “swiping” headlines is for chumps Confession: I HATE writing headlines. Here’s why… When you start studying copywriting, one of the first things you hear is the famous quote from adman David Ogilvy: “When you have written your headline, you have spent eighty cents out of your dollar.” In other words, the headline is responsible for...
  4. How a simple tool helps me write headlines with confidence In many cases, the success of a blog post or sales page rests on the strength of the headline. Says David Ogilvy, among the most successful copywriters of all time: On the average, five times as many people read the headline as read the body copy. When you have written...