t - e - s - t is a four letter word.™

By Grant A. Johnson, Johnson Direct

"Testing your direct mail campaign is smart. The pay off is two-fold:
1. a big picture look at what will happen when you combine your
list, creative, copy, format and other components.
And 2. the chance to make changes if the results aren't exactly
what you were expecting."

It still amazes me, after nearly 14 years as a direct marketing practitioner, how few companies understand and actually do enough direct marketing testing.

The only logical explanation is that test is a four letter word. And, people don't like to use four letter words, right?

From a direct marketing standpoint, we can test the list(s), the offer, the package creative, components of the package, the format, and so on. The ability to test and measure results is what differentiates direct marketing from all other communications efforts. Testing, when done correctly, will help a company decrease the number of pieces mailed and intentionally increase its return-on-investment (ROI).

Speaking of The Bottom Line
Testing allows us to determine, in a real world setting, what works, what doesn't work and why. Like direct marketing itself, testing is about numbers, ROI and data.

We test not only to increase our ROI, but also to learn. The more we learn, the better we can market to segments that emerge as we move our marketing programs forward. As the late direct marketing guru Dick Benson said, "You've got to determine if the information you can reasonably expect to gain is really worth the expenditure."

Myths & Misunderstandings
True or False: testing is expensive. It depends on how you look at it. Is it an investment in the future success of your marketing efforts?

In his book, Selling the Invisible, Harry Beckwith astutely points out that focus groups determined that fat-free pizza at Pizza Hut, skinless chicken at KFC and the McLean sandwich at McDonald's would all be hugely successful. These companies undoubtedly spent millions of dollars on proposals, planning and marketing the launch of these products. Today, none of these items are on the menu. In-store testing, if done at all, should have been more extensive. I argue that an investment in testing would have saved these companies money - and lots of it.

Keeping It So Simple
The concept of a grid, or matrix, is based on the direct marketers ability to test and track several lists, creative approaches and offers at once.

The results? A "big picture" that involves nearly every possible scenario that could make or break the campaign.

Our advice is to set-up a testing matrix and test within budget parameters. The goal is to break-even or make money on your testing while simultaneously learning as much as possible. A word of caution though--make sure that the test cells implemented are meaningful. If they're not, you will waste money. For instance, testing outer envelope teaser copy seldom produces significant lifts in response rates.

A Matrix Example
A non-profit client had been mailing a fund raising appeal to the same house file of 220,000 names for almost 15 years. Of the 220,000 names, 80,000 continually donated to the cause. The remaining 140,000 people were unresponsive, however, year after year the same package was sent to the entire database.

Complicating the situation was the absence of a donor profile and the lack of budget to rent overlay information.

The matrix solution involved four test cells: two different letters, the addition of a buckslip and an alternative piece to the brochure. Testing generated over 4,500 new donors with an average gift of $35.00, not including the clients' lifetime value. As a result, the testing investment more than paid for itself.

But the test program didn't stop there. We took the information learned to enhance the donor file with overlays. The client made a commitment to future tests and to eventually rent outside lists.

Back To The Bottom Line
Testing does not have to be complicated. On the contrary, the practice of testing involves simple techniques to collect data. Data in turn becomes knowledge, which you need to succeed, move forward and grow.

Testing is a progressive art and can help make marginal programs more successful and successful programs more profitable. For more information on testing look to the published works of industry professionals such as Dick Benson, Ed McLean and Bob Stone.

Remember, test IS a four letter word. But then again, so are these words: mail less, make more cash.

Back to Marketing Know-how