As discussed in Why We Resist Testing, there are many barriers to the process, so how do we make the uncomfortable transition to embracing testing? Many of the core principles of a testing mindset feel like a departure from a data-driven, optimization-centric approach. If these don’t come naturally to you, you are not alone. Before we dive in further, if you need reminding of why a Testing Mindset is so critical, revisit The Case for the Testing Mindset. Otherwise, take a deep breath, and get ready for metamorphosis, because:
Testing is not optional
Because of the many barriers for testing, it’s unlikely to be prioritized without direct intent to do so. Want to grow and remain competitive in your industry? Accept testing as a key to growing your program and your business.
Failure is a part of learning
It’s easy to be scared away by either the potential for failure, or failed tests in the past. In reality, failure is part of the testing process, and while you need to make sure you can afford the risk of failure in designing your tests, you shouldn’t beat yourself (or your marketing team) up for a test that doesn’t yield the results you hoped for. While the ideal outcome is that you’ve found some secret trove of highly valuable customers, most tests are likely going to help inform what you should deprioritize, or how you should limit future investments before you place massive bets on complete unknowns.
Financial optimization and learning are in constant conflict
In many cases, they are complete opposites. Financial optimization requires the application of learnings to ongoing decision making, but you have to run those initial tests to learn what you need to feed your optimization. We’ve seen many tests “fail” because the expectation for a new network or campaign is that it would be immediately profitable. This expectation is not reasonable for the vast majority of tests, which are investments in knowledge and insight, rather than investments in direct financial return.
Optimization takes time
Once you’ve learned what you need from your test, you still can’t expect perfectly profitable campaigns. Proper optimization to a target return on investment (ROI) requires consistent feedback loops and adjustments. Except by extreme coincidence, your ROI will always have an asymptotic relationship to your target: if you’re under-efficient, there’s more optimization to do. If you’re overly-efficient, then you’re leaving business on the table. Getting the systems in place to keep that balance in check takes time, and short-term results may not accurately reflect the long-term potential.
One test won’t give you all the answers
You’re never going to get a 100% certain result, and even the most dramatic results are less valid with each day that passes since the test concluded. The digital marketing landscape changes constantly, due to ad tech evolution, competition, and the previously discussed quantum entanglement theory of digital marketing. The constantly shifting sands mean that your test results give you a model from which to base your operating assumptions, but they need to be refined and checked over time. The landscape shifts so quickly and drastically that entire channels and ad networks that were a total flop a year ago may be the hidden gem your program thrives on today. A great testing strategy involves keeping an eye out for changes in the tide and validating your learnings along the way.
The bottom line? Always be testing.
Not quite bought in? Fair enough. We haven't even scratched the surface of some of the biggest puzzles that need solving to launch a test: prioritization, budget, and how to measure the results. All that, and more, will be covered in our next installment of The Testing Mindset series!