top of page
Sam Decker

4 Steps to a "Test-Before-Invest" Plan


Dwight D. Eisenhower said, “In preparing for battle I have always found plans are useless, but planning is indispensable.”


Have you planned your 2016 digital innovation bets? Here’s my hypothesis: I believe if you plan innovation with a data-driven approach, you will dramatically impact the return on your investments. Moreover, a data-driven planning process will change the way your team operates going forward.


Innovation planning can be derailed by focusing on the wrong problems and lower-impact solutions. Have you noticed in meetings that teams are prone to jumping to solutions early? Solutions are certainly plentiful in the dynamic digital space. Unfortunately, when it comes to long-term planning, the implications of focusing solely on a solution are greater because investments are high and opportunity costs are significant. Maybe there is pressure to do something a competitor is doing. Or there are new technologies that have been sold to you recently. Or perhaps everyone around the table gets to tee up their pet project.

There is a better way. A way to enter 2016 with a “clear head.” A way for the digital team to connect to the business as whole, focus on the biggest problems and the highest impact solutions. A way for everyone to learn, through planning, how to operate with data first.


1. Identify your capacity to innovate


All digital teams have table stakes and operational day-to-day duties. What’s left? Have you defined what percentage of their time or budget is allocated to innovation? I’m not talking about fixing known problems with known solutions. To innovate is to make an educated bet to solve a business problem with a disruptive solution.


Determine your capacity to change the business versus run the business. Which people are responsible for innovation? What’s the tech budget and outsourcing agency/consulting budget? What’s the appetite from the executive team? Make it explicit that you need to have capacity to innovate and test. You may have to make some tough calls on saying no to run the business activities needed for disruption.


2. Identify key business problems using data


Any attempt to add disruption should be pointed directly at a explicit business or customer problem. “Our site is not responsive,” for example, is not a business or customer problem. That’s a solution in search of a business problem. “We’re losing share with millennials” is a business and customer problem. “Our margins are deteriorating” is a business problem. “Our NPS scores are down” is definitely a problem.


Any problem you identify should map to corporate vision, objectives, goals and/or strategy. Digital innovation goes off the rails when it is done merely for the sake of innovating, and off-the-rails innovation is less likely to get corporate acceptance or integration. You want any digital program to be closely tied to company hot topics, customer targets, P&L, strategic initiatives and other goals. In short, tie digital innovation to the CEO or CFO agenda.


How do you determine the right problems? Data. Interview cross-functional stakeholders for a more complete view of your landscape. Looks at user feedback to find the biggest gaps to acquiring and delighting customers. Review competitive loss research to see where you are falling behind. Identify what segments of customers are not converting. Use call data and talk to customer service team. Find the leakage in the portfolio of conversion funnels. Study progress in the white spaces of where your company needs to grow.


Tie as much data as you can to the problems, preferably related to KPIs in the business that effect the P&L. Then assign low/medium/high attribution to the problem, as well as a degree of confidence that solving the problem will impact the P&L. Extrapolate that impact into a pro-forma revenue, margin dollar or cost saving number and you will have an idea of how solving that problem can make a big impact.


3. Identify solution hypotheses using data


The first misstep in planning is coming up with solutions focused on the wrong problems. The second misstep is when the winners aren’t chosen to solve the problem. Your goal is to steer the team towards the winners.


As stated earlier, we naturally gravitate around solutions at the start. You and everyone that works around you will have new ideas and hot topics. Mobile responsive. Digital infrastructure. Personalization. Social. Virtual Reality. User generated content. But these, and so many more exciting solutions in the digital space, can turn into investment black holes. Momentum takes over. Once launched, you end up wondering what problem you solved or whether you made any impact at all.


How do you find the winning solution hypothesis (a.k.a. ideas)? With data, of course. How do you do that?

  1. Identify the top business or customer problems, as discussed above.

  2. Get the group together to brainstorm hypotheses to solve the problems.

  3. Put the hypotheses in a grid and add columns for effort, impact and confidence in that impact. The discussion around confidence will unearth what data the person giving the hypothesis has for why their idea is likely to be a winner.

  4. For those with low confidence but high potential impact, identify opportunities to gather data that can increase confidence. This could be industry research, customer interviews, vendor case studies or analyzing your own data.

  5. Set a time to come back together and prioritize ideas with the biggest impact and highest confidence of impact against the largest problems.

4. Execute with a “test-before-invest” approach


Now that you have the hypothesis for solving problems mapped out, do you go all in? For the ones where you have enough data and confidence already, yes you do.

But sometimes there are several ideas that could be winners to solve one problem. In these cases, you want an agile, lean approach to test-before-invest your way to a clear winner. Your action plan is to run unbiased tests before making a big financial commitment. For example, a site redesign could be part of your innovation plan, and there is a data-driven way to approach redesigns as well.


How do you test-before-invest your way towards the winner? In simplest terms:

  1. Brainstorm hypothesis to solve problems and select the ones you need to test before investing.

  2. Identify the testing methodology. This could be A/B split tests, data analysis, user tests, surveys or other research.

  3. Create action plan with owners and deadlines to determine outcomes for these hypothesis.

  4. Find and execute the winning hypothesis with a plan for sustained optimization.

The other upside of this approach — as with this entire planning process — is there should be higher confidence in the result of the solution. And perhaps more importantly, learning occurs throughout the team, teaching a clear headed way to use data to make short-term and long-term decisions. The repetition of test-before-invest will make your team better every day. Imagine what planning will look like next year.


At that point, my hypothesis and Eisenhower should be correct: Planning may be even more useful than the plan itself.

Comments


bottom of page