Strategy Experimentation: How to Choose a Strategy?

To determine the best strategy for our company or product, we generally start by analyzing “which” strategies are appropriate given the situation and objectives of our company.

Diagrams are helpful in this case. They can be used to analyze options based on differing dimensions. For example, the following diagram shows which of the five strategies you should choose based on your maturity level and whether you are looking for formulation or execution.

Our strategies, on the other hand, are significantly different and unique from those demonstrated. So, when we examine and analyze our strategies, some frameworks can turn out to be useful and provide us with different perspectives.

Take McKinsey’s Four Lenses, one of my favorites. It states there are four lenses through which companies should view strategy: Financial, Market, Operation Model, and Competitive Advantage.

Yet again, there is another lens—our biased opinions—that influences everything. Sometimes it works in our favor, but other times it does not. So, to find out which strategy will work best for a company or product, experimentation is our best friend.

The Problem With Experimenting Strategies

We test landing pages, emails, and websites, but testing strategies? It sounds a bit impossible, time-consuming, and challenging.

In most cases, we do not test strategies because:

• They are usually tied to long-term outcomes—metrics such as brand preference and user lifetime value.
• They have long feedback loops, which makes us less agile.
• Generally, strategies are implemented company-wide, with no specific boundaries.

Conversely, when we experiment, we aim to validate a hypothesis within a reasonable timeframe and boundaries.

The combination of all these factors leads us to view strategies as something that we cannot experiment with most of the time.

Making Strategies Experimentable

In order to experiment, we need elements to be measurable in size and duration and have specific values. If these two dimensions are very large or infinite, the strategies are no longer testable. So let’s define these two dimensions:

• Size. How large is the scope of this strategy? E.g. company-wide, a particular product, season, audience, or even a situation.
• Duration. How long or in which time slots will we use this strategy?

The idea here is to shrink strategies in both dimensions to the point where we can experiment with them.

In addition to these two dimensions, we’ll need a few new metrics:

• LTM (Long-Term Metric). This is probably the first metric you were looking to evaluate your strategy with—before trying to experiment.
• STM (Short-Term Metric). We want to identify the short-term driving metric, cause, or factor that will push our LTM.
• Causality percentage. Since STMs and LTMs vary in their relationships, we’ll use a Causality percentage here. Finding the STMs with the highest causality to our LTMs is our goal.

Options Evaluation, and Next Steps

I like Jeff Bezos’ view on decision-making, being agile and making quick decisions—pretty much always—unless they’re irreversible.

When all of that information is put together, we will be able to create a table like the one shown below. The purpose of all strategies here is to increase Life Time Value (LTV), so the LTMs are the same. Here, my STM is Weekly Retention, but it really depends on your business in order to tell you what your STM should be.

A few points to note:

Depending on our interest in experimenting further, we can use different STMs with their Causality Percentages. This, however, may increase the uncertainty—since we chose the highest causality STM, and the other STMs are lower in the metric.

Two reasons make causality useful here: first, different STMs can be discussed, and second, forecasts can be made for business outcomes (based on STM<>LTM relationship).