What Really Counts?
(Editor’s note: This is a two-part post. Part 1, below, introduces the One Metric That Matters’ approach to metric measurement. For more on this and taking a lean approach to analytics, we recommend you read Lean Analytics, by Alistair Croll and Benjamin Yoskovitz. Part 2, to come, will take this approach and overlay it with enterprise transformation, particularly that of the Agile HR variety. Enjoy!)
We live in a data-rich world, filled with trackable, measurable interactions that can help us iterate on products and processes in nearly real-time. We have a wealth of information at our fingertips about user behavior, and we have a number of ways to validate just how successful we’ve been in a given initiative. This is, of course, good news: Generations before us couldn’t imagine the user-level data we have today, nor the tools we have available to churn it all into meaningful insight.
So we have a lot of data. But do we know what matters in this mess of numbers and metrics? What really matters when we talk about whether we are increasing user engagement with a given product or service? Are click-through rates or downloads really indicative of how likely your service is to appear next year? Let’s assume you have a mature product: how do you know what customers want in the next iteration? What measurements will you use to help take your solution to the next level (whatever it is – it’s all relative.)
Data is dangerous when we abuse it – and it’s very easy to do: When a product is in beta the metrics may not be relevant to a year-old solution with a solid customer base. In fact, we need to identify which metrics to shortlist and which will be the single metric that defines the success of a solution in the wild.
What is that One Metric That Matters (OMTM)?
The OMTM is less a concrete thing, and more of a guideline that helps you, the solution owner, identify how a product is doing. The concept comes from Ben Yoskovitz’s “Lean Analytics”, a book primarily for startups, focusing on deriving meaning from data. When it comes to OMTM, different stages demand different metrics, and of those metrics, there should really only be one that you rely on to tell you how things are going. This sole focus helps cut noise from the signal and allows you to make very specific decisions for very specific reasons.
Steve Glosky, entrepreneur and founder, has put together a nice table of metrics describing which metrics can be used at what stage of the hypothetical innovation program in a corporation. Out of the many steps listed in each step, the OMTM framework demands that you obsessively pick just one to track and decide on it.
What happens once you’ve identified your One Metric? You measure it! You’re free to track other metrics too, but the One Metric becomes your North Star. As things mature (or reverse course), change what that metric should be for the various stage. Glaveski summarizes, and suggests themes for each stage of growth (again, for that hypothetical Innovation Program):
Since your initiative or product may be different, the above tables may not apply to you. Nevertheless, the concepts remain the same: cutting data noise from signals, and investing in that sensing and responding to that signal, will empower you to make impactful decisions for product development. Reduce the search for non-significant numbers, and see real progress where it matters.
Author: Christopher Goscinski