Lies, Damned Lies, and Analytics: In conversation with Ben Yoskovitz

“Entrepreneurs are particularly good at lying to themselves. Lying may even be a prerequisite for succeeding as an entrepreneur – after all you need to convince others that somethings is true in the absence, of good hard evidence.” – Alistair Croll & Benjamin Yoskovitz (Lean Analytics)

The tech community loves celebrating success, and there’s nothing wrong with that. Looking at recent headlines there’s been no shortage of noteworthy success. Yet, these stories shouldn’t perpetuate the mythical notions of instant and overnight successes. Creating a bigger distortion field helps nobody.

If you’re not looking at the right numbers, the right way, and asking the right questions, your odds of success are most firmly found between slim and none.

Putting the human face and human touch into stories about entrepreneurship and the startup life is at the core of what we do, but there’s also no getting away from the fact that numbers matter too. Numbers such as page views, time on site, and even those vanity metrics showing up in the social sharing column matter to us. Numbers come into almost every story, like dollar amounts raised and earned, user growth, and team growth. When it comes to numbers though, it’s always worth circling back to Mark Twain’s great quote, “There are three kinds of lies: lies, damned lies, and statistics.”

At heart, I don’t consider myself a numbers guy. However, I do know for every early stage startup if you’re not looking at the right numbers, the right way, and asking the right questions, your odds of success are most firmly found between slim and none.

Catching up with Ben Yoskovitz co-author of Lean Analytics, (and current VP of Product at VarageSale) was a great opportunity to talk about why analytics matter.

Interesting enough, Yoskovitz didn’t see himself as an analytics guy early in his career, saying, “I had read ‘Four Steps to the Epiphany’ by Steve Blank, and most of the companies I had been involved with were B2B which the book was really about. It wasn’t really about the B2C business model. While reading it, I found myself saying ‘I made that mistake; I made this mistake’ and almost became embarrassed because it was as if Steve was talking directly to me, and saying ‘you made all of these mistakes’.”

It also proved to be good timing for Yoskovitz. It coincided with the early days of lean startup methodology and him working with three other partners to launch Year One Labs. “With lean starting to explode and looking at the works of Steve Blank and Eric Ries, it’s like wait a second,” he said. “There’s a framework here. There’s a model. It’s somewhat theoretical but clearly supported by some level of evidence, and also your own mistakes.”

With all of that in mind, Yoskovitz and the founding team decided that Year One Labs was going to be a lean accelerator. “We didn’t have a curriculum like a school, but we had program that we wanted companies to go through, and they unlocked funding a they made it through this different stages.”

“They say build, measure, learn. How do we measure? How do we use the data? Somebody has to write more about the data, and understanding the data.”

It was also a one year program versus the popular three-month approach. Here’s how Yoskovitz explained the decision: “My hypothesis was that companies needed longer to bake. It wasn’t about building for a pitch, it was about building to get them out the door with traction. We saw that for most companies it didn’t take a year, it took seven to nine months. But then when we looked at that and trying to apply lean principles you could see a very clear gap from theory to practice. Okay, they say build, measure, learn. We get the build part, in theory we get the measure and learn part, but it’s more about what do we measure? How do we measure? How do we use the data? How do we map these businesses to really focus on things that actually matter right now? My own experience in my own failures, the lean movement, and my own experiences running accelerators sort of combined to ‘somebody has to write more about the data, and understanding the data’.”

Behind the data points are real people, and Yoskovitz shared that “the risks with data is that you get so focused on it, and then you lose perspective on the humans behind it. The way I think about that, the data tells you what’s happening. Assuming you’ve instrumented it correctly, it won’t lie. But it doesn’t tell you why things are happening. In the book, we talk a lot about balancing qualitative information and quantitative. In fact, you absolutely need both.”

“A mistake a lot of companies make, is that they start with some qualitative stuff, think it’s informative enough to build something and then start measuring the data and forget to go back and talk with customers,” Yoskovitz continued, saying that it’s all about the balance between numbers and the people. “All the answers are not just sitting there in the data alone.”

Yoskovitz claims that the idea of finding one metric that matters is something startups have to embrace. “One of the things that derails startups is a lack of focus, and an attempt to tackle too many things, try to solve too many problems, to go after too many market segments. I do think at any given point in time, you’re attempting to solve one real problem which has one number. As a company evolves, I think one metric that matters becomes more of a health indicator for the business.”

There’s no shortage of metrics to track and measure. But when the team is small and you’re still in discovery and on the path to finding a product market fit, tracking too much can be a distraction. “You never know when you’ll extract value out of you data,” Yoskovitz said. “But it’s also so easy to get caught up in it that you get this paralysis by analysis. Maybe you don’t have just one metric, maybe you just have a big one at the heart of things and five smaller ones around the edges. That’s a lot better than 100! There’s a bit of theory there, that says just focus on less stuff. But also practically, you can just say ‘I need to lower churn; if I can’t lower churn then there’s no point in spending more money on acquisition’.”

In the end, what Yoskovitz is really talking about is intellectual honesty. “It’s so easy to delude ourselves,” he said. “Some of it is necessary, as entrepreneurs need to have some elements of distortion field around them to survive, but it is so hard. But then it’s also easy to just look at this data, and not look at that data and be like ‘oh, everything’s okay’ because it’s too hard to accept that it can be bad right now and just focus on the good stuff to make things look better than they are.”

With that said, Yoskovitz provided a final takeaway for entrepreneurs: “always ask yourself, am I being intellectually honest with what’s going on?”