High cost experiments

I’m beat this week. I’m normally at my desk at 8am for my morning slate of calls, but this week has started earlier and my routine has been completely thrown off. I’m excited to get to the weekend and get some rest.

This week, my team and I have been working on a few projects that we’re thinking about launching. I’m huge on trying new things and seeing if they work. We’ve had a lot of great success running these experiments in the past, although nearly all of them fail. One way or another, you learn something.

With that said, one question I always ask is what is the cost of this experiment. That cost may not necessarily mean just pure dollars. Time is probably the most expensive cost we have at a startup.

For a simple example, if we go and chase a theory that sending a calendly link directly to our calendars instead of having them set-up accounts will generate more leads. I can likely tell you with good certainty that theory will likely be true if we run the experiment. On the surface, most may think that “more leads are great, let’s do it!”

But then if you start breaking down the cost of running this experiment, then things start becoming much murkier. You will likely introduce many more unqualified leads. Let’s say you expect 100 new leads and your normal unqualified lead % is around 50%. With the new method, you may think this will go up to 80%. So you have 30 more unqualified leads.

At an estimated 30 minutes per unqualified lead call, then you have 900 minutes of employee time of cost to get 20 more qualified leads. That’s 15 hours of additional work to get those 20 qualified leads. Depending on the business or project, that may not be worth the time now that you’ve dug into the numbers.