Bursts of Color - The Power of Experiments
Last week I received a fun surprise in the mail: a signed copy of The Power of Experiments, a new book from my friend Mike Luca of HBS. The book is a nice roundup of experimentation in business and government over the past couple decades.
Testing Early and Often
When we first started selling ads at Yelp, we had no idea what customers would be willing to pay, so we picked a number and priced our initial packages at $50 per month. We wondered if the market would bear more, so the next month we raised prices to $100. To our sales team's surprise, new customer signups actually increased at this higher price. So the next month we tried $150, then $200, then $300... where we found a good equilibrium point that held for a couple years. Of course this testing wasn't very scientific, as we had few sales people and no real control group. Nonetheless, the results were pretty obvious (ACV up 6x with no decrease in velocity!), so it taught us the value of testing even when it's the "quick and dirty" kind.
Behavioral Economics
The beauty of experiments in human systems is that people so often surprise us. In fact, they often do the opposite of what rational economics says they should... like the pricing example above. I find behavioral economics endlessly fascinating for this reason. A few of my favorite books on this topic are:
Predictably Irrational, Dan Ariely
Misbehaving, Richard Thaler
Thinking, Fast and Slow, Daniel Kahneman
All Founders are Behavioral Economists
A few years ago Mike Luca (author mentioned above) and I co-wrote this article about Thinking Like a Behavioral Economist. The headline is that all founders and executives are choice architects for their teams and users, whether they want to be or not:
The idea of neutral decision making is in many ways an illusion... Just as architects choose where to put an elevator and where to put stairs, a choice architect shapes the way a decision is framed... The management team must choose when to default employees into a 401(k), how to shape the interview process, and whether to stock the kitchen drawer with candy.