Grow your B2B SaaS Startup

Join my insider's club to learn the secrets of today's fastest growing startups

Morgan Brown and Sean Ellis on Growth Hacking Without the Hype

Morgan Brown and Sean Ellis on Growth Hacking Without the Hype

Subscribe on iTunes

Today, on the Early-Stage Founder Show, I’m talking with Morgan Brown and Sean Ellis, co-authors of Hacking Growth, a book that lays out how today’s fastest-growing companies drive breakout success. It also happens to be available in stores today.

Morgan and Sean both have extensive, hands-on experience helping startups accelerate their growth and are the co-founders of GrowthHackers.com, the top online community for growth hackers.

I’ll be honest, before this interview, the term “growth hacker” rubbed me the wrong way. It seemed a bit to me like marketing’s equivalent of “rockstar developer” and was little more than Silicon Valley hype without much substance.

After reading their book and talking with them, however, I can unequivocally state that my initial impression of growth hacking was off the mark.

If you want to hear what growth hacking really is all about and how you can use it to dramatically accelerate the growth of your startup, then this is the episode for you.

[convertkit form=4951237]

Topics covered:

  • (00:19) – Discussing what Growth Hacking is and some of the misconceptions surrounding it.

    • (01:26) – Elements of Growth Hacking.
    • (01:46) – The different facets of a team that affect the customer journey.
    • (03:18) – The myth of the Growth Hacking silver bullet.
  • (04:40) – The role of the founder and CEO in driving the growth of a startup.

    • (06:36) – A real world example of how a founding CEO successfully navigated a team through a growth challenge.
  • (08:28) – The methodology of Growth Hacking.

    • (08:55) – The first step in determining if Growth Hacking is suitable for your startup.
    • (09:59) – A powerful way to determine if you have a product or service worth scaling.
    • (11:11) – Segmentation.
    • (11:56) – Getting an honest answer from your users.
  • (12:59) – Understanding the levers of growth.

    • (15:03) – LogMeIn as a case study in applying growth equations to a business.
    • (18:06) – A detailed look at the customer on-boarding experience.
  • (20:20) – High-tempo testing.

    • (21:16) – Four-step cycle of experimentation.
  • (24:11) – The process of prioritization.

    • (24:40) – I.C.E. – Impact; Confidence; Ease.
    • (25:19) – Impact.
    • (25:43) – Confidence.
    • (26:17) – Results of a successful ticket sales experimentation
  • (29:24) – Testing advice for early-stage founders.
  • (35:44) – Long-term Growth Hacking strategies.
  • (37:40) – The driving incentives behind the writing of, "Hacking Growth".

    • (37:55) – Sean's Goals.
    • (38:51) – Morgan's Goals.
  • (40:12) – Growth Hacking advice for early-stage founders.

Where to learn more:

To hear more from Sean and Morgan, check out their book, Hacking Growth. If you want to get involved in the growth hacking community, head to GrowthHackers.com.

Transcript:

Andy:  00:00:13  Morgan, Sean, thanks so much for coming on the show today.

Speaker:  00:00:17  Thanks for having us on Andy.

Andy:  00:00:19  Of course!  So before we dive into things, there's a lot of misconception about growth hacking, so why don't we just take this opportunity for us to clear up a few of these.  The first one is really just a lot of people see Growth Hacking as just a tactical approach, where the goal is really to find one silver bullet that is just immediately going to lead to that hockey stick growth.  00:00:37 [To Confirm:  Hockey Stick Growth?]  But I know there's more to it than that, so what exactly 00:00:40 in your opinion is Growth Hacking?

Sean:  00:00:43  Morgan, I'll take that one if you don't mind.  So to me, with the silver bullet approach it's dangerous.  And you do have a lot of people who are just talking about Growth Hacking being this very creative one or two things that's just going to blow it up.  00:00:59  But I think the problem with that is that it tends to 00:01:02 [Edit Out:  Restart]  

Sometimes it actually works, but it's really temporary and fleeting and for growth to really work well, it needs to be about long-term value creation and about retaining customers and building sustainable growth.  00:01:15  [Tweet]  

So that's really what Growth Hacking is about, it's taking experimentation across the full customer journey to drive long-term sustainable growth.  00:01:26  It's everything from acquisition to conversion, getting people activated.  Ultimately retaining and engaging people and just building value over time.  00:01:36  Really building an engine around what your product's value is and building a customer acquisition and value delivery engine around that.  00:01:46

Andy:  00:01:46  It's something you touched on a lot there, because other areas I want to talk about where that it's more than just — it needs to involve more than just the marketing department and it also needs to focus more than just on acquisition.  00:01:59  Can you speak to that a little more as well?

Sean:  00:02:02  I think that's what makes it really hard.  If what — when marketing was just about acquisition, it's a single team, it's not that much to co-ordinate, but growth happens across all departments in the business.  00:02:18  That customer journey sits in the product team, maybe a customer success team, and even engineering to some degree is touching it.  00:02:28  

It just gets a lot more complicated when it's going across those different departments and there's a lot more co-ordination around the experimentation that needs to happen across that full customer journey.

Morgan:  00:02:40  And Andy I think that's one of the challenges, but also one of the great opportunities.  00:02:46  I think that a lot of the misconceptions around growth hacking focus on the top of the funnel, the customer acquisition.  

But for me, Growth Hacking is a process that can be applied to any level that creates growth for the business.  00:03:00  Whether it's the pricing of your product to the customer on-boarding, to your retention strategies, 00:03:07 and so acquisition is really just one part of the growth part — — one part of the growth puzzle and growth hacking is a process that gets applied across all of it.

Andy:  00:03:18  That makes perfect sense.  One other thing you had mentioned earlier, Morgan, before we actually started recording, was about how people should be involved, 00:03:26  especially the CEO.  Because so many times, I know especially when Growth Hacking started gaining in popularity, it seemed like every startup immediately just posted a job looking for a "Growth Hacker" 00:03:37  thinking that would be all they would need.  But what is the right way for an early-stage founder to be approaching this in terms of hiring?  Is it just looking for that one growth hacker?

Morgan:  00:03:51  I think that's one of the big myths.  That there is someone out there that might ride a unicorn, they might carry Pixie dust, but that will come and solve all that ails you when it comes to growth 00:04:04  and the reality is, like Sean mentioned, it's much more complicated than that.  00:04:11  

Those people aren't carrying silver bullets along with them, and so we really struck about the structure of a growth team.  That growth team is made up of professionalized engineers, product people, UX designers, marketing specialists, that come together and collaboratively design experiments.  00:04:32  Ones that they can run and see if those experiments move the needle and create lasting, sustainable growth across the entire funnel.  00:04:40  

I think what's really interesting at the early stage, though, there the founder and the CEO is involved in this process, and I think Sean does a really good job of articulating it, so Sean, I'll let you take that part.

Sean:  00:04:53  Yeah, I think the founder and CEO should be a partner in driving growth.  00:05:00  It really depends on the size of the company.  00:05:05  If it's a super early-stage company, they might only have two or three people and in that case, everybody's wearing a lot of hats and basically, not just the founder, not just the CEO, but really all three or four people will be focused on growth.  00:05:21  

And as the company grows and you do start to get these silos, then having a CEO involved to break down some of the challenges of communication and buy-in that sometimes happen when the 00:05:35  product teams are not used to necessarily having data-driven experimentation with an objective.  

They're often a little more creative, 00:05:44 yet they hold some of those really powerful levers of growth.  00:05:49  Being able to have them involved often requires the CEO to step in and help them understand the importance of their role.  00:05:59  

Once a habit is sort of started then it's a little bit easier, and then in that case, the CEO might be a little bit of a lighter touch on things and should at least attend a weekly growth meeting 00:06:10  and break down some of the barriers if the growth team is having a hard time getting access to a designer, for example.  00:06:19  Being able to try to see what is going on with that designer.  Are they at capacity doing other things?  Do we need to put in a new dedicated designer — a dedicated designer for growth.  But really making sure that the resources are in place so that the tests can be run to drive growth in the business.  

Morgan:  00:06:36  Yeah, and Andy, just to kind of give an example.  00:06:39  When we were very early on at GrowthHackers.com 00:06:44 [Resource Mentioned] I was working for Sean and we kind of launched that community, we were about a year into it and we had come to realize that we our growth had stalled out a bit.  00:06:55  

And then Sean kind of dug — you know we're a small team.  One product person, one marketing person, one engineer, and when we kind of dug into why we had — why our growth had stalled out, nothing had really changed on the outside, but we came to realize through some self-examination, is that all of us had gotten busy wearing the many different hats, taking care of this community, but we had lost our focus on growth experimentation 00:07:25  and really being laser-focused on how we prioritized our day and our time and our focus on growth.  00:07:32  

And that's when Sean came in and said, "How many growth experiments have we been running?  Where's our focus?  Where's the resources being allocated to?"  And our product team is busy working on admin tools, and marketing was doing reporting, and we weren't really laser-focused on continuing to approach it and do growth experiments.  00:07:52  And I think that's something that early stage companies, when you have so many hats, so many things to tend to, that it can fall of your radar, and I think this process and this mindset helps avoid that.

Andy:  00:08:04  Right.  And especially when you get out of the mindset of looking at it like you're almost randomly trying a few tactics that you read about.  And you understand that there is a true methodology behind it.  00:08:15  It makes so much more sense.  And so you guys give a ton of great context to get us ready for this.  

And in your book, "Hacking Growth", 00:08:22 [Resource Mentioned]  which is actually out today, you go really deep, both into the methodology and the actual playbook for applying it.  00:08:28  For today, I think it would be important to cover the methodology of how to really apply the growth hacking mindset strategy, whatever you want to call it, in your startup.  

00:08:39  How do startups get started with growth hacking?  Where does methodology begin?

Sean:  00:08:47  I'll take that Morgan, if you don't mind.  So the first thing is, Growth Hacking is not for every startup immediately.  00:08:55  The one thing that you want to make sure is that you actually have something of value to scale, and so the very first thing that I recommend any startup do, is try to figure out if you have product market fit.  00:09:08  

If you have a product that is a must-have.  Because without that, you're not going to be able to retain customers and you're not going to be able to drive sustainable growth in the business.  00:09:16  The best signal of product — of a product being a must-have, is actually retention, but it's a little bit chicken and the egg.  00:09:26  You have to acquire people to retain them over time, and it takes a long time to [Inaudible 00:09:30] retaining them.  

So I have another way that's kind of an early indicator of retention, which is, you get some people on the product, so you don't know if you have something or not until you've got some people on the product.  00:09:41  You get some people on the product and then you just ask them once they've experienced it, so they've really — they've got through maybe a convoluted funnel, but some of those people got it and they're actually using the product in the way that you intended it.  

In a way.  Just ask them how they would feel if they could no longer use the product.  00:09:59  I found that that's just really powerful, because I give them a multiple-choice question.  Very disappointed, somewhat disappointed, not disappointed and I'm looking for the people who say they would be very disappointed without the product.  00:10:14  

If they say they would be very disappointed, they are likely to stick around on the product for a long time, but once you know who would be very disappointed, then you can learn a lot more about those people and you can start to really build your growth engine around including more people like those people.

Andy:  00:10:30  Is there some sort of rule of thumb?  Like you're looking for at least many, this sort of percentage of people to be very disappointed to know that you're at least on to something?

Sean:  00:10:40  Yeah. I mean, so — in the beginning you're looking for just a single person.  That's really scary when you start running a survey and everybody says somewhat disappointed, or not disappointed, but what I've found, is if I see 40% of the users on a product saying they would be very disappointed without the product, I've found that every one of those businesses has scaled long-term.  00:11:06  They've been successful.  The ones that I've worked with.  

And it's not the end of the world if you have less than that.  00:11:11  So one thing you can do is segment your users.  00:11:14  So let's say you're using a product that would only be useful for men and if you eliminated the women, suddenly that product goes up to above 40%.  So that's fine.  

So you just narrow it down to a segment.  And then I have other things I do.  If they use it this way, then it's above 40%.  Ok, so now I know this is the use-case I need to be pursuing, both in messaging and in on-boarding, and I've been able to pretty quickly move some products from a 10-15% to a 40% once I set the right expectations and fine-tune the on-boarding to deliver that user experience.

Andy:  00:11:56  And why is it that you focus on the disappointment on losing the product rather than their happiness with actually using it?

Sean:  00:12:03  I think you just get a more honest answer that way.  00:12:06  My very satisfied and your very satisfied might be different.  But I think disappointment is just a better signal of somebody — it's a more visceral reaction when you say, "How would you feel if you couldn't use this anymore."  

I've actually seen people panic when they see that question.  "Are you going away?  No please don't go away, I'll do whatever it's going to take."  00:12:34  People — sometimes founders get a little nervous that they're going to send the wrong signal with that, but to me I think that risk is worth it to know if you have something.  00:12:44  Because if you don't have something and you try to scale that business, there's a good chance you're going to run out of money, and that is a bad place to be.

Andy:  00:12:53  For sure.  But so say you do have something, what happens next?  What should happen next?

Sean:  00:12:59  Morgan, do you want to take that?

Morgan:  00:12:59  Yeah sure.  So I think once you identify that must-have product, I think what's really critical is then to go and understand what drives your growth.  What are the levers, what is the formula.  We talk in the book about a growth equation, which is simply a way to express the different factors that go into driving your growth.  

So for example, Andy Johns 00:13:33 [Resource Mentioned] who is a really well-known growth expert, talks about Amazon's growth equation being a combination of both the number of categories and verticals for products that they offer, the inventory in each one of those verticals, the conversion rate, the average order size, and the retention of customers who come back and buy again.  

So, each company has a growth equation to it.  It's kind of like a thumbprint that is unique to them.  00:14:03  But it's really important to figure out first what levers do you have at your disposal that contribute to growth.  00:14:13  

So whether that's the price of your product, or the use-case, the type of person that uses it, or how long they should be using it for.  They all go into this equation, and really, the equation shouldn't be — it doesn't have to be perfect, but it's really supposed to be heuristic, or a guide to think about, "This is how my business operates.  This is how my business grows, and when I have that framework in mind, then I'm not just throwing stuff against the wall to see what will help me grow, but I'm consciously applying my effort towards those levers in that equation that I know will drive me forward."  So, it's kind of a crystallizing, clarifying statement about how your company actually grows.

Sean:  00:15:03  I can actually give you a great example from that, that will bring it to light a little bit.  00:15:10  So when I was at LogMeIn 00:15:12 [Resource Mentioned] we had a really hard time, kind of going beyond the initial $10,000 a month in spending.  So we were — I took the traditional approach to marketing.  Marketers are trusted with acquisition.  I kind of tried to go deeper with optimization, but I was pushed out by the engineers and the product team.

Andy:  00:15:33  When you say $10,000 a month in spending, do you mean on ad spend?

Sean:  00:15:36  Yeah,  Adwords worked pretty well for us, so we could scale it to about $10,000 a month with a profitable return on investment.  00:15:44  As soon as I tried to go beyond that, I was having a really hard time scaling beyond that.  

We were a venture-backed company, so I had to be able to spend more or find a way to grow at a faster rate than we were growing and when I looked at what happened, I was driving a lot of signups, but most of those signups were never actually using the product.  00:16:04  

And so, if they're not using the product, then I'm not going to get the return on investment.  They're not going to buy, they're not going to tell their friends, all the value creation isn't going to happen.  00:16:14  

When we actually mapped out the value creation process like Morgan was just talking about, we found that there's a number of steps that lead to people becoming valuable customers, and most of those steps were very flawed.  00:16:26  

And so when I presented that data to the CEO, [Inaudible 00:16:30] [To Be Confirmed] bring in the CEO, what could he do, he looked at it and he said, "Oh, I see why you're having such a hard time generating growth within the business, this needs to be our number one initiative.  For the foreseeable future, what is that first user experience like, how do we optimize that first user experience to get more and more of those people actually using the product."  

00:16:50  Full effort of the team went into that for a few months, so in our case, they [Inaudible 00:16:57] but it was really that leverage point for growing the business.  We're able to increase the number of people who signed up and actually used the product by about 10X.  00:17:08  

It was single digits when we started, but we improved it by about 10X, and what that actually meant to our ability to drive customer acquisition was, instead of scaling to $10,000 a month, no new channels tested, just retesting the ones we'd previously tested, now scaled to over $1,000,000 a month with a three month payback on dollars invested.  00:17:30  

Once you kind of understand what path that that customer takes, then you figure out where it is most broken 00:17:39 [Tweet] and if I fix that, — no one is going to be retained if they don't get activated, and I just showed how hard it was to acquire if we don't have activation.  00:17:51  

So fixing that activation helped both the retention and the acquisition.  Today, LogMeIn is a five billion dollar business, but I literally think it would not be in business if we hadn't figured out how to get the whole team working together across that customer journey to get it right.

Andy:  00:18:06  And so, to back up a little bit, what does — and this may be obvious, but what does the actual process look like for figuring out what matters during the activation stage?  How do you go through and systematically see what parts of the on-boarding experience, the first experience, do we really need to look at here?

Sean:  00:18:30  I'll give you a real specific example from LogMeIn there, we had one place — so it starts with quantitative analysis.  So you're basically just looking at the data.  When we looked at the data, we saw that people who signed up were not downloading.  00:18:46  

The step at sign-up to download was very flawed and essentially, what we did was started to do some testing, so kind of spaghetti on the wall type testing that a lot of people do.  It wasn't effective, but we had the advantage of actually having the email addresses of the people.  00:19:05  

Now we did some qualitative research in addition to the quantitative data, and the qualitative research was literally just sending an email to people and asking, "Why did you signup for this thing and not download it?"  00:19:17  They consistently told us that they didn't believe it was free.  It was a freemium model.  

Once we knew the answer — now we were solving from known problem, which is people don't believe this is free, our next test tripled the download rate.  00:19:32  Just with that one step, the next test tripled the download rate, which was to give them a choice.  Download the free version, or download a trial of the paid version.  Once we saw we had a business model, it was a lot more credible that, "Ok these guys probably do have a free version and they're showing that they also have a paid version."

Andy:  00:19:49  And because this was a few years back when the freemium model wasn't as popular at all.

Sean:  00:19:55  Yeah, so this was right in the pioneering days of the freemium model, so people were a little less convinced that it was real when they saw it.  00:20:05  Especially when they saw something like, GoToMyPC,  which was the competitor at the time.  Spending hundreds of millions of dollars promoting that $20 a month product, and then they see that we were offering the same thing for free.  It just seemed to good to be true.

Andy:  00:20:20  And so once you have identified those growth levers, you touched a little bit on what you did in those examples.  But generally speaking, what is the next step once you see where you can get those leverage points?

Sean:  00:20:35  Morgan, do you want to take that?

Morgan:  00:20:36  Yeah, sure.  I think really that is when it's time — if you have product market fit, you understand what the levers are, then it's really time to move into what we call a high-tempo testing.  

A rapid experimentation process.  00:20:52  And I think, it's a combination of speed and control.  So when we talk about going fast, we talk about going fast like a formula race car, rather than a truck whose brakes have gone out down a mountain pass.  00:21:10 [Tweet]  But you want to move quickly, but thoughtfully.  

So we talk about a four-step cycle 00:21:16 which is a loop that you move through.  It includes coming up with new ideas.  New ideas for experiments.  Sean calls it unbridled ideation.  Where you take the data that you're seeing, along with the qualitative feedback that you get, to come up with a whole bunch of hypothesis around experiments that you can run to drive growth.  00:21:38  

And these can be experiments to improve on-boarding, experiments for new customer acquisition channel to try all the way down to experiments for pricing tests and throughout your funnel.  00:21:53  Really not limiting yourself to one area, but using the context of your growth model to come up with a whole bunch of ideas.  

And then once you have all these ideas, it's being really rigorous and prioritizing what to test first.  00:22:08  This is so key, because we only have so much time and so many resources to do so, so again, you can't just throw a bunch of stuff against the wall.  

You have to be really rigorous in prioritizing.  We can get into how we prioritize if we want, but everyone has a system for prioritization and then experimenting.  00:22:30  Actually running the tests at a high-tempo and then analyzing the data that comes back from those tests.  00:22:36  And that data, that data collection and analysis is so key, because that gives you the basis for your next turn through that process, and so the learning really informs where you go next, and that learning is what helps you compound your wins and cut your losses each pass through this high-tempo cycle.  00:22:57  

And that's kind of the theoretical framework.  And there's a couple pieces in there that are important that we can dive into, but at a high-level, that's how you should come to think about rapidly iterating to drive growth.  And Sean, I don't know if you have anything you want to add to that.

Sean:  00:23:12  Yeah, the one thing I would add to it, is that while the ideation is somewhat unbridled, generally it works best to apply that process that Morgan just mentioned to a specific objective.  00:23:23  

So basically saying, "This is the high-leverage opportunity in the funnel," like I was just talking about.  So, activation and [Inaudible 00:23:31] — if we can fix this, this is the impact that it will likely have to our overall growth and so let's focus on this.  

And so now when you are prioritizing you're trying to figure out, just looking at those ideas that map to that goal, which of those ideas make the most sense to test, and then you work that goal for awhile.  00:23:52  

You might have two or three simultaneous goals if you are a bigger team, but it's about leverage is such an important concept in growth, that if you don't identify where that leverage is, you're likely to dilute your efforts in maybe some areas that are less important to your overall growth in the business. [Tweet]

Andy:  00:24:11  Interesting.  And you touched on it a bit there, but I'm curious to dive a little bit deeper into the process of prioritization.  00:24:19  Obviously you want to make sure that the tests that you're running, the experiments that you are running, are closely aligned with those goals that are important to you.  But what else goes into deciding which tests to run first?  Do you use a prioritization framework to do this?  How do you guys approach this problem?

Sean:  00:24:39  Morgan, do you want to walk him through ICE?

Morgan:  00:24:40  Yeah, sure.  So Sean came up with a system that we use and that we talk about in the book, called ICE.  It stands for Impact, Confidence, and Ease.  And so, essentially what you are trying to do, is take each experiment, or each hypothesis, that you have and grade it, or rate it, on these three different vectors.  00:25:00  

What we say, is on a scale of one to ten, for each of impact, confidence and ease, give the idea a rating.  And then you take the average of those three scores to create a final score for the idea, and that allows you to force rank ideas accordingly.  00:25:19  So, impact is all about, "If we run this experiment, what do we expect to do to our growth?"  

There are some ideas that are very high impact and there are others, even if you execute them flawlessly, they will not be incredibly meaningful.  00:25:38  Particularly towards an objective that you're trying to run down.  

00:25:43  Confidence is a judge of, "Is this a blue sky idea or a wild hair idea?  Or do we have a really solid foundation for our understanding of how this is going to impact growth?"  And confidence comes from data analysis, past experience, other imports like, for example, if it worked in some other similar model, maybe a board member, or maybe a founder had an experience in the past.  00:26:12  What's the confidence of the team around this experiment.

Sean:  00:26:17  If I could just interject one thing on the confidence?  Incredibly, yesterday we ran an experiment in our business.  Actually a couple of days ago, based on something Morgan had done in his current business.  And we both have conferences with our businesses and Morgan had recommended that, "Hey if you do this then I think it will lead to 15 additional ticket sales.  00:26:40  

Very specifically, it consistently leads to 15 additional ticket sales when we do this."  So we said, "What the heck, let's implement it" and it literally led to 13 additional ticket sales in two days.  Applying what worked really well for Morgan.  

So it does show, while you don't want these growth hacks that cross from one company to the next, because they often don't work out of context, when something is a best practice or something that has worked recently somewhere else, that confidence score can go up quite a bit.  

As Morgan was saying, it's not just a wild guess, but something that is proven somewhere else and it may work in this context as well.

Andy:  00:27:23  And so while we don't want people to just rip off these tactics without thinking about the context and thinking how it applies, I feel like people are going to get mad at me if I don't ask you what was the strategy that you applied to get those extra ticket sales.

Sean:  00:27:36  [Laughter]  Ok, yeah.  Morgan, are you ok with me sharing this?  It's your baby.

Morgan:  00:27:40  Yeah, it's fine, it's fine.  Conference organizers everywhere:  Enjoy!

Sean:  This is really cool.  One of the things you find when you're selling tickets to a conference is that you have pricing tiers that go up every few weeks, or every month, and generally you get most of your ticket sales right as the price is about to expire and go up to the next level.  00:28:03  

Which makes sense, but Morgan said, "Hey one of them that has worked really well for me is, after the price has expired, send another email out the next day, saying, "Hey a few people have asked to extend — have asked to get the old price even though it has expired.  And since we gave it to them, we're going to extend it to every one else for a couple of days.""  And so, that was the experiment.

— and what's interesting is that it's not a lie in the email, because every time the price goes up, you get a flock of those emails.  Just putting out there, so we put a new code on that extended price and we ended getting an additional 15 ticket sales.  00:28:51  And we actually got an additional 21 ticket sales that some people bought some big block tickets, where they probably realized they could save even more if they bought big block tickets.

Andy:  00:29:04  Interesting.  And so, building on that into talking about running these tests.  One thing that I'm immediately thinking of is typically A/B testing with CRO kind of stuff, where there is a — you need to have a certain amount of traffic if you're testing certain elements on the landing page or whatever it actually is.  00:29:24  

If you are an early-stage startup that doesn't necessarily have a huge volume of leads or new trials or anything like that coming in, how should you think about testing?  Is there a point where it's just too small to actually rigorously run some of these tests.

Sean:  00:29:42  Yeah.  So in an earlier-stage company, what you don't want to get caught up in is super-incremental testing.  Because the fewer people that you have flowing through your site, the more likely that you are going to need to run — you're not going to have enough people to make a read until six months later if you're doing a super-incremental test like changing one word in the headline, for example.  00:30:05

So you want to do these big bold tests that fail spectacularly or succeed spectacularly, but you're going to get a signal one way or the other.  So a lot of times that helps you to run a test on a smaller sample size, but I do think that your — that you do go a bit more by gut at that level, and there's some sort of purist testing people that will probably beat me up for saying that, but I just — I think a lot of it is — testing is part of your research process, so you're running usability tests to see where people are getting stuck.  00:30:45  

You're running surveys, you're keeping an eye externally to what your competitors might be doing.  You're doing all of these things to try to figure out a better way to do things, and sometimes you start to run a test and you're not seeing enough signals so you end it early.  

But I think a lot of times, people get caught up in trying to run 'the perfect test' and as a result they run only two or three a year, and in an early stage startup, you need move around a lot more than that, but you want to have enough discipline so that you're not just randomly changing a bunch of stuff without paying attention to the data.  00:31:23  But I don't know Morgan, we haven't really talked about that.  Do you agree with that?  Or do you have a different perspective?

Morgan:  00:31:31  Yeah, no. In general, I definitely agree, so I think this is where people take advice from different stages of companies and apply it to their situation incorrectly.  

For example, if you're at Twitter or Facebook, or Air BnB or Uber, and you want to run an experiment, that's going to go through a team that does experiment review.  Right?  And they're going to — it's going to be a team of data scientists who are going to select the audience, make sure there aren't any confounding factors, make sure the experiments designed properly, with the proper statistical significance and power and all of this applied to it.  00:32:09  

And that makes sense, because we're talking about hundreds of millions of people, and they're at that point where a small incremental change deployed to hundreds of millions of people can make a really big impact.  

00:32:22  Startups, on the other hand.  People with small traffic sources like Sean said, should be looking for really large swings.  So you're actually testing for impact.  You're trying to see what is going to move the needle.  

Whether it's positive or negative, even just finding out what those things are really matters.  00:32:41  And so, you shouldn't take the incremental approach really early on.  Like, "Hey I'm going to change a button colour" or, "I'm going to add a testimonial here."  

You should be thinking about redesigning the entire page or recreating the entire user on-boarding experience.  And then the data.  00:33:02  When we talked about experimenting, we're not just talking about A/B tests.  We're talking about in-channel experiments, we're talking about wholesale user experience rewrites and that type of thing.  00:33:15  

And so then in those cases, what you're really looking for is data that's both quantitative and qualitative right?  So, like Sean said, usability studies, user testing, running things through UsabilityHub 00:33:32 [Resource Mentioned] and UserTesting.com 00:33:33 [Resource Mentioned] to find what works better for users.  

There's a whole bunch of ways to find out if what you're doing will make a positive impact rather than just A/B testing.  And there's a lot of really great advice on low traffic experimentation that's written on blogs like ConversionExcel.com 00:33:52 [Resource Mentioned] and whatnot.  But I think that early on, you're going for impact, you want to use the data where you have it, and then you want to use heuristics and other outside tools that will help you know if you're headed in the right direction.  

And then, of course, on the back-end 00:34:07  you're looking at metrics.  The analytics that you're looking at are things like retention.  You're not just looking at the click-rate on the page.  That gives you kind of an idea if you're headed in the right direction or not.

Sean:  Just one thing.  When Morgan was saying you're looking at retention, you still do tend to have some early indicators on the data as well.  00:34:31  You don't want to look at click-rate, but at the same time, if nobody clicks, you know it's not going to work.  

So you kind of want to get an early read on upper funnel stuff to tell you if you think it's going in the right direction.  And then, even if you end that test say, based on that information you end it after just a week or something, you want to go back and revisit the results and see what did the retained usage of the longer term goals, did the test still win based on the longer term goals, or still lose based on the longer term goals.  00:35:08  But you don't necessarily need to run it the entire time.  You just need to study the cohort that was affected by it.

Andy:  00:35:13  Yeah, that was a great answer from the both of you guys, because this is something, especially with testing, where it is so easy for people to get caught up in things that ultimately don't make that much of a difference, and at the earlier stages when they don't have the big sample size, focusing on impact, focusing on the things that are going to make the bigger difference, that are easier to measure.  And ultimately, easier to get that gut feeling for is really good advice.  00:35:37  So I'll make sure to link up some other resources listeners can go to to look more into this.  00:35:42 [To Add]

00:35:44 But what I want to ask now, is once people have got this growth hacking machine in motion, is it just repeating this last loop of testing at a high-tempo?  Is that the long-term play?  Just keep going through these loops more and more frequently?

Sean:  00:35:58  It is continuing to test at a high tempo, but it's also continuing to study the data to try to find those high leverage opportunities to apply that loop to.  It's a lot of data analysis and then understanding qualitatively what's going on behind the quantitative data, and then applying the loop of testing to that opportunity.  And so, it's really moving from one opportunity to the next and really trying to focus the efforts around where you're going to get the biggest bang for the buck.

Morgan:  00:36:39  Yeah, I think that is really the key.  Your objective isn't always the same.  And so, it's constantly, as you achieve certain objectives, re-evaluating what's the next milestone, what's the next objective, what's the next area for growth, and then applying this framework and resources towards that objective and constantly moving the ball forward.  00:37:03  Zillow, which is a publicly traded company 00:37:06  [Resource Mentioned] they run something called, "The Play", which is a year-long initiative for the company.  And each year, they set a new Play.  And then that helps direct the growth and product, and marketing teams towards that big objective.  

Now smaller companies might set a new objective every six weeks, or once a quarter, or something like that.  But constantly evaluating where the next big opportunity is and deploying this process and resources toward it is the secret sauce.

Andy:  00:37:39  [Inaudible 00:37:40]  I have to ask at this point, what was it about writing this book, "Hacking Growth", what was it about that that made you to prioritize and write that now?  How does this fit into your overall growth hacking strategy?

Sean:  00:37:55  Well, Morgan and I probably have somewhat different goals from the book.  For me, the book is really aligned with my broader goal with GrowthHackers.com of educating people about where we think marketing is headed, where we think growth is headed.  

Marketing of yesterday is really all about acquisition.  Marketing going forward 00:38:21 [Tweet] is a cross-functional approach that requires a much different thought and integration with the rest of the team.  

And the more that we can advocate the benefits of that and educate people on how to be effective with it, the more likely we're going to create customers for our tools that help them be successful.  So there is a direct business benefit for me.  

I think Morgan and I are both just really passionate about it so we also wanted to write the book.  But Morgan, do you want to share some of your specific objectives around it?  00:38:52

Morgan:  00:38:51  Yeah, I mean, for me, to be totally honest, growth is my hobby.  Much to my wife's dismay, this is the stuff that I like to do and think about.  And I think what Sean said is really important, is that marketing constantly evolves.  

The way products grow constantly evolves.  00:39:10  The way that today's largest companies, fastest growing companies grew are fundamentally different from how the companies of say the past 50 years grew.  00:39:23  And so, there's a lot of confusion out there, in terms of, "How do I do it?"  

There's so many listicles on growth hacks and a lot of confusion on how this actually works.  And there's some really smart people doing some really incredible things to systematically grow these companies and it's not an accident that it happens.  

So really just documenting how they do it.  Helping marketers, product people, entrepreneurs, executives, understand what's changed and how they can apply it to their business is really the motivation for me.  00:40:01

Just to get to go deep on all of these companies and kind of distil it down into something that hopefully is digestible for anyone trying to grow.

Andy:  00:40:12  And so, to tie it all together, what advice would you each give to early stage founders who definitely are seeing the power in this growth hacking methodology, but are unsure of how to actually get started with it?

Sean:  00:40:24  My advice is pretty simple.  Validate product market fit.  So we talked about make sure your product is a must-have, understand the value that makes it a must-have, and then calibrate your business to that value, through rapid testing.  00:40:40  

So, everything is about re-orienting your business, whether it's your messaging, your on-boarding, the channels that you're trying to reach, but all of that should be around ultimately to experience the value your product has to offer.  

Morgan:  00:40:54  The only thing that I would add, is that as an early stage founder, is to pull your team together and affirm to all of them that they have a hand, a role, and a responsibility in driving growth.  

So, if your engineering department or your CTO thinks that your head of marketing does nothing but spam customers, that's a cultural problem.  Or if your product person only wants to refine the user experience, but isn't thinking about plans and [Inaudible 00:41:25] because that's not what they're interested in, that's a problem.  

So I really think aligning the team around, "Hey, we're here to create value for our customers, and ultimately growth comes from creating more value."  

00:41:41  And everyone on this team has a role to play in it, and we're going to work going forward to make that happen.  I think this is the first thing to do right out of the gate.

Sean:  00:41:52  Yeah, Morgan just to add one thing on what you just said.  That role that everyone plays in growth is so important.  And where I saw that, the lightbulb went off for me so much, was when I was at Dropbox. 00:42:04 [Resource Mentioned] 

We were less than ten people, everyone was an engineer there except for me, and this was an interim role.  I was only planning to be there for six months, but my big goal was to create this culture of growth where everyone understood what the role was.  

After I left, it took another nine months for the next marketer to come into Dropbox.  But that whole team was executing a process that we've talked about and they're the fastest company — the fastest SaaS company that ever reached the one billion revenue runway, and I think it's just because they've created this machine, where everybody — very engineering driven culture, but a culture of engineering that cares a lot about customer value and growth.

Andy:  00:42:49  You guys shared so much today, but even with that said, the book, "Hacking Growth" which is actually out today, goes into all of this in much more detail.  And half of it is dedicated to what you call, "The Growth Hacking Playbook", which gets into a bit more of the specific examples, and the tactical side of it.  But if listeners do want to check that out, which I really hope they do, what is the best way to do so.

Morgan:  00:43:13  I would say, go to GrowthHacker.com.  They can learn about the book and what's in it, and the process that we've talked about, and pick it up at their favourite retailer. 00:43:26  [Where To Find Out More]

Andy:  00:43:27  Awesome.  I'll make sure to get that linked up in the show notes and guys, I just want to say thank you so much for your time today.  It was a lot of fun chatting.

Morgan:  00:43:34  Thanks Andy.