For success, outsource QA testing to your customers

Aug 8, 2013
Vote on Hacker News


There are a surprising number of startups, especially in France, working their hearts out to build and launch a beta without having signed any paying customers. This build-first validate next approach is flawed.  Involving paying customers as soon as you start working on your software gives you an incredible edge in building the right software as it forces you to tackle an adage from the seventies: If you make something idiot proof, someone will just make a better idiot!

Let’s look at the how these first few customers can help you build the best software possible. (I’ll spare you the other obvious benefits of early customers: cash-flow and traction, two neat little features.)

Even if you only are at the idea stage, strive to secure a few paying customers to validate your idea. If people are willing to give you their credit cards, then you are onto something and you should start to code in earnest.  Now, we do enterprise software and I agree that you cannot go to an enterprise customer with “just and idea”.  However, you can (I promise you can!) get money from a customer just by using a mockup.  You do not need a full beta to go prospecting.  If you need to create a prototype, check out Balsamiq mockups: It is the simplest, most effective, tool to create mockups you can “sell” to your customers.

Creating software is not building a field of dreams; people will not come just because you build it; it is unlikely to work even if James Earl Jones is one your early backers. (* this last sentence is also a hint of a forthcoming post about adapting your software to cultural differences, first non-US reader to get back to us gets a prize!)  As you actively engage your early customers in your development effort you will get tremendous benefits:

  • Your software gets evaluated by users who are only concerned about using the software in a way that benefits them, not just completing the test cases you give them. You can see how their reality stacks against your beliefs.
  • Your marketing team gains “user stories” that will help market the software better.
  • You start generating buzz.  Your customers are excited to be part of the process and are more likely to talk about you to their peers.  This generates envy (the peers now want to participate). Word of mouth trumps SEO any day of the week
  • You learn about your software’s limitations and whether it makes sense to address them.
  • Most importantly, you build stronger ties with your customers because you are now embedded in their activities.  You understand their core challenges and are better positioned to produce solutions that get the job done.

For most startups the QA stage is likely the one where involving your customers will yield the greatest ROI, which is why I am suggesting “outsourcing” QA to your customers.  Do not get me wrong, I am not saying that your customers should solely handle your entire QA effort; it would be a recipe for disaster. Neither do I view testing and QA as being the same. Testing is ensuring the code works. QA is making sure the code works for the end-user.  As a startup you should have most of your testing protocols automated and running in conjunction with your builds.

QA, on the other hand, will be the time consuming, resource hungry item, which is why getting some of your customers involved in the process will put you ahead.  Here are some quick and simple steps on how to make it happen:

  • Set the right expectations with your QA customers:  Be upfront with your customers about what you the time commitment and level of feedback needed.
  • Collect metrics at all stage:  Try to quantify everything and have before/after metrics available to assess if your planned features are really hitting their targets.
  • Use Gamification: Gamification techniques are fantastic tools to build a better customer, we’ll post more about gamification in the near future but let’s say that if you are not building gamifications techniques that help QA the software, you are missing on great opportunities to build an more effective product.

HTML5, Agile practices, and to some extent the Hollywoodization of the startup world, give the impression that success is just a matter of coding an idea and putting it up on an app store with a bit of SEO behind it. The hard truth is that effective apps are the ones that win and 97.63% of the time, an effective app is the result of an effective QA cycle.  Getting some of your paying customers involved in QA’ing your site will help you build software that succeeds.