If you’re into agile, you’ve probably encountered this question: If iterating over user feedback is so fundamentally important, how come Apple has repeated success with their apparent one-off, non-iterative, grandiose launches of new products?
Is this the place for user feedback? (I told you once)
Agile skeptics love this argument. To them it’s a once for all win over advocates of frequent releases. They probably feel like Michael Palin in his surreal Monthy Python sketch with John Cleese, where they enthusiastically have an argument about whether they have an argument. Desperately fighting and imminent defeat after failing to stop arguing when “time’s up”, Cleese responds “I might be arguing on my spare time”, to Palin’s pointing-in-your-face-gotcha-finger.
“No you’re not”. “Yes I am”. “No you’re not”. “Yes I am” …
Similar surreal arguments happen in software companies. There are always someone who doesn’t want to go into production in short, regular intervals, and there are always someone who wants it.
So what’s Apple doing? Iterating on their spare time? Maybe the rest of us should look to Apple and grant the skeptics right. Stop nagging ourselves with those tedious deployments, not to mention the even more tedious user feedback. We’re talented developers, trained professionals, experienced craftsmen – let’s go with our vision and gut feelings just like Jobs did!
Both within the agile community and other places I encounter this misunderstanding disturbingly frequently. Moving on from the triumphant Apple argument, the skeptics continue: Users lack fantasy. They don’t know what they want, and will ask for solutions within a narrow mindset framed by whatever it is they use today. Henry Ford nailed it forever with his famous quote: “If I had asked people what they wanted, they would have said faster horses.”
This is where I think things get tricky: In a sense the skeptics are both right and wrong. It’s true that users lack imagination and even lie some times. Hence, they are right that we shouldn’t ask users what they want. They are however wrong about not frequently releasing to production.
The fact that users don’t know what they want is no excuse not to iterate.
We should still interact with our users – and we should use our software as medium. First and foremost we must have empathy with our users. We need to fundamentally understand them, better than they understand themselves. If we do ask users questions, we don’t ask about features. We ask how they feel when logging into their computer. We ask what’s their favorite software, and why. We ask why they are using our product.
When iterating cleverly we’re somewhere in the middle. (Comic: Calvin & Hobbes)
From this insight we might get tons of ideas for features. We verify them one at a time, quickly. We refine them and verify the refinements in a tight build-measure-learn loop: Cut a feature into the thinnest slice possible, and deploy it. Monitor what happens. Deploy two different versions of the same feature and monitor what happens. Pull a feature which is hardly used and see what happens.
What’s interesting about this approach, is that neither the overacting agile crowd, nor the agile skeptics will agree with you. To them you’re either not listening to your users, or you’re letting them dictate a product without direction. (Thanks to Kent Beck for this perspective).
From Eric Ries book The Lean Startup: We really did have customers in those early days – true visionary early adopters – and we often talked to them and asked for their feedback. But we emphatically did not do what they said. We viewed their input as only one source of information about our product and overall vision. In fact, we were much more likely to run experiments on our customers than we were to cater to their whims.
In the Scrum world there is a lot of discussion about a Definition of Done. Jeff Sutherland has a long list of steps that need to be carried out before a task is done. To me he’s missing the most important part: That you’re not done with a feature until you’ve analyzed the behavior of paying customers using it.
Focus groups, user dialog, staged usability tests in artificial environments and situations can all be sources of insight, but they can also be deceiving. The interesting part is not what the users say – it’s what they do, on a daily basis: The Proof of the Pudding is in the production usage data. Hence we should develop a way to quickly try out features, gather usage statistics, opinions and other knowledge. It might also be a good idea to recap statistics classes from the university days, make A/B-testing easy, and if your product is in the public, use sites like usertesting.com.
I don’t believe Apple isn’t iterating. Having hardware products they cannot iterate like a software company. Having a marketing strategy which builds up hype around spectacular launches calls for subtlety. But you don’t really think they came up with all those products without seeking insight into their users?
Arguments shouldn’t be about whether we release frequently or not. It should be about how we do it. So if you haven’t already, I encourage you to start a discussion with your team about sources of insight into users, tools for gathering such insight and what iterative strategy would support this. You may also want to reiterate how your organization is set up.
You will be innovating before you know it. @hauge2
