UX Strategy, page 19
However, if it’s not that obvious, the analysis phase involves synthesizing all the feedback from the interviews to determine your team’s next action. Did the guerrilla user research validate or invalidate your assumptions? Was the experiment a failure because the execution was sloppy? Or, did something unforeseen occur such as in the case of the treatment center operation, in which we learned the business model did not work? Your goal is to use the analysis as a decision point to pivot in a different direction or double-down on further experiments that actualize the value proposition.
At the bottom of the template is a row called Validated Learnings. This is where you can put your high-level findings for each participant and track which participants either validated or invalidated your solution hypothesis, as shown in Figure 8-9.
Figure 8-9. Validated Learnings sample cell
To do the analysis, you need to take a step back as you did in Chapter 5. Zoom out from all the details that you were just bombarded with and think carefully about the big picture. You might find yourself doing one of these things:
Assess to determine if the correct customer segment was reached, by looking at your provisional personas or initial customer discovery research. If it was not the correct customer segment, begin making new assumptions about the right one.
Assess if the problem that your product is trying to solve is an actual problem based on the feedback that you heard. Was it a small problem or a big problem?
Assess if the solution that you showed was on target. If people were not truly excited about the value proposed in the solution prototype, think through possible ways to improve it.
If the value proposition was validated, congratulations! However, don’t stop there. Determine if there are any easy fixes that can be made to improve the user experience.
If the value proposition was not validated, assess why immediately. Was it because you had the wrong customer, problem, and or solution? Is it fixable? How can you change the product or user experience?
Listen to the signal. If the experiment was an abysmal failure, resolve to take a break while you reset your entrepreneurial or intrapreneurial clock.
If you have a client or stakeholder who does not believe in your research and wants to build the product regardless, you face an existential challenge wherein you are trying to balance your principles and your pocket book. Only you (and your spouse) can answer this one.
Enter your insights, answers, and conclusions into a spreadsheet in the UX Strategy Toolkit at the bottom of each participant’s column quickly. You want to make the turnaround on your team’s decisions fast.
You are now at the end of your guerrilla user research, and you are once again at a crossroads:
You invalidated your value proposition. If you are wrong about your original customer segment, go back to Chapter 3 (customer discovery).
You invalidated your value proposition. If you are wrong about your solution, go back to Chapter 4, Chapter 5, Chapter 6, and Chapter 7
You validated that you have product/market fit. Go build a functional MVP and move on to Chapter 9
Recap
Conducting guerrilla user research (especially with stakeholders present) can feel intimidating at first, but the more you do it, the less scary it becomes. It has the benefits of immediacy and transparency. Plus, your entire team and the stakeholder will be better off learning sooner rather than later if your solution works. Plus, you do it as a team; everyone is equally invested in the outcome. They have the benefit of really seeing how users will experience the product.
* * *
[56] Klein, Laura. UX For Lean Startups. O’Reilly Media, 2013.
[57] Buley, Lean. The User Experience Team of One. Rosenfeld Media, 2013.
[58] http://www.nytimes.com/2004/05/06/technology/for-technology-no-small-world-after-all.html
[59] Rudorff, Raymond. War to the Death: The Siege of Saragossa. Hamilton, 1974.
[60] http://www.nngroup.com/articles/how-many-test-users/
[61] Maurya, Ash. Running Lean: Iterate from Plan A to a Plan That Works. O’Reilly Media, 2012.
[62] Portigal, Steve. Interviewing Users, Rosenfeld Media, 2013.
Chapter 9. Designing for Conversion
You can’t play in the man’s game, you can’t close them—go home and tell your wife your troubles. Because only one thing counts in this life: Get them to sign on the line which is dotted. A-B-C. A-Always, B-Be, C-Closing.
—BLAKE FROM THE DAVID MAMET SCREENPLAY, GLENGARRY GLEN ROSS
IF YOU WANT TO BE A CLOSER, YOU MUST CONSTANTLY TWEAK YOUR UX strategy to increase successful outcomes for user engagement and customer acquisition. You need to design efficient funnels that do everything from engage first-time visitors to eventually convert them to repeat customers. Customers include anybody and everybody that you need to engage for your value proposition and business model to work. And yes, users who don’t pay for your service are customers, too. As people begin entering from the top of the funnel, you must immediately track and measure all the crucial data points along the way so that you can validate your product’s success.
This is known as designing for conversion or optimizing your product. The process ties together all the tenets, as depicted in Figure 9-1. In this chapter, I’ll demonstrate how successful UX strategy uses analytics to optimize the UX design from the user’s first impression of the value prop to the user becoming happily addicted to the product. A tool called the Funnel Matrix will demonstrate how to align your entire team on different stages of customer acquisition by identifying measurable metrics for them to take action on to pull the customer into a deeper level of engagement.
Figure 9-1. The four tenets of UX strategy all need to be served up right now
Seeding Growth Hackers
Growth hacking is a term coined in 2010 by Sean Ellis, a marketing blogger and entrepreneur.[63] The concept behind it is for product teams to come up with extremely clever, cost-efficient ways to increase customer growth. Facebook, Twitter, LinkedIn, Airbnb, and Dropbox are companies that have all used growth hacking techniques to become successful. Hardcore growth hackers are a crossbreed of marketers, coders, and analytics experts. They are masters of analytics tools, traffic generation, and product optimization with a deep understanding of the innards of search-engine optimization (SEO), ad platforms, and social media tools. They are called hackers because they are ruthlessly focused on growing the business by any means necessary. They push the limits of traditional marketing using techniques such as A/B tests, landing pages, viral factors, email deliverability, and social media integration. The goal of growth hacking is to tie viral and paid ad campaigns to user engagement metrics so that you can identify the most valuable marketing channels. Growth hacking entails continuous tinkering with the product’s sales funnel so that it is fully optimized for acquiring new users and getting them more deeply engaged.
In the case of TradeYa, our core team knew we were out of our depth. We could design, strategize, and develop the MVP. We could fill out the Funnel Matrix—we knew what kind of answers we needed. What we didn’t have was the expertise to get those answers from the metrics and analytics reports in front of us. We didn’t know how all the tools and dashboards worked, but we had to run hardcore tests to refine and perfect our MVP to relaunch in 30 days. It was the Christmas holidays, and we were pretty sure it was a fantasy to hope for a highly advanced growth hacker with usability testing and design skills to end up at our door. Plus, we had a budget of only about $5,000. So, we decided to hack together some growth hackers of our own.
That’s how the TradeYa MVP Apprentice Program was born. Jared and I wanted to diversify our budget with as many fresh minds as possible who would work collaboratively on connecting all the analytical tools we needed to update the existing MVP. They would use the Funnel Matrix to plug in the feedback data from each other of their initiatives so that we could validate all the assumptions the core team had made about the levels of user engagement. Figure 9-2 shows the blog post we used to attract the talent we needed.
Figure 9-2. Help wanted: experimenters for testing the TradeYa MVP
Within 72 hours of JLR Interactive’s “Help Wanted: Lean UX Apprentices” post blasting through the Twitterverse and LinkedIn, all 8 spots were filled with qualified individuals, hand-picked for this challenge. They came from all over Los Angeles and included professionals with advanced skills in architecture, marketing, engineering, and psychology, with degrees from MIT, Cornell, NYU, and UCLA. Although some of the apprentices already had an understanding of UX design and the Lean Startup approach, the intent was to take advantage of the cross-pollinating catalyst effect of their diverse expertise and backgrounds from outside of UX by throwing them into the deep end of hands-on MVP testing and optimization.
We were really lucky. Our apprentices were eager and focused when they met for the first time in an intensive workshop on January 2, 2013. During a whirlwind three hours, the eight apprentices were brought up to speed (see Figure 9-3) with the history, philosophy, and MVP of TradeYa. First they learned about the principles of conversion-centric design for customer acquisition. Then we discussed how they would stay aligned through collaboration and task delegation using a cloud-based tool that I devised called The Funnel Matrix.
Figure 9-3. Jaime (on left) and Jared (on right) talking about funnels while instructing the eight TradeYa Lean UX apprentices
TradeYa’s testing phase ran from January 1, 2013 to February 28, 2013. Over the course of the first 30 days, the apprentices learned about analytics tools, set up those tools for TradeYa, and then acted as users themselves—trading and bidding on one another’s items—to test the product’s transaction funnel. They would then verify whether the tools were capturing the key metrics accurately. Any changes in hypotheses were also continually updated in the Funnel Matrix, which the entire team could view and track. For our measuring tools, we ended up using Google Analytics and KISSmetrics—the development team added JavaScript code to all HTML pages.
Jared and I closely watched and guided the development to ensure that we got the data points we needed. This way, when we launched the MVP, we would be able to track all the desired actions required for users to successfully trade.
Lessons Learned
To design for conversion, you really need to bring together a cross-functional team including designers, developers, product managers, and marketers. The marketing and sales team creates a list of initiatives and must be able to act quickly on it. The development team also has a list of metrics that they need to track and measure.
The output of the Funnel Matrix technique directly informs the user flow, feature list, and a short list of wireframes to be designed or redesigned that are optimized for a desired user action.
The data and metrics help with every stage in the sales funnel to make continual improvements and give insights into your customers’ actions.
Using the Funnel Matrix Tool
A funnel is a cone-shaped utensil with a tube at the apex for conducting liquid or another substance through a small opening. When I put oil into my car’s engine, I use a funnel to increase the success rate that oil will make it directly to where it’s supposed to go. The funnel is the mechanism I use to avoid waste.
In the ecommerce world, waste happens when potential customers aren’t funneled into the engine of a product. Somewhere along the way, the customers did not sign up, activate their accounts, initiate a transaction, complete a transaction, or some other reason. In other terms, the customer did not experience the value proposition correctly and might have clicked away without satisfying his needs. The design of the ecommerce or digital product funnel didn’t convert him into an engaged customer.
In the book The Entrepreneur’s Guide to Customer Development,[64] authors Brant Cooper and Patrick Vlaskovits use a Funnel Matrix to represent how a prospective customer goes “from Internet ‘Googler’ to satisfied user.” I was inspired to expand the concept and create my own tool for dialing in the metrics directly correlated to UX strategy.
The purpose of filling out the Funnel Matrix, which you can see in Figure 9-4, is to force the entire team involved with the product—stakeholders, product managers, marketers, visual designers, developers, and everybody else involved—to think through all the required actions potential users and customers must make as they progress down the funnel to become repeat users. The Funnel Matrix’s other purpose is to validate, measure, and learn ways to better optimize the user’s experience to increase conversion rates. After I began experimenting with, using, and tweaking this tool, I quickly discovered that it allowed me to be more empirical and less precious with my own UX strategy process.
Figure 9-4. Funnel Matrix tool empty template
The Funnel Matrix tool shows different stages of user engagement and acquisition as well as the criteria to evaluate for each stage. I created this Funnel Matrix using Google Spreadsheets so that my team could collaborate and work on it simultaneously. (A copy of the template is available in the UX Strategy Toolkit.)
Note that how you use the Funnel Matrix will depend on where you are in your product development cycle:
If you have an existing product or MVP like Jared had with TradeYa, your team can use the entire matrix to optimize it. Tweak and measure your metrics until you get it right.
If you are designing wireframes for your first functional MVP, use the Funnel Matrix to speculate on the levels of user engagement and the key metrics you might use. Then, you will test the top (suspect) level with something like a landing page test, which I’ll discuss later in the chapter.
If you are still in the conceptual part (storyboards, prototypes, and so on) of your UX strategy, take this as opportunity to get an overview of where you will end up when your team is ready to design for conversion. I’ll also teach you what to do when you’re ready to validate the top of your funnel.
Why a Matrix and Not a Map?
At this point, a few UX strategists might scratch their heads and wonder why I’m not advocating for a journey map (also called an experience map). A journey map looks like a flowchart and shows all touch points. It’s a visual representation of the user’s journey and interaction through the UX of your product. It is generally created from output following a collaborative brainstorming session with product stakeholders. Sometimes these maps are quite complex to decipher, especially if you were not a participant during the session. If you haven’t seen one of these maps, do a keyword search for them on Google Images.
When used effectively, a journey map can aid the UX strategist and stakeholders in envisioning cross-channel experiences through digital and nondigital touch points. Personally, I find that the map takes too darn long to create and lacks accountability because it is rarely benchmarked to the product’s reality after it begins to be released. Here’s what I’ve seen happen all too often: the key stakeholders, internal UX team, or both are called together for a consensus-building session. They break into groups, jot down ideas on sticky notes, and then post them on a wall or board. Everybody stands back while the most assertive participants reorganize the concepts into clusters. At the end of the session, someone takes photos of the lovely yellow papers. The designer turns these concept clusters into what often appear to be zany circuitous informational diagrams that would potentially give Edward Tufte[65] a stroke. The posters are then hung or stored somewhere in the office to hopefully influence employees on their way to the bathroom.
If you strive to be more empirical in your strategic process, you need to develop this approach and deliverable into a system of disciplined procedures that are gauged and updated throughout the product ideation phase and development cycle. That’s why I prefer using a cloud-based matrix as a centralized data depository. With this customizable tool, everybody on a team can work collaboratively and even from multiple locations. You can usually do the first pass in two to four hours, and afterward it is easily accessible for everyone to update. The output is easy to interpret, and, more important, the matrix can serve as a central data depository for metric reports after the product is released and iterated upon. When, in his book The Lean Startup,[66] Eric Ries talks about lean accounting with real-world metrics, this Funnel Matrix is how you can harness it for the UX. So please put your sticky notes away (at least, for now). Let’s begin to explore and fill out the Funnel Matrix tool. (For a different opinion on journey maps, check out what Experience Strategist veteran Holly North has to say about them in Chapter 10.)
Rocking the Funnel Matrix
The first time I tested the Funnel Matrix tool was on TradeYa with Jared. For months, we had been sifting through analytics reports that measured hundreds of user actions and trying to make big-picture decisions based on them. But we figured it would be more efficient to try to make smaller decisions based on validated data output from singular user actions. We needed to focus our time, energy, and money on being able to make meaningful changes to our MVP in order to answer questions coming in from our investors. Thus, Jared, the developers (who Skyped in from India), and I spent an intense four-hour period filling out the exercise.
