How to prepare an effective end-user research study analysis in 10 easy steps

Analyzing an end-user research study can be a daunting task. There is a mountain of data collected and the researcher may not know how to begin organizing his/her findings. The following is a solid presentation framework that should help bring order to the chaos:

1. Restate project objectives - Every project-specific deliverable which builds upon a body of knowledge to support strategic and tactical recommendations should restate core project objectives. Deliverables generated as a result of project activities should clearly and continuously illustrate how project objectives are being met.

2. Define research objectives - Project objectives are business-focused. Research objectives are typically aimed at understanding behaviors, trends, and information-seeking agendas of end-users relevant to stated project objectives. Research objectives should not be generic. They should be aimed at identifying ideal end-user behaviors that would likely achieve project objectives.

3. Summarize key findings from previous research studies - Prior to spending a client's money engaging in additional research, the researcher should seek to understand and summarize relevant research studies that have already been conducted. Identify gaps in existing knowledge about the end-user segments, and illustrate how the current research completes the total portrait of the end-user.

4. Describe research methodology - It is best to assume that the client or audience for the research presentation is not an expert in behavioral research methods. Describe, in detail, how end-user segments were recruited, ways in which behavioral research differ from market research (emphasis on sample size), and the manner in which the interviews or contextual inquiry was conducted.

5. Elevate high-level research findings and trends - Trends will likely emerge from the research that cut across all end-user segments. It is best to elevate these important trends early on in the discussion. Additionally, it is best to introduce early recommendations to set the expectation that each finding in the research presentation will be accompanied by actionable recommendations. EVERY finding should be actionable. Clients need to understand the value of end-user research and IA researchers are responsible for emphasizing the business value that is an outcome of this type of research study.

6. Introduce behavioral personas that emerged as a result of the research - After high-level findings are discussed, the researcher can introduce the personas that were formed based on the major differences in behavioral trends observed. Discuss the behaviors that divided the personas first, then introduce each persona. When introducing each persona, be sure to include a fictitious name, photo, description, motivations, relevant key tasks, and barriers to completing key tasks. Support major persona traits with actual video clips, photographs, and end-user quotes from the research. Key tasks are better presented visually, as task flows, rather than as a list of bullets.

7. Aggregate key tasks from personas into an end-user mental model - A mental model is an important first step in designing an intuitive information architecture that facilitates the completion of key, end-user tasks. The researcher has already identified the key tasks from each persona. It is a simple matter to group these together and assess gaps in existing content needed to support each key task.

8. Develop a list of recommendations to fill gaps in content - Begin to tackle each gap illustrated in the mental model by suggesting a tactic or recommendation for content that would satisfy key end-user tasks.

9. Assign level of effort (L.O.E.), value to the business, and value to the end-user to each tactical recommendation - Documenting recommendations gives the client a vision of the overall solution. Assessing level of effort and value of each tactic helps the client prioritize each component of the solution and put tactics on a road map for implementation.

10. Define next steps - This presentation is a component of a larger set of activities that is meant to accomplish specific project objectives. Conclude the research presentation by connecting the research to its impact on project objectives as well as introducing the next activity in the project plan.

A quick look at 5 end-user research techniques

1. Contextual Inquiry

Summary - Contextual inquiry is an ethnographic research technique that involves the facilitator observing and interviewing participants in a setting where the participant naturally performs relevant behaviors to the subject of the research. Contextual inquiry is best utilized to uncover behaviors and information seeking agendas prior to designing a system, application, or interface.

Pros - Participants feel more at ease when performing relevant tasks. Task recall is easier in the context of the surroundings where relevant tasks naturally occur. The researcher is able to observe environmental factors, constraints, and technologies used. The researcher is also able to observe actual documents and information that is kept close at-hand, when actions and behaviors are performed.

Cons - Travel is sometimes required to interview locations, and this may be expensive. Additionally, unexpected environmental issues may prevent the optimal recording of audio or video. Contextual inquiry can also be dangerous, especially when participants are recruited from Craigslist. There is some risk associated with this type of field research. Don't perform this research by yourself. Always travel in pairs!

2. Lab-Based Usability Testing

Summary - Usability testing is an analysis or validation of a design in which success or failure is measured according to how easy it is for participants to perform key tasks using the design. Designs can be low fidelity (e.g. - a wireframe) or high-fidelity (a visual design comp) and testing can be performed on paper, on electronic files, or on functional prototypes.

Pros - The laboratory provides a measure of control in which unexpected changes in the environment and the testing equipment can be eliminated or minimized.

Cons -If the usability test includes a computer, there will always be differences in the set-up of the lab-based computer and the user's own computer. Additionally, the laboratory may be an unsettling environment for the participant. In general, participants feel more at ease in their own homes or places of work, as well as using their own computers.

3. Focus Groups

Summary - Focus groups are interviews with a group of participants in a laboratory-based setting (or a conference room). In a focus group, attitudes and perceptions of a group of people can be collected during the course of a moderated discussion.

Pros - A focus group allows a larger number of opinions to be collected at one time vs. 1-on-1 interviews with participants. This type of research may be more convenient for clients to observe. Focus groups are great for brainstorming ideas because individuals within the group tend to feed off of the ideas of other individuals in the group.

Cons - Focus groups are not great for understanding end-user behavior. Group dynamics play a role in the responses of each individual in the focus group. There will generally be one or more vocal participants who will influence the responses of the other participants in the room. The reverse may also be true. Some participants may rarely speak at all. Therefore, it is difficult to obtain the same level of unbiased information from each individual in the group.

4. Surveys

Summary - Surveys are used to collect feedback from a large number of respondents. Surveys may consist of closed or open-ended questions, or they may be a mixture of both.

Pros - Surveys are an ideal research tactic when a statistically significant number of responses to specific questions are needed. They can be deployed via email or from a 3rd party, hosted platform. Survey tools are generally inexpensive and generally include reporting and charting capabilities from collected responses.

Cons - Survey research is not moderated, so a researcher has no opportunity to probe responses to questions with follow-up questions. Additionally, although closed-ended responses are easier to generate reports from, they are limited in the richness of data that open-ended responses provide. A researcher learns only what the limitation of the tool allows him or her to learn.

5. Card Sorting

Summary - Card sorting exercises are typically, but not always, conducted as a 1-on-1 activity. The goal of a card sorting exercise is to understand end-user information organizational models. These models typically inform end-user taxonomies and information categorization schemes.

Pros - Tools exist to allow researchers to conduct a larger number of card sorting exercises online, similar to deploying a survey. The benefit of using an online card sorting tool is the ability to generate reports and quickly assess the most ideal taxonomy from analysis of all of the card sorts conducted.

Cons - Card sorting is difficult to explain to end-users. Therefore, deploying card sorting exercises without a face-to-face explanation of how to take the test properly, may result in a high-rate of abandonment. Finally, taxonomy development is a professional discipline. Taxonomists and information architects should not rely solely on untrained, end-users when constructing a logical categorization scheme for a system.

5 Steps to delivering an effective presentation

There's a client presentation scheduled for next week and your PowerPoint deck feels a little loose. Additionally, there's a 150 page wireframe deck that has to be reviewed as part of the agenda. By the way, there is no formal agenda. Is this a problem waiting to happen? Most definitely. Here are 5 steps to course-correcting this situation:

1. Identify goals for the presentation - Goals and objectives should be the starting point for any presentation. Being able to identify the most beneficial outcome of any situation is vital to understanding the road to get there. Delivering an effective and persuasive presentation absolutely requires this important first step. A good presentation is born from an understanding of its purpose. Visualize the desired outcome of the presentation, document this vision, and proceed to step number 2.

2. Identify the barriers to achieving the stated objectives - Identification and documentation of the presentation objectives will force the presenter to think about the barriers in the path to achieving these objectives.

3. Develop a narrative outline with the construct of: "objectives > challenges > solutions > measurement of solutions" for the presentation - All effective presentations do exactly the same thing. They tell an organized story that illustrates the impact of recommendations to what matters most to the business. If you've completed steps 1 and 2, you already know what story it is that you are going to tell (and have the chapters documented as well). A good narrative could begin like this:

I. Introduction - Achieve measurable business objectives by improving the User Experience on platform X.

A. Opportunity 1 - Achieve Business Objective X
1. Summary of Current Challenges
2. Strategic and Tactical Recommendations
3. Analyze the Success of Recommendations

B. Opportunity 2 - Achieve Business Objective Y
1. Summary of Current Challenges
2. Strategic and Tactical Recommendations
3. Analyze the Success of Recommendations

C. Opportunity 3 - Achieve Business Objective Z
1. Summary of Current Challenges
2. Strategic and Tactical Recommendations
3. Analyze the Success of Recommendations

D. A Prioritized Road Map to Implement Recommendations

E. Next Steps

4. Complete the presentation by not veering off of the course of the outline - Now that the narrative outline has been constructed for the presentation, create the rest of the slides that support each chapter of the outline. Use stakeholder notes and data to define business objectives and current challenges. Use internal team and user research data to define strategic and tactical recommendations. Use business defined key performance indicators and a measurement framework to define how you will measure the success of proposed recommendations.

5. Show don't tell - Each slide of the presentation should be an illustration of a concept or recommendation. Objectives, barriers, and strategic recommendations can be summarized and visualized as information graphics. Tactical recommendations related to the user experience can be visualized through personas and interaction design. Visuals have more impact and emotional appeal than text.

5 Trends in media consumption

Within the last five years, many industries have had to evolve due to advances in technology as well as increased social connectedness. Here are 10 examples of how these changes have affected media consumption:

1. Media is increasingly less physical - Music is a perfect example of an industry where technology is constantly morphing the format and delivery of the media. Analog records gave way to cassette tapes. Cassettes moved aside for Compact Discs. Within the last five years or so, MP3 files erased the need for Compact Discs. The gaming industry is also undergoing the same changes. Physical games will slowly be replaced by digital downloads of video games from digital stores.

2. Media has become portable - Continuing with the music industry example, the reason for the constant change in the format of the media is likely due to the consumer demand for portability. The Sony Walkman introduced the concept of taking music with you. That trend has continued to evolve with mp3 players, iPods, and now, smart phones. The demand for portability has forced technology to shrink the size of media formats so that the consumer can travel with his/her media.

3. Devices are connected to the cloud - Portability underwent a dramatic shift when media playing devices were able to wirelessly connect to the cloud. Not only did physical formats become digital files (eg - CD to mp3), but digital files are now becoming obsolete as well! There is no longer a need for storage of digital files when digital media can be stored in the cloud. Streaming media has been around for years, but the prevalence of wireless access will surely bring this method of media delivery into its maturity.

4. Consumers are hyper-social - MySpace, Facebook, and Twitter have connected people and ignited conversations in unprecedented ways. With this "hyper-connectedness" comes sharing of information, recommendations, and multimedia. New applications such as Pandora, Miso, Tunerfish and GetGlue, all enable end-users to broadcast and recommend multimedia content to peers of their social graph. As devices are increasingly connected to the cloud, it is easy to see how social discovery of content will affect multimedia content consumption overall. Entertainment industries such as Music, TV, and Video Gaming are beginning to understand the importance of connecting to existing social networks to generate content recommendations to consumers.

5. Fidelity and portability are no longer mutually exclusive - A few years ago, the world was introduced to High Definition Television. Back then, HDTV existed in a world of 42 to 60 inch flat panel televisions that were anchored onto the walls of media enthusiasts' living rooms. Again, the demand for portability of media has made HD and near-HD quality viewing experiences possible on hand-held monitors. Personal viewers in the form of eyewear monitors can now deliver HD and even 3D video content while immersing the viewer in a theater-like environment, even when he/she is on an airplane.

10 Tips for Conducting Better User Research

When considering enhancements, optimizations, or planning new content development, a business should aim to better understand the informational needs, tasks, and behaviors of its consumers across all digital channels. Here are 10 tips to keep in mind when designing a behavioral research study:

1. Recruit active information-seekers - Market research panels have databases of willing test participants that match demographic profiles, however, these participants are likely "professional test takers." When conducting research on a given subject, it is better to recruit subjects in the act of actively seeking information related to a relevant topic to the research study. This type of recruitment can be accomplished by launching a Google cost-per-click advertisement that solicits participation and is triggered when relevant keywords queries to the research topic are used. A list of customers who have opted into receiving email communications is also a great platform for recruiting participants to a research study.

2. Analyze existing consumer research - Nothing is worse than spending money on research that yields little to no useful results. Look to the marketing or the customer service department of a business to understand what is already known about target consumer segments. Identify gaps in knowledge about consumers, or seek to update outdated research.

3. Formulate hypotheses - Business stakeholders will volunteer hypotheses about their target customer segments. Secondary research may also suggest certain motivations, tasks, and behaviors. Designing a study with some informed hypotheses may help make lines of questioning more relevant from the get-go.

4. Develop a comprehensive list of questions - It is always a good policy to be over-prepared when conducting an interview. A research study is often commissioned by a client paying for the time the researcher is spending with the end-user. Think through and document every possible line-of-questioning and probing follow-up question. The study guide isn't a script but it should assist the research facilitator in case a line of questioning reaches a dead-end. The idea is to keep the subject talking about the subject and maintaining a calm and natural atmosphere.

5. Ask open-ended questions - Questions that naturally result in a "yes" or "no" response do not lend themselves to a dialogue. Ask questions that require subjects to be descriptive and engage in a conversation.

6. Do not ask participants to speculate - The goal of behavioral research is to understand and observe behavior. Asking a participant what they might do in a given situation or what they think they might want on a web site leads them to speculate on their behavior. Speculation leads to false and biased data.

7. Perform research in context - Behavior is best observed in the context of where it naturally occurs. Bring participants into a conference room only as a last resort. It is better to observe participants' natural behavior wherever they come into contact with relevant brands or perform actions relevant to brands. Empathy Lab believes better data is gathered when interviews with participants are conducted in homes, stores, and offices. Performing a study in the context of which a task is normally performed aids end-user recall of the task in question.

8. Videotape, record audio, and/or photograph participants as well as their environment - It may feel awkward for participants at first, but nothing proves a point like a video clip of participants performing relevant tasks. Use an external microphone or a video camera with great audio capture so that the audio is clear when making clips for a highlight reel. Additionally, researchers must obtain participants' consent prior to videotaping them.

9. Bring a research assistant - Facilitating a research study requires concentration and focus. The interviewer must make sure that all important lines-of-questioning are brought up in the interview, the conversation is natural, and questions are not "leading" the participant to a biased result. Therefore, the facilitator should not be the one taking notes or handling the technical equipment (such as a video camera or audio recording device).

10. Have a designer, copywriter, or strategist assist with the research - True collaboration in Design requires the Information Architect and Visual Designer to both understand and empathize with end-user needs. Rather than relay this information via presentation of findings, it is advantageous to have the collaborating partner accompany the researcher when the research is conducted. Similarly, a project may benefit from having a strategist or copywriter accompany the Information Architect in the research study. The right research partner depends on the type of project for which the research is being conducted.

:: How to Increase Utilization of an Intranet

It's not difficult to figure out how to increase the utilization of a corporate intranet. It's simply a matter of understanding the daily workflow of its intended end-users.

An important point to keep in mind, is that employees don't go to an intranet solely for the purpose of reading corporate press releases...they go to perform useful tasks. Consultants simply need to ask employees what they do on a day-to-day basis, illustrate routines as end-user workflow diagrams, and create opportunities in taxonomy, design and architecture that facilitate employees' primary tasks.

Information Architects use a form of behavioral research called "contextual inquiry" to better understand the daily work routines of corporate end-users. Contextual inquiry involves interviewing end-users in their place of work. That doesn't mean a conference room...it means at their desks, in their cubicles, and in their offices.

What are some benefits of contextual inquiry?

1. Contextual inquiry enables IAs to take notice of their end-users' files, folders, and email "inboxes," in order to document trends in information organization (ultimately informing intranet taxonomies).


2. Participants feel more at ease, and are able to better recall daily work routines, when they are interviewed in familiar work environments. Usability labs and conference rooms often make participants nervous and disoriented, causing them to speculate rather than recall specific information and tasks.


3. Participants can walk their researchers through previous intranet and web interactions, exposing issues and potential opportunities.

Luckily, corporate intranets are intended to satisfy specific needs. That makes them easily engineered to achieve those needs. Whether it is facilitating primary, work-related tasks, educating and training staff, or providing vital documents, engineering a better user experience on a corporate intranet can be achieved through primary research.

Jonathan Lupo - VP / Information Architecture - Empathy Lab