The project:
Find the marketable features for a target demographic.
The client wanted a list of marketable features that will ensure their app is a giant success in the app store.
The challenge:
This research project was very open ended, with minimal information about the product’s current or planned feature set. Our task was to find what the target demographic really needed in their lives, so that it may be built and marketed to them. Without an understanding of what the company has planned, the task was doubly difficult. Additionally, as the project progressed, the goal of the research was changed multiple times. Not radically different, but just enough to make us start our data harvesting and insight generation from scratch.
What we decided to do is this:
Customer has a need for a solution -> talk to customers -> Features that users want
What we wanted to do:
Ideally, once we interview users, we build mental models about how they behave. These mental models manifest themselves as hypotheses, that can be reinforced with numbers from a survey.
Unforeseen factors:
The client needed the project wrapped up in 4 weeks. By our assessment, it needed at least 6. We managed to compress our timelines anyway. But then we faced another hurdle.
The increase level of scrutiny over the process meant that we had to deliver not just the final result, but also formalize the intermediate steps into excel sheets, a format in which the research data can be consumed/reviewed by the client team.
Moreover, the client requested a sample of the results of a more extensive research study using a limited sample set we were going to gather from our research. This is a particularly odd request because there are virtually no statistically significant behavior insights that can come from any interview with such a small sample size. The only way out of this is to support the behavior patterns observed with data from a survey sent out to hundreds of people.Qualitative data = words = interviews
Quantitative data = numbers = survey
What does this mean?
Interviews tell you what people do and why they do it.
Surveys tell you how common is this behavior in a large population.
So we conducted our interviews.
We formed these mental models about how users behave. We established a few behavior patterns about what people do and how they do them, but we lacked the support of proper numbers to say for sure that this is the behavior in the market that we can target. Additional research by means of a survey will bring more confidence in our recommendations.
What went right
The entire team’s user interview skills got leveled up. (yay!)
Key things we learnt about interviewing people
- have a clear goal in mind about what you want to find out in the end
- it’s okay to go off script
- make sure you cover all the information you need to gather
- be non judgemental
- do not ask leading questions
- silence and pauses are okay
Our Deliverables:
- Day-by-day Project plan
- User Research script
- Survey
- Create transcripts from interviews
- Map transcript quotes to data required for the project
- Aggregate multiple data to generate insights
- Presentation of research insights
Processes developed:
- Basecamp project management process
- Extraction of data from interview transcript.
What can be done next time to make things better:
- Use a transcription service.
- Have a clear understanding of what we are trying to find out vs and what we have to deliver.
- Spend more time on planning the steps required to reach the final deliverable.
- Need a better process for identifying what bits of data are relevant to our project – do this by establishing the relationships between quotes, questions being answered, datapoints, and how they ultimately contribute to the final result.
- Restrict the number of things being researched at once. Or plan multiple studies to explore various behaviors.
- Establish timelines for turnaround of research, to better be able to negotiate with clients about timelines.
What I learnt:
- Knowing process is critical to negotiating timelines with clients. When we ourselves had no clear idea about how long each step of the project would take, we could not offer convincing arguments when the client compressed our timelines.
- Maintain clear documentation of what’s been agreed upon. There’s a surprising amount of wiggle room in interpreting past decisions, and having a clear goal thats visible to all the team members (yours and the clients’) ensures that moving of the goalpost mid-project is less likely to happen.
- Stand firm on your process. You are the designer. Your client is hiring you for your expertise. Clients will try to break your process. They will do it in the first meeting. They will do it during the project. I must be able to remind them that rushing things or skipping steps will affect the quality of the final result, and they agreed to give us the time to do what we do best, the way we do it. The way we do things has brought us success enough times that they have heard of us and hired us.