Research without Resources
I was hired as the only designer on the marketing team of a small luxury skincare brand. An esthetician reported to the marketing team that customers were complaining that the online quiz determining skin issues was not giving them accurate results. The quiz was intended to diagnose users with one of our nine skin types, which would point them towards the line of products best suited for their skin. Our marketing consultant suggested that we reword the questions while our CEO theorized that users were not educated enough to answer the questions correctly. I asked for the space to research the reasons why our users were not getting accurate results.
“Maybe we can reword some of the questions.”
“They’re uneducated about skincare. If they read the blog before taking the quiz there results wouldn’t be wrong.”
Key Constraints
New to Research
The company had never done user research or even market research before. They had never had a dedicated designer before. I was working in new territory with a very limited timeline and budget.
Limited Time & Budget
The marketing consultant we worked with was serving as the marketing director at my company. She had worked with UX Designers in the past and she believed in the importance of the research in the design process. However, she was contracted for 3 months and would be leaving the company soon and we would loose that expertise and support. The CEO was unsure of the value of exploring the problem before making some quick changes to the visual design. She decided that my research would be something I could work on after finishing my regular marketing and graphic design work and that the project didn’t need any dedicated funding.
Limited User Access
Another key constraint was a lack of access to users. The CEO did not want me to contact any customers or site visitors. This stipulation combined with the lack of budget meant that I was creative with my research methods.
Collecting Data
Spreadsheet of customer comments organized by concern
Customer Service
I began by searching our previous customer service tickets and emails and finding comments about the quiz. I knew that these customer service complaints were not wholly representative of our users experiences but they did tend to cluster around a few clear pain points. Users were often dissatisfied with their results, confused by the why we asked them to describe their skin, and in search of a solution to a specific problem that we didn’t provide answers for in the quiz.
“It put me in a Skin Type of 4 which after looking at
what constitutes a 4, it doesn’t sound like my skin”
After receiving a customer comment, our customer service approach was to address these complaints by conducting an in-person or virtual consultation with our esthetician. I interviewed our staff esthetician about how she determines skin issues during in-person consults with customers. The issues she looks for and the products she recommends are the same as the ones that the online quiz was supposed to identify and recommend. I compared the questions she asks to those in the quiz and found that they were completely different. She had noticed that many of her clients came to their consultation dissatisfied with their quiz results but enjoyed their conversation with her and purchased products after getting her diagnoses.
“A lot of people think they’re two skin types. They say “I’m a 5 and a 7” or “I’m between 4 and 6” But you can’t be two skin types.”
Consultation pamphlet describing skin type 3
Skincare diagnostics sheet for face to face consultations
Moderated Usability Testing
After learning more about what our customers where frustrated with in the quiz, I wanted to observe users taking the quiz. I knew from my own experience with the quiz that there were some clear usability issues but I wanted to see how others approached these problems and if they saw other issues that I had not discover yet. I also hoped that quotes from many different sources would help me in convincing our CEO of some of the problems I was beginning to see with the quiz. I was not allowed to discuss problems with the quiz with anyone outside of the company so I opted observed company employees from various departments taking the quiz. Most of these people had taken the quiz before, most were very educated about skincare, and all of them had trouble understanding both the interface of the quiz and the wording of the questions. I interviewed each of my users about their experiences taking the quiz. I observed users taking the quiz and asked them to talk through their actions thoughts and feelings as they took it. Some of their comments mirror those that I had seen in customer service tickets, such as inaccurate results and confusing wording. However they also noted something that no one had specifically written in about. The wording of some questions felt judgmental and/or discriminatory against people of color. I anatomized the comments and reactions I received from my fellow employees and raised this major concern to my CEO’s attention.
“What does this have to do with my skin type? The answer to that question wouldn’t change my skin type.”
Competitor Comparison
At this point, our CEO asked to see a comparison of what competitors in the industry who had online quizzes were doing so we could better understand our reputation and what the industry expectations were. I collected examples of competitor quizzes and identified their word choice and branding style and the user interface and logical scoring patterns that they were following. I push the CEO to make changes in the branding and tone across the site and I focused my efforts on the tone and accuracy of the quiz.
Original quiz question
Competitor quiz question-asks about skin conditions, never race.
During this time I also asked to see the original logic that was used to make the current quiz. After the original logic behind the quiz was found and the issues with people getting inaccurate the results became very clear. Because of the way the branching logic was drawn, out of the nine skin types a person could have only eight were even possible results and the quiz was weighted heavily towards just two of them. This explained a whole host of problems we were having including the complaints we were receiving from customers who didn’t fit those two highly weighted skin types.
Problem Validated
The research I had done so far validated that parts of the quiz were giving users inaccurate results and negative experiences. I saw that the in-person consults were more effective in determining the customers skin types. So, I made a quick sketch of what the logic of the quiz would look like if it followed the structure the estheticians were following in their in-person consults. From there, I tried to redesign the branching logic of the quiz to mimic the process that in-person consults followed. Through the design process I found that the questions and possible results of the quiz were too complex for a branching logic structure to follow. I determined that we would need to design a quiz with scoring logic. Designing and testing new logic would take time and we would need a developer and the money to hire one.
Presenting to Stakeholders
With this knowledge, I presented my findings to the CEO and marketing team. I presented the UI patterns I had found in competitor quizzes, showed the quotes and findings from our team and the customer service tickets, and explained the discrepancy between the in-person consults and the online quiz’s logic. I detailed how our desire for accurate results required us to create a new logic structure for the quiz.
I recommended that we proceed in designing and testing a new accurate quiz logic but that we also proceed with some basic edits to the current quiz. The wording and CSS of the quiz was something I could change quickly so I pushed that we make some quick edits for basic usability and inclusivity. We could not immediately make an accurate quiz but we could solve for some of our users’ pain points.
Presented quotes from customers
Presented research implications
One of our competitors’ quizzes
Next Steps
My proposal was accepted and I was able to design a new quiz. We were able to move forward with the research I had done but, if I could have gone deeper with this project I would have preferred to interview users outside of the company. I also wish I had been able to work on a design team with other collaborator during this process. That said, I had the opportunity to work with many non-designers throughout my company and I learned how to talk about and present design research with team members in different roles. I also learned how to balance my on-going projects while pushing for something new and how to ask for buy-in to a project along the way.