Internal Product Research – Amazon
Certain data points, names, and details have been altered or generalized to maintain confidentiality and respect internal privacy requirements.
Contextualizing the Problem
As part of the effort to understand and continually improve the user experience for employees, the research team launched the first UX Benchmark survey in February 2023. The purpose of this survey is to provide the product team with actionable data to inform design and development of their product's features and improvements for 2023 - 2024.
The goals of this survey are to:
- Better understand the current user experience.
- Establish a benchmark rating that serves as the baseline for UX improvements year over year (YoY).
- Identify the key areas of focus and prioritization for to improve the user experience.
Research Methodology
The standard Usability Metric for User Experience (UMUX) scoring using a 5-point Likert scale was used to create the benchmark score. These scores are converted using a 0 – 100 grading scale in order to assign the UX Grade. CSAT and interviews were considered to create the benchmark score, but UMUX was ultimately chosen due to its advantages for scaling and studying product experience.
User sentiment was also gathered to assess product satisfaction and impact. Participants were asked to explain why they provided the scores they did and identify both positive aspects and challenges when using the product. Additionally, participants were asked questions related to features, data, help and support, and potential barriers to performance. Product opportunities were identified by conducting quantitative analysis based off established research methods.
The survey request was displayed to current users via a link on the application site. Over 1300 usable responses were received, but because there is no estimate of the potential total user population, statistical significance or potential margin of error was unable to be calculated.
Insights
Data was cleaned and then thematic analysis was conducted to create themes from the responses to find the areas of opportunities and successes. Quantitative analysis was conducted to measure product satisfaction, trust, data accuracy, and to identify areas of opportunities.
Impact and Results
After analyzing the insights, I worked with the product management team to create a report that highlighted the successes and challenges of their product and recommended development opportunities to guide their 2024 and 2025 product plans.
Research Report