YouTube | End-to-End Research
My Role
UX Researcher, Team Lead
My Responsibilities
Project Scoping, Study Design, Protocol Development, Participant Recruitment, Moderation, Data Analysis, Presentation
Research Methods
In-depth Interviews, Comparative Evaluation, Survey, Heuristic Evaluation, Usability Testing
Tools
Remote interviews, Qualtrics (Survey)
IMPACT
My team and I scoped, set up, executed, and evaluated an end-to-end research project that uncovered key user needs and mental models as well as highlighted high-urgency usability pain points that needed to be addressed in the next iteration of the product. Our work provided the YouTube team with specific avenues to explore for new product features based on our research on content creators’ needs and motivations.
These avenues included:
Enabling creator collaboration by easing the initiation of creator relationships
Integrating file management and communication with the Creator Studio to take advantage of existing behaviors on GroupMe, Messenger, and TweetDeck
Considering a shift from pure content publishing to a creator community that encourages a sense of ‘togetherness’ and fosters co-creation and creativity
Through our evaluation of the product design, we generated a list of design opportunities to give the client team key improvements to make. These recommendations included:
Improving feedback and consistency throughout the video upload process
Utilizing a consistent design language through buttons and icons
Increasing visibility of hidden features and functions that novice users had difficulty locating
Mapping feature function to user expectations or intuition to minimize confusion and onboarding / learning time
ABOUT
The Creator Studio team at YouTube was planning to revamp and grow the Creator Studio and needed to better understand the needs, habits, and behaviors of YouTube creators, those who create, publish, and share content on the platform.
As team lead, I worked with my student team to devise a research plan that consisted of 2 phases: Needs Assessment and Product Evaluation. These phases would answer our client’s questions and also generate key product and market opportunities for future product development.
Phase 1 | Needs Assessment: 1:1 interviews (remote, in-person), Comparative analysis, Survey
Phase 2 | Product Evaluation: Heuristic evaluation, Usability testing
METHODOLOGY
1. In-depth Interviews | 2. Comparative Analysis | 3. Survey | 4. Heuristic Evaluation | 5. Usability Testing
The goal of this end-to-end research project was to better understand the behaviors and mental models at play with creators and highlight usability pain points to be improved on in the Creator Studio. We used multiple research methodologies to explore these aspects.
PHASE 1 | NEEDS ASSESSMENT
1. In-depth Interviews
GOAL: understand user needs, motivations, behaviors, and mental models at play with content creation on different platforms, including Snapchat, Instagram, YouTube, and Twitch.
1-on-1
range of creator experience (Novice = <1k subs, Emerging = 1-100k subs)
3 Novice, 4 Emerging
Experience creating and publishing content on different platforms
18-30 years old (specified by client)
2. Comparative Evaluation
GOAL: understand how competitors were performing relative to Creator Studio.
We evaluated the market landscape by exploring the features, design, and market strategies of 7 competitors in the content sharing space. Competitors were categorized by their relationship to YouTube’s Creator Studio.
Direct: Facebook, Vimeo
Indirect: Snapchat
Partial: Twitch
Parallel: Amazon Video Direct
Analogous: Soundcloud, Twitter
The primary factors of comparison were content editing, social networking, and creator collaboration.
3. Survey
GOAL: further explore how content creators collaborate and coordinate the co-creation process.
We used findings from the interviews and comparative analysis to organize the survey and utilized qualitative and quantitative approaches to achieve our research goals. The survey received a total of 250 responses in 7 days.
PHASE 2 | PRODUCT EVALUATION
Phase 1 research methods gave us deep insight into the minds of content creators and a well-rounded understanding of their needs, desires, motivations, and mental models.
We then shifted priorities from understanding creators to evaluating the Creator Studio itself. Phase 2 was dedicated to investing opportunities for design improvement that would offer a better end-to-end user experience.
4. Heuristic Evaluation
GOAL: quickly identify usability issues within the user interface
We conducted individual heuristic evaluations of the Creator Studio (Beta), combined our individual analyses, and collaborated to generate a list of usability pain points ranked by priority. The platform and all workflows a YouTube creator would experience was evaluated against the following factors (modeled after Nielsen’s heuristic approach):
Feedback: visibility of the system status
Metaphor: match between the system and real world
Navigation: user control & freedom
Consistency: consistency & standards
Prevention: error prevention
Memory: recognition rather than recall
Efficiency: flexibility or efficiency of use
Design: aesthetic & minimalist design
Recovery: error recovery
Help: help & documentation
5. Usability Testing
GOAL: put the Creator Studio through its paces, uncover and highlight key usability issues and pain points that creators would experience when using the product
With the heuristic evaluation giving the team direction for redesign strategies, we ran 1:1 usability sessions. We recruited Novice creators and Experienced creators (as defined above) to complete relevant tasks and provide feedback.
FINDINGS SUMMARY (NDA)
Here’s a summary of the research findings, post-NDA-redaction:
YouTube Creator Studio is a combination of a content creation platform and a social platform
YouTube does not sufficiently engage communities
Creators, especially experienced creators, use Creator Studio as a ‘middle man’ tool with video editing and social sharing occurring in other platforms
Initiating creator collaboration is difficult, especially for novice creators with limited networks
Creators frequently use texting, group messaging, and file management outside of YouTube to coordinate collaborative content creation (Google Drive, GroupMe, iMessage, TweetDeck, Messenger)
The Creator Studio needs to utilize a consistent design language and minimize icons and buttons that function in unexpected ways
RECOMMENDATIONS
Enabling creator collaboration by easing the initiation of creator relationships
Integrating file management and communication with the Creator Studio to take advantage of existing behaviors on GroupMe, Messenger, and TweetDeck
Considering a shift from pure content publishing to a creator community that encourages a sense of ‘togetherness’ and fosters co-creation and creativity
Improving feedback and consistency throughout the video upload process
Utilizing a consistent design language through buttons and icons
Increasing visibility of hidden features and functions that novice users had difficulty locating
IF I COULD REDO THE PROJECT, I WOULD…
Diversify the participant sample
Interview and usability testing participants were University of Michigan students, which narrowed the participant pool and likely generated homogenous feedback and perspectives.
Enlarge sample size for survey and usability testing
For statistical power, the survey sample ideally would have exceeded ~400 (based on confidence interval and power calculations).
While 5 usability sessions offered useful insight into the product’s design, 2-3 more sessions would have helped prioritize certain pain points over others through frequency and more qualitative data.