Background
- MOMA Homepage Re-design [otherwise known as Google's Google used for google-wide internal search].
- The team had developed two design prototypes for Moma Homepage to improve navigation, content organization and to increase overall engagement.
Primary Research Goals
- Evaluate these high-fidelity concept mockups to decide if they can measure up to become more engaging platform designs for Googlers on a unified google-wide range.
- Finding out the crowd favorite and most favorable design amongst the two, by conducting an AB Test. Furtherly finding out how these designs rank compared to the one live on Moma right now.
- Analyzing the current experiences and usage patterns of the participants on the current live Moma homepage.
Study Approach
Usability Testing:
- 60 minutes 1:1 remote sessions (semi-structured interview + concept testing of two designs)
- Counterbalanced cognitive walkthrough of 2 clickable prototypes
Study Timeline [1 month]
Recruitment ~1 week
Research Plan + Discussion Guide ~1 week
Usability Sessions ~1 week
Report Readout and Presentation ~1 week
Research Plan + Discussion Guide ~1 week
Usability Sessions ~1 week
Report Readout and Presentation ~1 week
Recruitment
- The intended design is to reach all Googlers of APAC, Americas and EMEA regions as Google Homepage is used worldwide by all Googlers.
- Recruitment involved a total of 16 participants worldwide: 8 participants from Americas and 8 participants from APAC and EMEA regions.
- Partnered with UXI for worldwide Googler search and incentive distribution.
- Designed and sent out user survey requiring criteria for recruitment:
- Tenure at Google : The study required a wide spread of Nooglers [0-1 years], Experience professionals [2-5 years], [5-10 years] and [10+ years]
- Location: 4 from APAC, 4 from EMEA, and 8 from America.
- Frequency of visit on Moma Homepage: Range of frequency was from [Never -> Occasionally -> Frequently]. Ideally a spread of equal visits of each frequency range was looked at while recruiting.
Design Prototypes
Design 1: Filters Feed
Offers users the ability to filter the card system on top of the homepage to select their topics of interest.
Unique Element in Design 1
Verified Tag
Design 1 had a verified tag, where the verified tag alongwith an employee's name will appear at the end of each section to mark the content as verified.
Design 1 had a verified tag, where the verified tag alongwith an employee's name will appear at the end of each section to mark the content as verified.
Common element in both designs
Team Celebrations Card
A card on the homescreen of Moma would display a teammate's birthday or googleversaries
Design 2: Categorial Feed
Offers users content pre-organized by Category
Research objectives
Live Moma Homepage
Research Objective 1: What is the current experience of Moma Homepage like?
1.a. Identifying the types of users of the Moma Homepage.
Research Objective 2: Identify the missing elements/anything they wish to remove/like/dislike regarding the homepage.
Design 1: Filters Feed
Research Objective 1: To understand participant's attitudes behind wanting to personalize their Moma Homepage
Research Objective 2: To understand whether participants want to organize the content on homepage by categories and if adding filters make the homepage feed less overwhelming.
Verified Tag
Research Objective 1:Do users perceive the content as important if the verified tag is present?
Design 2: Categorial Feed
Research Objective 1: Understand how easily participants are able to find content.
Research Objective 2: Understand how content sorted by relevance is easier to navigate and consume.
Team Celebrations Card
Research Objective 1: What is the users reaction on adding a celebrations page in the MOMA homescreen?
1.a: How does that add on to the personalization aspect of the homepage?
Research Objective 1: What is the current experience of Moma Homepage like?
1.a. Identifying the types of users of the Moma Homepage.
Research Objective 2: Identify the missing elements/anything they wish to remove/like/dislike regarding the homepage.
Design 1: Filters Feed
Research Objective 1: To understand participant's attitudes behind wanting to personalize their Moma Homepage
Research Objective 2: To understand whether participants want to organize the content on homepage by categories and if adding filters make the homepage feed less overwhelming.
Verified Tag
Research Objective 1:Do users perceive the content as important if the verified tag is present?
Design 2: Categorial Feed
Research Objective 1: Understand how easily participants are able to find content.
Research Objective 2: Understand how content sorted by relevance is easier to navigate and consume.
Team Celebrations Card
Research Objective 1: What is the users reaction on adding a celebrations page in the MOMA homescreen?
1.a: How does that add on to the personalization aspect of the homepage?
Report Metrics
Severity Key
Severity of each research finding was rated amongst the following keys and then presented. This helped the clients bifurcate the design issues from requiring immediate assistance to just being useful insights to have.
This Worked: No hesitation or user irritation
Critical: Leads to task failure or causes extreme user irritation.
Moderate: Occasional task failure for some users, moderate hesitation or user irritation
Minor: Some hesitation or user irritation
FYI: A qualitative insight, behavior or trend
Critical: Leads to task failure or causes extreme user irritation.
Moderate: Occasional task failure for some users, moderate hesitation or user irritation
Minor: Some hesitation or user irritation
FYI: A qualitative insight, behavior or trend
Report Qualifiers
Research Findings
1. Current experience with MOMA homepage
The Moma Homepage is mostly used to start searches and most of the participants usually have a quick look at the cards to see if something interesting is going on when they have time from work.
MOMA homepage browsing was distributed in three types of browsing
1. Targeted Browsing:
2. Casual Browsing:
3. Rarely Browsing:
1. Targeted Browsing:
- 1/3rd of the participants search the home-screen for something specific (googler news, talks at google, etc).
2. Casual Browsing:
- Majority chunk of users casually browse the homepage by making a quick browse on the titles, interesting news and any new information that caught their eye.
3. Rarely Browsing:
- A small number of participants rarely browsed the MOMA homepage as they had very less time on their hands besides work to spend on the Moma Homepage.
- It was also observed participants who were part of the 'rarely browse' cluster, were googlers within 5-10 years of working range.
2. Design I: Filter Feed
Filters
Working [Success]
All participants except few in Americas noticed the filters and understood what they were and how they worked.
Personalization [Moderate Issue]
Some participants felt that filters is not giving them the personalization level they would like to have: being able to view whatever they choose to rather than choosing amongst what is provided to them.
Card Arrangement
Difficulty to read [Moderate Issue]
Most of the participants said it is difficult to read through the mix of small and big cards: the different orientation of the info in the two different cards seems to decrease the readability of each card as well.
Co-relation of content to titles [Moderate Issue]
Most participants also struggled with co-relating icons present on top right corner of the cards to the cards itself due to their size and also the icon's co-relation with their titles was harder.
Verified Badge
Unnecessary [Critical Severity]
Most of the participants found verified badge ‘unnecessary' as they assumed information on Google internal pages will be verified/trustworthy content.
Working [Success]
All participants except few in Americas noticed the filters and understood what they were and how they worked.
Personalization [Moderate Issue]
Some participants felt that filters is not giving them the personalization level they would like to have: being able to view whatever they choose to rather than choosing amongst what is provided to them.
Card Arrangement
Difficulty to read [Moderate Issue]
Most of the participants said it is difficult to read through the mix of small and big cards: the different orientation of the info in the two different cards seems to decrease the readability of each card as well.
Co-relation of content to titles [Moderate Issue]
Most participants also struggled with co-relating icons present on top right corner of the cards to the cards itself due to their size and also the icon's co-relation with their titles was harder.
Verified Badge
Unnecessary [Critical Severity]
Most of the participants found verified badge ‘unnecessary' as they assumed information on Google internal pages will be verified/trustworthy content.
3. Design II: Categorical Feed
|
Card Layout
Organization of data [Success] Many participants loved uniformity of cards/boxes as they felt it was easier to read through as compared to the first prototype. Few participants preferred everything organized for them as they did not want to spend time searching for what is present. Googler News::Category Scrolling and Size [Moderate Issue] Many ppts said they do not like horizontal scrolling section. They said it interrupts the vertical scrolling when they just want to have an overview of all categories. Some users felt the ‘Googler news’ section was too big compared to other categories and would prefer to view all sections equally. |
4. Team Celebrations:: Common Element in Both Design
Loved [Success]
Most of the participants liked to have team celebrations cards on the Moma homepage: it is a nice reminder to connect with your teammates.
Privacy [Minor Issues]
A couple of users were not happy with the celebrations section as they themselves did not want to celebrate their birthdays. They also wanted the freedom to be asked if their birthday's should be displayed on the Moma Homepage
Most of the participants liked to have team celebrations cards on the Moma homepage: it is a nice reminder to connect with your teammates.
Privacy [Minor Issues]
A couple of users were not happy with the celebrations section as they themselves did not want to celebrate their birthdays. They also wanted the freedom to be asked if their birthday's should be displayed on the Moma Homepage
Final Results
A/B Testing
Explanation [No clear winner]:
Prototype 1 and 2:
Both the prototypes were equally preferred by half of the participants and the difference of liking became based on each individual's liking and preferences on what they wish to see in a homepage.
Likes and Dislikes
There were strong reasons for likes and dislikes for each prototypes.
Prototype 1
Likes: Most of the participants who preferred 1 over 2 loved viewing a single view for everything that they are interested in browsing at.
Dislikes: The homepage seemed very busy. Users were less likely to spend much time doing this especially in my working hours.
Prototype 2
Likes: All participants who prioritized the Prototype 2 over the rest loved the organization of the page.
Dislikes: Participants who chose #1 Prototype over Prototype 2, had a simple preference for viewing things all at once and enjoyed selecting what they want to view. There weren’t many negative remarks about Prototype 2.
Prototype 1
Likes: Most of the participants who preferred 1 over 2 loved viewing a single view for everything that they are interested in browsing at.
Dislikes: The homepage seemed very busy. Users were less likely to spend much time doing this especially in my working hours.
Prototype 2
Likes: All participants who prioritized the Prototype 2 over the rest loved the organization of the page.
Dislikes: Participants who chose #1 Prototype over Prototype 2, had a simple preference for viewing things all at once and enjoyed selecting what they want to view. There weren’t many negative remarks about Prototype 2.
Explanation:
Prototype 1 [Winner]
Page arrangement (‘organized content’) seems to be much more important than the possibility to ‘control’ or select the content with filters. Prototype #2 help users to identify through a quick scan interesting and relevant information without missing out hidden cards.
Prototype #1 feels busier that the current version but some participants said they can get quickly to the topic of interest with filters.
Prototype #1 feels busier that the current version but some participants said they can get quickly to the topic of interest with filters.
Live Moma in 'Americas and APAC/EMEA'
The current design is easier to read/scan and therefore sometimes preferred by participants when visiting the page having the time to browse all cards and not excluding some with filters.
Executive Summary
Winner:
●Prototype #2 was voted as the most favourite by 6 out of 8 participants in APAC and EMEA
●In Americas there was an almost equal level of preference for both the designs with a very slight skew towards Prototype #2
Organization of data:
●Most of ppts in all regions loved the organizational layout of Prototype #2
Current vs New Designs
●Most of the ppts preferred the new designs compared to Live MOMA
Celebrations Cards
● Most participants in all three regions liked having Celebrations Cards on the homepage.
Verified Badge
● Most of the ppts in all regions thought they believed the content in MOMA as verified and thus, did not find the Verified Badge as useful.
●Prototype #2 was voted as the most favourite by 6 out of 8 participants in APAC and EMEA
●In Americas there was an almost equal level of preference for both the designs with a very slight skew towards Prototype #2
Organization of data:
●Most of ppts in all regions loved the organizational layout of Prototype #2
Current vs New Designs
●Most of the ppts preferred the new designs compared to Live MOMA
Celebrations Cards
● Most participants in all three regions liked having Celebrations Cards on the homepage.
Verified Badge
● Most of the ppts in all regions thought they believed the content in MOMA as verified and thus, did not find the Verified Badge as useful.
Recommendations
Organization of data:
Card Layout:
- Information gained by current homepage dislikes and new designs strong suits it was recommended that users prioritize structure and organization of data as priority and thus the redesigning should include this as a top priority.
- Since most of the participants had difficulty to read through the mix of small and big cards, it was recommended to avoid mixing cards with different info orientation on the same row since to increase its readability.
- Remove the horizontal dialogue box of the Googler news in #2 Design: Based on cognitive psychology of a brain and eye orientation, a typical browsing pattern of a website page is from top left hand side to bottom right hand side corner. Thus, adding a horizontal dialogue box breaks the natural flow and could lead to stopping the browsing of the home-screen altogether.
- The Design #1 had failed in providing the personalization factor and it was observed that based on the usage patterns participants were more inclined towards the finding the data in less time than wanting to personalize the homepage. Thus, the focus of new designs should be more towards making the browsing simpler than adding personalization to the homepage.
Card Layout:
- Most participants couldn't recognize the cards from just the icons in #1 Design layout of cards. Thus, consider making a stronger visual labelling of the cards according to the category they belong to.
- Most participants loved the team celebrations card and wanted more of it. For example adding additional features like 'promotions' along-with birthdays and google-versaries.
Next Steps
- In order to understand the difference of preference between AMERICAS and APAC & EMEA regions: Try to find out if Googlers in those regions use Moma Homepage differently? Check if the behavior on the page in those region is different (scrolling rate, clicks on the cards, etc…
- Consider creating an 'ideal design' which brings out the best features liked by users found out in the research as there was no clear winner in Americas.
Challenges
1. Recruitment:
2. Having different research findings in different target areas
3. Handling users
4. Clearly Communicating with Stakeholders and being accommodating
- Most participants who signed up for the study were Nooglers i.e. 0-1 years of experience.
- To make the study more favored towards the larger audience we needed more participants in the range of 2-5 years of experience, to 10+ years of experience. We couldn't move ahead with the study without having those participants.
- Thus, we had to re-tweak our user survey questions and send out a whole new batch of invites specifically targeting to users within those years of experience.
- We also potentially had to adjust a few meetings with the stakeholders and had to space out the days of user testing to incorporate the later found participants.
2. Having different research findings in different target areas
- We found very different results in the regions of APAC/ EMEA, compared to Americas.
- We had to hypothesize a number of varied reasons behind these results and present it to the stakeholders with well founded rationality.
3. Handling users
- We don't always get the smoothest usability test participants.
- In a few sessions I personally had to handle strong-willed users with patience and compassion to have a successful usability test.
4. Clearly Communicating with Stakeholders and being accommodating
- Learning to adapt report to deliver to executives and external stakeholders
- Adapting to their informational needs - succinct answers to high-level, strategic questions.
What I would do differently?
If I had more freedom over my research study rules:
- I would facilitate a UX Research intern/junior to observe the sessions and take notes to help give undivided attention to what the user is saying and performing in the testing sessions.
- Run the scenarios for a longer time when the participant requires time to grasp concepts, or when the participants have a lot more information to give than the time requires.
- Increase the number of target participants for the study for getting more accurate results.
