top of page

Onboarding Survey Analysis

UX Research

Project Overview

This project was completed during my time at Rocket Central. I was responsible for conducting the quarterly onboarding survey analysis. The goal was to gather feedback from new hires and identify areas of improvement in the onboarding process. The insights gathered through this research would be reported back to stakeholders, enabling them to make data-driven decisions and improve the overall onboarding experience at Rocket. This specific case study will focus on the November 2022 - January 2023 quarter.

My Role

UX Researcher

 

​

Time Frame

3 months (Nov 2022 - Jan 2023)

Research Process

Objective
Plan
Methods
Conduct
Synthesis

Research Tools

QualtricsXM_RBG.png
Microsoft-Excel-Symbol.png
112-1126616_microsoft-powerpoint-presentation-microsoft-office-power-point-2017-logo.png

Objective

Why is this project important?

By analyzing the data collected through surveys, Rocket's onboarding specialists would gain valuable insights into the effectiveness of the current onboarding process. This research project would enable them to identify any pain points, areas of improvement, or potential bottlenecks that new hires may encounter during their onboarding journey. By thoroughly examining the survey responses, the onboarding specialists can gauge the overall satisfaction level of new employees, understand their expectations, and pinpoint specific areas that require attention. With this knowledge, they would be able to make informed decisions and implement necessary changes to enhance the onboarding experience. 

Goal

Ultimately, the completion of this survey analysis UX research project would ensure that new hires at Rocket receive a seamless and engaging onboarding process, leading to increased job satisfaction, productivity, and long-term retention.

Plan

Project Plan

Having a project plan in place allows the onboarding specialists and other stakeholders to gain a clear understanding of the project's scope, deliverables, and milestones. Although the plan had already been established before I took the lead on this project, I made adjustments to align it with the specific insights I aimed to extract for that quarter. 

​

During this quarter, my objective was to analyze and compare the survey responses between bankers and non-bankers, with a specific focus on determining if either group was significantly influencing the overall results.

Methods

How it worked

The onboarding survey was sent out to new hires after 1-month, 3-months, and 6-months of being hired. The survey was done in Qualtrics and each time the survey questions remained the same. The team members were expected to give short answer responses. They were asked about their experience as a new hire, their expectations, and improvements. 

At the end of each month for the quarter, I collected the data received for that month, exporting it to an Excel sheet for better organization. To facilitate sorting, I added two additional columns labeled 'tag' and 'feedback', where I assigned tags to each response based on the comment's topic, such as 'Recruiters' for comments related to team members' experiences with recruiters. Additionally, I used the second column to classify comments as positive or negative using the labels 'Pos' and 'Neg,' which allowed for easy separation and presentation of data based on different labels. Leveraging PivotTables, I generated various tables examining different groups. With these tables, I created graphs and charts for better data visualization and presentation.

The General tags I used:

​

  1. Train = Comments related to Roadmap training, role-specific training, and professional development

  2. Recruit = Comments related to recruiters or the recruiting process

  3. Coms = Comments related to communication

  4. Culture = Comments related to company culture or ISMS

  5. SoA = Comments related to the Stamp of Approval process

  6. Tech = Comments related to technology

  7. Organizational Context (O.C.) = Comments related to knowledge about the FOC

How I tagged comments:​

“Christy is amazing. She worked with me throughout the entire process. Called to check in & kept me up to date. She deserves a raise!”​

Recruiters, Pos

“Smaller groups but understand that is difficult with so many starting at the same time. Lots of people were asking questions in the chat that were either already answered or going to be answered so it was distracting.”

Train, Neg

Comments that were longer and contained multiple tags were tagged as follows:

“I was hired to be an agile product owner and the job was described as such. It would be beneficial to train others at RH on Agile methodologies if the company wants to use this approach, as there is some misalignment here. RH has created their own definition of the role, which does not equate to the industry standard definition. Although implementation of Agile is different at every company, the gap here is large enough that it causes confusion. I am receiving training on the business side, which will help me be more effective.”

Coms, Neg

Train, Neg

Train, Pos

Results: Where We Left Off

Downward trend in total responses

Untitled design (24).png

Pos vs. Neg tagged comments

Previous quarter vs. this quarter

Untitled design (22).png

General Tag Breakdown

Previous quarter vs. this quarter

Untitled design (23).png

General Tag Breakdown Differences

Screen Shot 2023-07-17 at 2.39.36 AM.png

Results: Bankers vs. Non-Bankers Deep Dive

Bankers vs. Non-Bankers

To ensure unbiased analysis, we examined the data separately based on responses from team members who are bankers and those who are not.

Pos vs. Neg: Bankers vs. Non-Bankers

Untitled design (26).png

General Tag Breakdown: Bankers vs. Non-Bankers

Untitled design (27).png

  • The general breakdown of tags between bankers and non-bankers were about the same.

  • Non-bankers had 8% more comments regarding culture than bankers

  • Bankers had 11% more train comments

Culture: Bankers vs. Non-Bankers

Untitled design (29).png

  • Comments related to culture were mostly positive from both groups.

  • Culture related comments made up 43% of all tagged comments from Nov 22- Jan 23.

  • Bankers had more culture comments than non-bankers.

Train: Bankers vs. Non-Bankers

Untitled design (28).png

  • Overall, training comments are shown as negative for both bankers and non-bankers.

  • Training related comments made up 35% of all tagged comments from Nov 22 – Jan 23.

  • Bankers had more negative training comments than non-bankers by 24%.

  • Bankers reported not feeling prepared and need additional AMP training.

    • This was mentioned at all 3 marks.

      • 1-month, 3-month, 6-month

Conclusion

I enjoyed having the opportunity to lead the analysis of the quarterly onboarding survey. By delving into the data, we were able to uncover valuable trends and patterns that shed light on the strengths and weaknesses of the onboarding process. These insights not only inform immediate improvements but also lay the foundation for continuous enhancement in future quarters.

​

The success of this project relied on meticulous data collection, thoughtful analysis, and effective communication of the findings. By leveraging both qualitative and quantitative data, we gained a holistic understanding of the onboarding experience, ensuring that our recommendations were rooted in user needs and aligning with Rocket's overarching goals.

​

Ultimately, this project demonstrated the value of user-centric research in driving positive changes within an organization. Through the integration of data-driven insights into the decision-making process, we fostered a culture of continuous improvement and elevated the onboarding experience for new hires at Rocket Central.

bottom of page