User Testing Online Booking at RWJBH
User Experience Researcher (UXR) // Montclair State University’s Dept. of Sociology // May - June 2023
RWJ Barnabas Health (RWJBH) is the largest and most integrated health care system in New Jersey since its 2016 merger with RWJ Health System. RWJBH provides treatments and services for 5 million residents in 9 counties through its vast network of hospitals, trauma centers, rehabilitation centers; primary care and specialist care outpatient centers; academic and research centers. To meet this increase in patient demands, RWJBH began its “migration from multiple electronic health record (EHR) systems to a single, universal EHR,” called Epic Together, to allow users to ‘’conveniently participate in [their] healthcare online” in January 2020.
CHALLENGE: Despite growing trends in the adoption and use of online scheduling by hospitals and their patients, RWJBH users continue to struggle to find a provider and book online appointments and instead resort to calling the hospital’s telephone service, which is expensive and time-consuming for RWJBH staff.
Minutes per call: 8 min. (30% wait time)
Cost per call: $1-5 per appointment
Annual loss from missed appointments: ≅ $150 billion per year
SOLUTION: To improve the learnability and usability of its online booking system to reduce the high costs of telephone scheduling, improve conversion rates for uses to its new patient portal, and to increase patient satisfaction.
UXR METHODOLOGY
Moderated Usability Studies: I conducted five remote moderated usability tests via Zoom and asked users to do a concurrent “think aloud” (CTA) as they completed the following tasks and answered follow-up questions: 1) tour and navigate from the homepage; 2) find a provider (an urologist); and, 3) book an online appointment. Video and audio of the studies were recorded with users’ consent.
System Usability Scale (SUS): a System Usability Scale (SUS) was given to the same five participants to quantitatively assess the learnability and usability of a website by scoring 10 standardized questions with one of five responses that range from Strongly Agree to Strongly Disagree. It is a reliably valid tool, though not a diagnostic one.
PARTICIPANTS
Five participants were carefully selected to participate in this user research study based on their availability, willingness to be recorded and eligibility. I was specifically interested in recruiting individuals aged 27-56, as this group typically shows a preference for online booking and scheduling appointments after hours. Here is a breakdown of participants based on demographic information collected via survey at the end of user testing:
Age: 50% aged 35-44 years; 50% aged 45-55 year
Gender: 66.7% female; 16/7% male; 16.7% nonbinary
Race: 100% white
STUDY LIMITATIONS
While this research study provides important insights into why RWJBH users struggle with online booking, there are several limitations that are to be considered: usability studies only conducted on urology department; small sample size (n=5); selection bias (participants were friends and family members that were easy to access and schedule); time constraints; lack of incentive for participants; and, all participants were white. Though this does not reflect my deep belief in inclusive research practices, I do not believe this impacted the results of the study.
USABILITY STUDIES
The chart below depicts the average time spent and level of difficulty experienced by the user test participants per task performed.
3 out of 5 users struggled to use the search functions to find a urologist
5 out of 5 users struggled to schedule an online appointment
PRIMARY OBSTACLES TO SCHEDULING ONLINE APPOINTMENTS
Several obstacles to online booking were uncovered during usability testing. Users felt that suboptimal search functions, unsatisfactory web navigation, and a lack of systems integration led to a poorer user experience than was expected for a health system of this scale.
Additional Findings: Usability Testing
When users get to a Book Online page directly from this providers’ profile page (Neel Shah, MD), it does not pre-populate the form and users are required to input the doctor’s name, specialty, etc. (see images).
Even after filling out the form, users cannot book online; they receive a message that they will “receive a phone call to book.”
When users mistype a word in the search bar, the user gets redirected to the Find A Doctor page and their typo gets repopulated in the field titled: Physician Name.
Each time a user presses a CTA button - a new tab opens and users have no access to a back button or exit.
When users click on providers’ “Profile”, there are several malfunctioning CTA buttons
This video demonstrates the difficulties users had when searching for a urologist and booking an online appointment.
SYSTEM USABILITY SURVEY (SUS) RESULTS
Results of the questionnaire mirror the findings from the usability studies.
Additional Findings: SUS
50% of users said “strongly disagree” when asked they would like to use this website frequently.
50% of users found this website “somewhat” unnecessarily complex or complicated.
50% of users said, “there was too much inconsistency in this website.”
50% of users said they “somewhat agree” that the website is quick to learn to use.
100% of users said this website was NOT well-integrated.
SUMMARY OF KEY FINDINGS
The following findings are based on the opinions, attitudes and experiences shared by the majority of participants of the usability studies and SUS questionnaires.
RESEARCH INSIGHTS
The following findings are based on the opinions, attitudes and experiences shared by the majority of participants of the usability studies and SUS questionnaires.
RECOMMENDATIONS FOR IMPROVING ONLINE BOOKING
Implementation of the following recommendations will immediately improve users’ access to online booking and streamline the scheduling process, but has the potential for immediate and long-lasting impact:
Reduce costs and increase revenue
Increase user engagement, utilization and adoption of Epic Health
Decrease physician and staff stress
Reduce barriers to patient care
Improve patient satisfaction and trust in the RWJBH brand
Thank you to Professor L. Stevens and classmates at Montclair State University for their continual feedback and guidance.