NYU Grossman School of Medicine is known throughout medical education as an innovator and a thought leader. Ranked No. 2 in the United States for research and development, the school, which counts several Nobel laureates amongst its alumni and staff, has recently garnered significant attention for its innovative approach to medical education. With a tuition-free initiative, cutting-edge three-year MD degree program, and a ‘Curriculum for the 21st Century’, NYU Grossman School of Medicine is ushering in a new era of medical education designed around the needs and values of today’s students.
These initiatives are part of the Grossman School of Medicine’s larger mission to address the scientific, social, and economic aspects of the healthcare system by using the latest technology to enrich medical education.
Re-imagining their MMI online during the Covid-19 pandemic, NYU Grossman School of Medicine saw the benefits of a virtual interview, but their process in Zoom fell far short of the ideal. For the 2022 admissions year, with pandemic restrictions lifting, some schools opted to simply switch back to their tried and true in-person methods. NYU Grossman school of medicine, however, found a better solution.
Transitioning their MMI into Kira Talent’s Live Interviewing platform, the school of medicine aims to eliminate the technical hassle of the Zoom process while still leveraging the accessibility and cost benefits of an online experience.
“Each of the two formats, virtual and in-person, has its pros and cons,” shared NYU Grossman School of Medicine’s Associate Dean for Admission and Financial Aid, Rafael Rivera.
“For us, the positives of virtual interviews outweigh the negatives.”
The evolution of admissions interviewing at NYU Grossman School of Medicine
“We started utilizing the MMI process in 2013, operating it in-person until 2020 when we shifted it online via Zoom,” shared Anne Holtzman, the Program Manager of Admissions and Financial Aid at NYU Grossman School of Medicine. “We grant around one thousand interviews per admissions cycle, so that process would have us seeing an average of 16 to 20 applicants per day, four days a week, from September to December.”
“We hosted the in-person process on our campus, booking upwards of nine rooms for the MMI, a presentation from the Dean, and a coffee hour with current students,” explained Holtzman. “While one group of applicants was completing their MMI, another group would be moving through the other events.”
“With Zoom we faced a significant challenge trying to replicate this experience,” she continued. “The platform isn’t meant for a large-scale interview format. We couldn’t run circuits concurrently, and we didn't have a timing system that would give the applicant and interviewer enough notice that their station was ending. It was a lot of time and effort for a process that didn’t provide a top-notch experience.”
“With Kira, we're able to interview more applicants than we ever have before in less time.”
Scaling the MMI with Kira
“One of the benefits to the Kira platform is its ability to allow us to scale more effectively,” shared Dean Rivera. “When we were doing in-person interviews, physical constraints — such as the size of our conference room or the number of interview stations that we had — limited the number of applicants we could interview per day. Our process in Zoom was still negatively impacted by those constraints because we couldn’t run more than one circuit concurrently.”
“Kira has allowed us to streamline the operations, improve our efficiency, and decrease our faculty, interviewer, and staff utilization.”
“We're able to interview more applicants in less time, which has allowed us to go from four days of interviews per week to two days per week,” Holtzman added.
Reducing time-on-task for their interviewers not only helps NYU reduce reviewer fatigue, it also positively impacts their bottom line.
“There's an opportunity cost for our interviewers,” Dean Rivera explained. “The hours that they spend interviewing are hours that they’re not seeing patients, doing research, or applying for grants. That has a direct bottom-line impact for the institution.”
“Kira has helped us significantly reduce the time required from our faculty by enabling us to conduct multiple circuits in parallel as opposed to just one at a time,” he continued.
“With in-person interviews, we needed faculty in the room for four hours. With Kira, we only need them online for one hour.”
Kira’s Live Interviewing platform has the capacity to host over 1000 participants at a time across multiple concurrent circuits. By automatically moving applicants through to the right stations at the right times while interviewers assess them in real-time using built-in rubrics, the time savings for NYU Grossman School of Medicine’s admissions staff were significant.
“With our old process, we needed two people in the room for 14 to 16 hours a week conducting interviews,” Dean Rivera shared. “Kira has brought that down to one person overseeing the process online for two hours.”
“That’s an almost 16-fold reduction in the number of person-hours taken up by the MMI.”
“The organization, scheduling, and orchestration of the MMI used to be a full-time, two-person job,” shared Dean Rivera. “With Kira, one person can now easily oversee the entire process with far less stress.”
The ease and efficiency of Kira’s AMP integration
Some of the real time-savings for NYU Grossman School of Medicine come from the integration of their live MMI in Kira with their existing process in AMP.
“Kira’s integration with AMP has made everything so much easier,” shared Holtzman. “We schedule our interviews in AMP and that schedule gets transferred to Kira in a single click. We conduct our interviews in Kira, and our interviewers record their scores and notes in Kira. Then, overnight the system automatically transfers all of that data back into AMP so our committee members can see everything just as they always have.”
“Our interviewers love the platform — especially our returning interviewers who have gone through the in-person interviews and the Zoom interviews.”
“When we were using Zoom, they would have to toggle back and forth between Zoom and AMP in order to record their scores, all while maintaining the screen share so applicants could view the station prompt,” Holtzman explained. “Interviewers were constantly worried that they would accidentally show applicants the evaluation page. Kira has completely eliminated that concern by offering a seamless experience for both our interviewers and our applicants.”
The pros and cons of medical school virtual interviews
“Each of the two formats, virtual and in-person, has pros and cons,” Dean Rivera shared. “For us, the positives of virtual interviews simply outweigh the negatives. An in-person admissions process carries a significant cost for both the institution and the applicant. From travel cost to accommodation bookings to the cost of taking time off of work or school, the burden on applicants adds a potential layer of inequality to the interview that is not always top of mind for schools. Add to that the various costs our school incurs in order to host an in-person MMI, and it’s clear that the virtual interview is financially better for both us and our applicants.”
“It offers our interviewers more flexibility as well,” Holtzman added. “When we were interviewing in person, interviewers would sometimes have to cancel last-minute because traffic was bad, surgery ran long, or someone at home was sick."
"Kira has mitigated so many of those factors, as they can now interview with ease from their office, from home, or from wherever they have a computer.”
For many schools, a reason to maintain the traditional in-person MMI, no matter the costs, is the opportunity it presents to bring applicants on campus where it’s easier to engage them in recruitment efforts. But, for NYU Grossman School of Medicine, that status-quo thinking is getting turned on its head. Rather, they’re evolving their recruiting and interviewing processes to align with the highest standards of diversity, equity, and inclusion in admissions.
“There are programs that will feel that the interview day is a vital part of their recruitment activities,” shared Dean Rivera. “It’s true that the structured MMI format limits the ability to have those casual conversations that help form an emotional connection with applicants, and that the virtual setup alters how a school can promote itself in the eyes of the applicants.”
“However, I would argue that we need that structure and those limitations in the interview process” he continued. “Structured assessment improves our ability to accurately predict applicant performance and helps us reduce bias in the admissions process.”
“Recruitment and assessment should not compete for time. You have limited time to assess the applicant’s suitability for your medical program — recruitment can happen separately.”
Reimagining recruitment
“In our own admissions process, we’ve taken the recruitment-focused events that we used to conduct during our in-person interview day, and we’ve recreated them online,” Holzman shared. “We conduct interviews on Tuesdays and Wednesdays and invite that week’s applicants back on Thursday when they meet with deans, faculty, and current students via Zoom. Dean Rivera holds regular office hours where applicants can connect with him virtually to ask questions about the school, and we host various informational webinars which are scheduled throughout the admissions season.”
“In the spring, we invite accepted applicants to virtual and in-person sessions which culminate in a campus visit in April,” Dean Rivera added. “This approach helps us concentrate our efforts. The first part — the interview — is about identifying who would be a good fit for our school. The second part — the recruitment events — is about showcasing who we are and forging that connection with applicants.”
Building a better applicant experience
“We sent a survey to our applicants after their interview to ask them how the day went and what they thought about the MMI in Kira,” Holtzman shared. “Several applicants had interviewed with us before so they had a direct comparison to the nerve-wracking Zoom process.”
“With Kira, our applicants were significantly more comfortable,” Holtzman shared. “They felt the platform was simple and intuitive, removing any technical stress and letting them focus on their prompts and the interview itself.”
Discover what 2022's applicants had to say about their experience with Kira Talent
A seemingly small aspect of virtual interviewing – station timing – provides a perfect example for the school.
“It used to be a big problem for us with Zoom because our MMI consists of both single and double-length stations,” Holtzman explained. “We didn't have a timing system that could give the applicant enough notice before they switched stations. As a result, applicants would sometimes get cut off mid-sentence and then would show up in the next station all frazzled and concerned that they hadn’t put their best foot forward.”
“Kira has completely eliminated this concern,” she continued. “The built-in countdown timer provides both an auditory and visual alert before the end of a station, giving applicants time to wrap up or offer any final thoughts.”
“Our faculty, staff, students, the dean, and some of the leadership who interviewed for us, they all loved the convenient and seamless experience in Kira.”
