At Notre Dame University’s Mendoza College of Business, a mission to enroll purpose-driven leaders is at the core of the school’s admissions decisions. Celebrating its 100th anniversary in 2021, the school’s rich history, awe-inspiring alumni network, and consistently strong rankings attract a wide audience of applicants.
Recognizing how difficult it was to ascertain their applicants’ academic aptitude, social capabilities, as well as their potential to succeed in their chosen field from only a resume, transcript, test scores and a couple of essays, Mendoza sought out a better way to connect with their applicants.
In 2015, the school added timed video and timed written questions through Kira Talent, as a way to give applicants the ability to exhibit their full potential while simultaneously streamlining the overall process for the admissions team.
Starting with the MBA program, Mendoza has since expanded their use of Kira assessments across multiple graduate business programs, providing the team with a more multi-dimensional picture of who an applicant is, beyond what they can show on paper.
Getting to know applicants earlier and more efficiently
Prior to Kira, Mendoza conducted interviews on a rolling basis, interviewing around 90% of all applicants who applied. Although this helped give the team a more well-rounded view of their applicants, the burden on the team was significant.
“The process was overwhelming our recruiting and admissions teams. Coordinating the interviews, with all the scheduling and rescheduling, and accommodating different time zones, was labour-intensive,” shared Maria Stutsman y Marquez, Director of Graduate Recruiting & Admissions at the University of Notre Dame’s Mendoza College of Business.
“We needed a structured way to provide our admissions committee with more in-depth insights into our candidates prior to sending out invitations to interview.”
With Kira, Mendoza is able to get a better sense of their applicants earlier in the admissions process.
Timed video and timed written responses provide reviewers with a more holistic view of each applicant, while streamlining the overall process. The platform’s integration with Slate further simplifies the process, allowing reviewers to view the responses with the rest of a candidate’s application.
“Now we have an efficient way to see and hear from everybody before we make that decision of who to invite to a formal interview,” shared Stutsman y Marquez. “We can conduct fewer interviews because we’re able to make more confident decisions about who we’re inviting to those interviews.”
Discover how UCD Smurfit used Kira to gain deeper insights into applicant fit
Finding the hidden gems in your applicant pool
With their new holistic approach in place, Mendoza’s team soon found a number of exceptional students who they may have missed with their old process.
“You’ll see someone’s scores, for example, and they’re not really where they should be. But then you’ll watch their video responses and they have great language skills, they express themselves clearly, and you can see their passion and potential to succeed,” shared Debby Herczeg, Assistant Director of Admissions at Mendoza School of Business.
“Kira has helped us find a lot of diamonds in the rough.”
“With our old process, I would estimate we were missing up to 10% of applicants who were hidden behind low test scores.”
With the ability to create an assessment specifically tailored to a program’s needs – from the questions asked to the competencies applicants are evaluated for – the Mendoza team can more quickly identify the candidates who will be successful in their programs and in the workplace.
“Kira really helps us uncover the humanness of the applicant profile you’re reading,” explained Stutsman y Marquez. “The candidates are no longer a static profile of cognitive scores, academic background and the essays they wrote. The admissions committee actually gets to see how they present themselves in a variety of in-program and real-life contexts.”
“Yes, Kira helps save us time. But, more importantly, it enables us to find the hidden gems in our applicant pool,” added Herczeg. “That’s been the biggest benefit. With Kira we get to meet every applicant and go into admissions decisions with a better understanding of their talents.”
A structured approach to reducing admissions bias
“With Kira we saw an opportunity to gain a deeper understanding of who our candidates are, while bringing a new level of structure not only to the interview process, but also to the application process as a whole,” shared Stutsman y Marquez.
One element that Kira brought to the table was independent review. By enabling multiple reviewers to independently evaluate an applicant's responses on their own schedule, Kira helps Mendoza gather more diverse perspectives without necessarily adding more time to the overall evaluation process.
Built right into the Kira platform, custom rubrics add further structure to the reviewing process by ensuring all reviewers are evaluating applicants based on the same criteria. Reviewers are able to reference the defined criteria on the rubric and score applicants as they review their responses.
“For us, having that structured process in place is so pivotal to creating a level playing field as much as possible for every candidate,” Stutsman y Marquez added.
“As we look to support the building of a diverse cohort, Kira plays an important role in helping us reduce bias by creating a more structured approach for all candidates, and this is critical to bringing in distinct voices and perspectives to the community.”
At the University of Waterloo School of Pharmacy, the admissions interview is a cornerstone of the admissions process. Centred around the knowledge that an applicant’s fit for the program is dependent on more than just their grades, Waterloo has long invested in creating an engaging and holistic admissions experience for their applicants.
“We really want it to be clear to applicants that, while grades are a component, we know that there’s so much more to them than that,” explained Kaitlin Bynkoski, the Director of Admissions & Undergraduate Affairs at the School of Pharmacy. “We’re continuously learning and experimenting with new ways to get deeper, more comprehensive insights into our applicants.”
For many years, interviews took place on campus, with hundreds of hopeful applicants travelling to Waterloo for a weekend of assessments, interviews, and tours. As such a core piece of the admissions experience, it was difficult to imagine the process going any other way. When COVID-19 forced most of the world to go remote, the school prioritized not only the interview but the experience that surrounded it.
“We needed a tool that was built with our team and our process in mind,” Bynkoski explained. “And that’s how we ended up finding a home with Kira.”
Why Waterloo Pharmacy is powering their admissions process with Kira
- The pitfalls of choosing the wrong platform
- How Kira has enhanced an already holistic process
- Transitioning admissions interviews online - seamlessly
- Reaching a 97.6% applicant interview rate with equitable admissions
- Saving 20+ hours of work with built-in data analysis
- Future-proofing admissions for repeatable success
A round-hole process meets a square-peg platform
Amongst a written skills assessment, a campus tour and other events, the focal point of Waterloo's interview weekend was the panel interview. For about 45 minutes, each applicant would sit down with a panel of two practising pharmacists and a current upper-year pharmacy student.
“Both interviewers and applicants travelled from all over to be on campus for the weekend - it was quite intense for everyone involved,” shared Bynkoski. “Some of our interviewers worked the entire weekend and stayed in accommodations near campus in order to participate in the process.”
“There’s an incredible amount of administrative work needed to make a weekend like that run smoothly.”
So when the COVID-19 pandemic forced most of the world to go remote, the admissions team at Waterloo decided to take their interviews online for the first time.
“The first year we went online, the asynchronous interview platform we chose served its purpose for that year, but both the reviewers and our administration team felt like a lot was missing,” shared Bynkoski.
A lack of customization options not only created more work for the admissions team as they adjusted their process to fit the constraints of the platform, but without the space to include those personal touches, the overall experience fell flat. Many reviewers voiced concern over the fact that they weren’t able to create a connection with applicants, and Waterloo School of Pharmacy knew they had to make a change.
Wanting to regain that face-to-face connection, the school was determined to recreate their token panel interviews. The seemingly easy solution was to jump on the same tool they were already using for internal video conferences, Microsoft Teams. But after a few trials and many errors, the idea was quickly abandoned.
“There are so many disadvantages to doing interviews with an off-the-shelf platform,” Bynkoski explained. “Coordinating 250 individual meetings with interviewers popping in and out is an immense amount of work and an often chaotic experience. Links getting missed and wires getting crossed, it’s just bound to happen.”
“We faced so many challenges, both technical and organizational,” Bynkoski continued. “And with no dedicated platform support, the burden was once again on us to organize and execute the entire weekend, this time with the added stress of relying on a technology we weren’t experts on.”
“Those platforms really weren’t designed for the way admissions teams interview, and they take away from the actual interviewing experience.”
An all in one platform for ultimate peace of mind
For 2021’s incoming class, Waterloo was determined to regain the quality and connection of their traditional interviews, but Bynkoski and her team still believed they could achieve that without giving up the benefits of being virtual. That’s when they connected with Kira Talent.
“The combination of using Kira’s Live Interviewing and Asynchronous Assessments was a home run for us,” shared Bynkoski. “We could have our real-time interaction with applicants and our structured skills assessment, and use both data points to create more comprehensive evaluations of our applicants. That’s what really solidified our decision to switch to Kira.”
The dual-capability of the Kira platform to support asynchronous and real-time interviews meant that Waterloo was able to conduct the entire second stage of their admissions cycle with one software. Waterloo’s fundamental skills assessment was transitioned from its traditional paper format to being completed by applicants through engaging, video and written responses to the school’s custom questions. They were then able to complete what previously was an in-person interview with Kira’s real-time video interviewing solution.
“For our fundamental skills assessment, the flexibility that the Kira platform offered to applicants and reviewers was a huge benefit,” explained Bynkoski. “And the real-time interviewing allowed us to maintain that face-to-face connection.”
“Getting the benefits of both synchronous and asynchronous solutions on the same platform not only made our process more efficient, but it created a very cohesive and professional admissions experience for our applicants.”
Virtual interviewing designed for admissions teams
From the first interaction to the last, Kira’s intentional platform design and streamlined process helped the admissions team create more memorable connections with their applicants.
“Our applicants were so thankful for the kind of experience that we were able to offer them,” Bynkoski shared. “Other schools came up with all kinds of substitutes, but what we heard from applicants was that they didn’t measure up to the experience we created with Kira.”
Discover what real applicants thought of their experience with Kira Talent
One week before their panel interview, Waterloo’s applicants received communication from Kira inviting them to complete their registration on the platform. The invitations, fully customized with the School of Pharmacy's branding, walked applicants through their device setup, encouraged them to use the practice suite to prepare for their real assessment, and connected them to Kira's 24/7 technical support if necessary.
“The reviewer training that Kira provided was also incredibly helpful,” shared Bynkoski. Jumping into Waterloo’s scheduled training sessions, Kira’s dedicated support team helped reviewers practice with the platform.
“Previous systems didn’t offer that, so in the past, I had to walk reviewers through a process that I was only just learning myself,” Bynkoski added. “Kira made sure that the reviewers felt really comfortable working with the platform, and we had no issues on interview weekend.”
With all the pieces in place, Waterloo’s applicants and reviewers could go into their assessment with full confidence.
“The platform is seamless,” shared Bynkoski. “The functionality is so smooth, and the custom branding allowed us to bring our unique identity to the platform. Every detail is so thoughtfully planned out. Everything down to the ability to record our team asking practice questions so that applicants could meet our faculty and community before they completed their assessment.”
“The degree of customization that was available to us, plus the ability to add our own unique flair to the platform – I hadn’t seen that anywhere else.”
When it came to replicating their panel interviews, Kira’s Live Interviewing enabled Waterloo to finally create the virtual interviewing experience they had always envisioned. From prep time and prompts to rubrics and scoring notes, the platform’s built-in features created a cohesive and professional experience that was never possible with their previous solutions. By simply clicking on the unique link included in their invitation, applicants and reviewers were automatically placed in the right interview rooms, at the right times.
“With Kira, our interview schedule was loaded into the platform so there was no day-of stress or coordination. All anyone had to do was follow the link from their emailed invitation and sit back as the platform automatically moved them through the rooms at the right times. It completely eliminated the work and worry of constantly toggling between screens.”
Discover what applicants and reviewers thought of their experience with Kira’s Live Interviewing
“You really can mirror your existing process with Kira, which a lot of other tools don’t allow you to do,” explained Bynkoski. “Where other platforms have you adjust your process to fit their platform, Kira approaches it in the opposite way.”
“You get a platform that is already designed for the higher ed admissions process and the flexibility to make it uniquely your own.”
“The customized branding and the ability to add branded videos right into the platform not only made the applicant experience visually appealing, it also acted as a valuable touchpoint for us to promote our school and our community,” shared Bynkoski.
Building a more equitable future by bringing more applicants to interview
With equity top of mind, Bynkoski feels more confident in how the school’s new process with Kira is able to engage more diverse candidates. With applicants often applying to more than one school, the travel and time costs associated with mandatory in-person interviews quickly add up. By removing those common barriers, Waterloo’s admissions process becomes more accessible to applicants.
Our virtual admissions interviews ran last weekend. Thanks to everyone who participated! By the numbers:
-230+ interviews
-25 interview panels
-75 interviewers (50 pharmacists – including 41 alumni! - and 25 pharm students) pic.twitter.com/bawFURPhBq— Waterloo Pharmacy (@UWPharmacy) May 3, 2021
“From a diversity and inclusion perspective, that’s a big change because it doesn’t cut out candidates just because of location or financial constraints,” Bynkoski explained. “If applicants can only afford to travel to one school, they’re having to make a very important decision very early in the admissions process.”
“With virtual interviews, applicants are able to make more informed decisions about where they want to study,” she continued, “and it ensures that we, as a school, have the opportunity to show them what we’re all about and why they should choose to enroll at Waterloo.”
For the applicants who do want to experience the atmosphere on campus, the School of Pharmacy plans to host open house days where applicants and their families can tour the campus outside of the stress surrounding the interviews.
“We see a substantial difference in how we’re engaging students to come to see campus when it’s an open invitation and not a requirement,” shared Bynksoki. “It’s a fun invitation to come and experience the atmosphere on campus, and it’s their choice whether or not to do so. Not wanting or being able to incur that cost does not affect their candidacy for the program.”
“We’re seeing now that not all students want to return to campus in the same ways,” Bynkoski added. “Yes, some can’t wait to get back in person, but there are many who prefer virtual interactions and the safety and convenience they provide.”
With Kira Talent, Waterloo’s School of Pharmacy increased their interview acceptance and attendance rate to 97.6%.
“More applicants accepted our interview offer and showed up day-of when we used Kira Talent, compared to our traditional process,” explained Bynkoski.
“As we seek to engage more diverse students in higher education, we need to address those issues and come up with new ways to make our processes accessible and equitable to every candidate, not just the ones that have the means to participate on-campus.”
Find out how more pharmacy programs are enhancing their admissions process with Kira
Saving 20+ hours with at-a-glance analytics
“Kira brings efficiency to each stage of our admissions process,” shared Bynkoski
Once invitations are sent out, administrators are able to see which stages of the process applicants have completed, whether it’s opening their email, registering on the platform, or completing a practice assessment.
“It helps us avoid any empty interview slots due to miscommunication or lost emails,” explained Bynkoski. “We can check in and see how our applicants are interacting with the platform before interview weekend.” With Kira’s admin dashboard, administrators can also see, in real-time, which reviewers and applicants are online, ready for their live interview.
However, it was post interview weekend where Bynkoski and her team experienced Kira’s real time-saving power.
“Since reviewers didn’t have to be on campus in order to grade the skills assessment, we had more volunteers than ever,” she explained. “We were able to have multiple reviewers evaluate each assessment without adding any time to the overall process. And because they could log in at whatever time to complete their review, Kira provided reviewers with flexibility and convenience that they really appreciated.”
As reviewers scored their assigned applicants, Kira’s real-time analytics dashboards provided key data points for the admissions team. Where in previous years Bynkoski had spent countless hours organizing the interview scorecards, inputting the data into their system, and then analysing and reviewing it all, with Kira it was already taken care of.
“With Kira, all that information was in my hands instantly,” shared Bynkoski. “Applicants and reviewers are finished when they leave interview weekend, but for the administration, there’s still weeks and weeks of work that goes into organizing, inputting, and analyzing all this data.”
“Having everything, including the scores from both our fundamental skills assessment and panel interview, recorded and organized in one platform and downloadable at the click of a button, saved us around 5 minutes of data entry per applicant. That means that this cycle, we saved more than 20 hours of admin work, and ultimately, this enabled us to spend more time completing a thorough and holistic file review rather than inputting data.”
“I think most people see the front end of the Kira experience and appreciate the design and functionality, but they don’t even realize everything that’s going on in the background,” continued Bynkoski. “We could see all our interviewers’ scores, monitor if anyone was consistently scoring above or below average, and make adjustments if needed.”
“We were able to show our admissions committee that we had multiple reviewers and their averages were the same, which helped us make more defensible decisions. In a paper-based system, you would never be able to get the level of analytic data that you get from Kira’s platform.”
Rave reviews and repeatable success
In a post-interview survey, Waterloo School of Pharmacy found that 90% of reviewers preferred this year’s experience with Kira over the previous years’ interviews.
“Kira Talent filled the void in last year's interviewing process,” shared one reviewer. “It was awesome to be able to conduct a regular interview online.”
“The system is way more time-efficient than in-person interviews,” added another. “I was a lot more effective in grading with the moveable rating scale than I was on paper.”
With 96% of reviewers rating the platform 4 or more stars, Bynkoski and her team are confidently moving forward with Kira Talent for future admissions cycles.
“We're committed to virtual interviews for the next five years. But from what we've seen with Kira, I'm optimistic this will be our process for the foreseeable future,” explained Bynkoski. “From the sustainability gains that we’ve seen and the cost savings around that, to the equity and accessibility of the online process, to those higher interview acceptance and attendance rates, all of these are reasons why we feel confident saying that we’re staying with Kira long-term.”
From the anticipation of college acceptance letters to the thrill of high school graduation, spring was an incredibly exciting time for us prospective college students. But now that the excitement has settled, it’s easy to get overwhelmed by all the big life changes that are fast approaching. From dorm room decorations to class selection and financial planning, preparing for university can at times feel as overwhelming as imagining my first day on campus.
Without the proper support, students can easily get knocked off course. 1 in 3 students falls victim to summer melt each year. This challenge for schools can end up costing them hundreds of thousands of dollars in lost tuition revenue.
As an incoming freshman at McGill University, my summer internship at Kira Talent provided the perfect opportunity to lend a student’s perspective on the challenges that cause summer melt. Through my own experience, as well as interviews with other incoming freshmen to universities throughout North America, I've cultivated some actionable insights for schools wanting to mitigate summer melt this year.
1. Engage your students on social media
The most effective way to keep students engaged over the course of the summer is to meet them where they already spend the majority of their time - online.
Instagram Takeovers
Holding student-run social media takeovers, where current students temporarily take over the school’s official Instagram account, provides incoming students with a way to actively engage with the school community in an authentic and informal way. Glossy photos on a website can make campus life seem like a different, slightly intimidating, world from the one they’re used to. Live streams and takeovers can help students relate to the school in a way they’re already comfortable with.
Creative Contests
Hosting fun activities like trivia contests via social media is another way to sustain authentic engagement while giving incoming students pertinent information about your university. With a simple prize of a school-branded sweatshirt or campus coffee shop gift card, it’s a low-cost way to keep the excitement fresh throughout the summer.
Facebook Groups
To help us get settled before we even reach campus, McGill created Facebook groups for students based on their residence hall assignments. Being able to easily get to know the students I’ll be living within the Fall has helped soothe my nerves and has kept me excited to meet my new online friends in person.
Leveraging technology to connect incoming students with professors, alumni, and their peers offers them a unique opportunity to integrate into the campus community and start building those important connections from the comfort of their smartphones.
2. Give students a peek into the future
Another great engagement technique is hosting live virtual events. Zoom, Facebook, Instagram and YouTube all have features that allow for live streaming, enabling a large group of students from anywhere in the world to interact in real-time.
This summer, Notre Dame University hosted live Zoom calls where incoming students were able to meet with each other, professors, and current students in order to get a better idea of what life would be like on campus. When I talked to my classmate about her decision to attend Notre Dame in the Fall, she told me that these virtual live streaming events helped solidify her decision to attend because they made her feel like she was already a part of the school community, reaffirming that Notre Dame would be a perfect fit for her.
New York University took a similar approach, offering incoming students a glimpse into what their future at the school would look like through their Student Showcase series. In program-specific Zoom calls, current NYU students discussed what they were up to at the university, including class projects, research programs, and extracurricular activities.
My classmate, who was deciding whether NYU was truly the school she would fit best at, told me that hearing from current students, who had been in her shoes only a few short years before, made it easier for her to visualize herself as a student there. It not only eased her nerves but also gave tangible moments she could look forward to if she chose to attend NYU in the Fall.
Discover how the University of Michigan leveraged virtual events to increase applicant engagement
3. Focus on one thing at a time
The journey from high school to university can be incredibly overwhelming for all sorts of reasons. The neverending to-do list – from registering for courses, to managing finances, to potentially moving to a new place – provides a full summer’s worth of work.
Gentle reminders from a school to take things one step at a time can make a huge difference. It’s easy to feel lost with all the different tasks that need to get done, and guidance from a knowledgeable source (the school) can ease the transition from high school senior to university freshman.
Try mapping out a simple timeline or checklist for students, and plan your outreach and due dates accordingly. Not only will this help your incoming students feel supported throughout the summer, it will give them a taste of the level of support they can expect over the next four years of college.
Checklists are a great way to reduce applicant stress and make your admissions process more accessible!
Discover another 6 simple ways to attract applicants by improving accessibility.
4. Connect with parents and guardians
Parents and guardians play a key role in guiding students through their college decisions. Keeping parents and guardians involved throughout the summer can help provide them with the peace of mind they need to confidently send their child off in the Fall.
Often busy with their own multitude of responsibilities, they don’t need as many touchpoints as the incoming students, but simple and concise communications can be very effective. Offer parents the opportunity to ask questions, let them know how you’re engaging their child, or connect them with other parents so they can begin to feel like part of the school community as well.
While summer melt can be a persistent issue for any school, good communication is what my fellow incoming class of peers and I have found to be the differentiator when it comes to engaging us in a way that makes us feel a sense of belonging. Preventing summer melt is a marathon, not a sprint, and keeping the lines of communication open throughout the summer is the best way to keep students, and their support networks, engaged all the way to the finish line.
Within the field of higher education admissions, few disciplines are as challenging to recruit for as pharmacy. With many distinguished programs across the country and only a limited number of students invested in pursuing a pharmacy degree, admissions teams need to ensure that they’re constantly finding new and improved ways to reach and engage potential applicants.
For the University of Michigan College of Pharmacy, a top-three ranking school in the United States, in-person events and on-campus conversations played a significant role in that strategy. From campus tours to peer lunches, these events helped prospective students get a feel for the school’s culture.
When the COVID-19 pandemic forced schools to cancel all in-person events, the admissions team needed a way to pivot online without sacrificing the key components of their applicant experience.
Luckily, U-M Pharmacy’s Assistant Dean for Student Services and Admissions, Mark Nelson, had been introduced to Kira Talent.
Selling the on-campus experience
“Before Kira, our process consisted of a team of faculty and student services staff who would review written applications before inviting select students to an on-campus interview,” shared Kristin Heinrich, the Assistant Director of Admissions and Recruiting at Michigan College of Pharmacy. “Running over the course of an entire day, applicants would have their interview with faculty as well as current students, but most of the day was spent learning about Michigan Pharmacy.”
Applicants were given tours of campus including Michigan Medicine where they’d have most of their rotations. Current students mingled with applicants over lunch, inviting them to ask questions, get advice, and engage in more casual conversation.
“The whole process really allowed applicants to get a sense of the atmosphere here on campus and decide if Michigan was the best fit for them,” explained Heinrich.
Over the following weeks, Heinrich and her team would review notes compiled from the various interviewers and would combine those scores with the other components of the application in order to reach an informed decision.
“At the end of the day, as admissions professionals, we just really want to make sure that students are finding the right fit,” Heinrich continued. “The on-campus experience was crucial to our interview process and to our yield process in general. It was definitely a big flip for us to go virtual.”
Discover how Purdue University made campus visits more accessible to applicants
Bringing the college atmosphere online
A primary focus for Michigan Pharmacy was how to provide a professional and efficient virtual interview process, while still being able to give applicants a taste of life at the college.
“As we started to explore our options, Mark introduced the rest of the team to the Kira platform,” shared Heinrich. “After connecting with the team at Kira and seeing the functionality of the platform, we decided that this was the best option for us.”
In previous years, the college interviewed around 100 students per cycle. “Coordinating Zoom interviews for all those students would have been an immense task,” explained Heinrich. “We really wanted to maintain the sense of ease and simplicity that we cultivated with our on-campus visits, and Kira allowed us to do that.”
Michigan Pharmacy engaged the university’s creative team to help film their welcome and closing videos and video questions, which were used to create an engaging and fully branded experience in the Kira platform. “The incorporation of introduction and conclusion videos created a very sleek experience for our applicants,” shared Heinrich.
The entire onboarding process, from the first discussion through to the launch of the assessment, took a total of about three months.
“I was shocked how quickly we were able to get it all going, given how many different components we had in the assessment,” shared Heinrich. “I credit that to how responsive and engaged the Kira team was.”
“Whatever components we wanted, whatever changes we wanted to make, our Client Success Manager was always quick to respond and happy to help.”
Discover what schools thought about their experience with Kira Talent
Increasing efficiency to connect with applicants
“Through Kira, our process has become more efficient,” shared Heinrich. “In terms of coordination, it’s more efficient in that I don’t have to do the manual work of organizing reviewers’ schedules and making sure that they show up on the right day at the right time.”
The on-demand nature of Kira’s Asynchronous Assessments means the college increases efficiency on the reviewing end as well. No longer needing to work around everyone’s individual schedules, Heinrich was able to assign more applicants to each reviewer, creating room in the schedule to hear from more applicants.
"Kira increased the number of students we interviewed by 53%."
With no minimum qualifying threshold for GPA or test scores, Michigan Pharmacy assesses applicants holistically, weighing cognitive and non-cognitive variables equally. Opening up the interview to more applicants has helped strengthen the school’s commitment to holistic review by enabling them to get a better look at more of their applicants prior to making a decision.
“With on-campus interviews, we had to be a lot more selective with the candidates we invited to interview, as we only had so much room in the schedule,” explained Heinrich. “This year, we had much more flexibility to invite applicants to complete a Kira assessment.”
“If our admissions committee was on the fence about an applicant after reviewing their written application, we were able to gather more data points about the applicant before making a decision,” she continued. “It helped us make more informed admissions decisions.”
Learn how Northeastern Pharmacy is using Kira to assess applicants' in-clinic skillset
Building equity into the admissions process
“Kira has allowed us to maintain consistency and fairness across our process, despite all the turbulence of the past year,” shared Heinrich.
Invested in fostering diversity, equity and inclusion in the higher ed space, Heinrich and her team were thrilled to see how the addition of Kira prioritized fairness in several different parts of the school’s admissions process.
“Kira has removed financial barriers for our applicants.”
“When we held on-campus interviews, we provided lunch, but the cost of transportation and accommodations fell on the applicants," Heinrich continued. "Kira has removed that financial barrier as well as the time-cost barrier associated with moving their classes or work schedules in order to attend the interview.”
From a reviewing standpoint, the platform has also fostered an equitable process by maintaining consistency across applicants.
Using timed video and timed written responses, Kira’s Asynchronous Assessment ensures that every applicant gets the same preparation and response time. By removing the influence of the reviewer, the platform also helps to mitigate potential biases.
“Within our old process, reviewers would often ask additional follow-up questions or clarifying questions during the interview,” shared Heinrich. “And while that’s very natural in an interview, it unintentionally invites a lot of bias into the process.”
“When reviewers ask follow-up questions, they give the applicant insight into what they’re looking to hear and give them the opportunity to expand upon their answer accordingly,” she explained. “The problem is that these follow-up questions are not consistent across reviewers or across interviews. In situations where they don’t ask the same follow-ups, the reviewer may walk away with the impression that that applicant had a less comprehensive or insightful response.”
“Our process with Kira eliminates those biases, giving each applicant the same fair shot.”
Keeping applicant schedules top of mind
With Kira, Michigan Pharmacy can offer applicants an interview that can be completed without causing any disruption to their schedules. By enabling applicants to complete their assessment on their own time, Kira removes a substantial obstacle that often prevents applicants from following through with the interview.
“In previous years, our interviews were always on Thursdays,” shared Heinrich. “If an applicant couldn’t make it on a Thursday there was a lot of work in order to figure out how to get the student to campus and interview them.”
“With Kira, students can complete their assessment at whatever time and place is most convenient for them,” she continued. “It gives our applicants a lot of flexibility.”
Engaging applicants through online experiences
“Finding different ways to engage the students is always important,” shared Heinrich. “We wanted to make sure that with Zoom fatigue and everything else, applicants had many different ways in which they could communicate with us.”
In previous years, the college leveraged on-campus events as a way to engage applicants and get them excited about the possibility of enrolling at the college.
“It was a crucial part of our recruitment process,” shared Heinrich. “So this year, we re-created the key elements online.”
Michigan Pharmacy organized and hosted several events running throughout the admissions cycle in order to capture the attention and interest of prospective applicants.
“We centred a lot of our outreach on social media and had current students host Q&As, which helped us meet the applicants where they are,” shared Heinrich.
More formal information sessions were also scheduled for applicants, allowing them to interact with the student services team, as well as current students at various stages of their degrees. “Applicants could ask questions about the student experience, covering everything from the lack of parking to what rotations are like, to how important it is to connect with faculty,” explained Heinrich.
“We were able to give applicants a glimpse of their future at Michigan, from being an applicant all the way to an alumnus” shared Heinrich. “All it took was a little creativity and a willingness to adapt our traditional events to get the most out of the online environment.”
“By taking our process online, we’re able to reach more applicants in unique ways,” Heinrich continued. “And our process with Kira allows us to engage more of those applicants in a way that’s efficient, fair and professional.”
Discover how more pharmacy programs are enhancing their admissions processes with Kira Talent
Today’s admissions leaders are faced with a difficult challenge. Pressure to meet enrollment targets is increasing just as technological innovation is opening the doors to all sorts of non-traditional educational opportunities. For colleges and universities, a deeper look at your admissions process to see how unidentified accessibility barriers are blocking candidates from applying to your school could uncover a significant group of underserved applicants.
In the United States today, one in four adults have a disability and one in five live with a mental illness.
“Having accessibility-related communications makes your school seen by our often untapped community,” explained Alexa Soares, a current MBA student living with depression.
“Many students with disabilities and mental health problems hold back from applying to colleges because they don’t feel that they fit the description of what the school is looking for,” added Jimena Vergara, a current MBA student living with anxiety.
Without the proper admissions tools and strategies in place, schools are inadvertently creating barriers for these applicants and, as a result, are missing out on a large pool of talented applicants.
“There is a huge movement surrounding inclusion in higher education that’s happening right now and accessibility is a part of that,” added Laura Chavira Razo, a current MBA student who is hard of hearing. “Schools that don’t realize this and adjust their processes accordingly are going to be seen as outdated and antiquated. Innovative, forward-thinking schools will be the ones that attract top students.”
The Higher Ed Admissions Accessibility Report contains the latest research and insights from leading voices in the higher ed admissions and accessibility spaces, as well as first-hand experiences of students with accessibility needs as they navigated various admissions processes. These doctors, business leaders, engineers, and philanthropists lend insights into how schools can reach a wider range of talented applicants with just a few adjustments to their existing admissions processes.
Based on these discussions and findings, this report provides seven simple ways to reduce accessibility barriers in your admissions process - helping you attract a diverse pool of differently-abled applicants.
For the University of Detroit Mercy School of Dentistry, the advantages of virtual interviewing are not a new discovery. Long before the pandemic made remote interviews a necessity, Detroit Mercy Dental was using them to help make more informed admissions decisions.
A private institution in the Jesuit and Mercy tradition, Detroit Mercy Dental not only provides students with a top-notch education and ample clinic experience but cultivates in its students a life-long commitment to serving the community.
In the past, the school relied on cognitive variables to assess and admit students. After reviewing GPA and Dental Admissions Test (DAT) scores, successful applicants were invited to an in-person interview. But as these cohorts progressed through the program and landed on the clinic floor in their third and fourth years, Detroit Mercy Dental noticed that while students were academically very strong, several were missing the mark with their interpersonal skills.
Discover how to redesign your admissions process to address the era of grade inflation
“We knew we needed more data points beyond just the cognitive variables in order to make better admissions decisions,” shared Dr. Steven Chang, the Director of Admissions at Detroit Mercy Dental.
“Since 2016, Kira Talent has been helping us identify applicants who will succeed in both the classroom and the clinic.”
Trusting the experts
When Detroit Mercy Dental first decided to incorporate holistic review into their process five years ago, the admissions team briefly considered developing a tool in-house. But taking on the work – from design and building to maintenance, upgrades and user assistance – wasn’t feasible.
Instead, Detroit Mercy Dental turned to Kira Talent, where they were able to build a custom process that was backed by a full-service team with years of experience in the virtual admissions space.
“What we’ve learned over time is that you always want to go with the experts,” shared Dr. Chang. “Partnering with a company where this is their focus; this is their area of expertise, ensures you’re staying innovative.”
In the five years Detroit Mercy Dental has used Kira, the platform’s continually evolving features have added new depth to the school’s admissions process.
Discover how Kira’s newest features can enhance your admissions process
Through timed written and timed video responses, reviewers get an authentic picture of who applicants are off the page. And with the structured reviewing and on-demand availability of Kira’s Asynchronous Assessments, Detroit Mercy Dental has been able to streamline their process and reduce reviewer fatigue.
“It’s an easy and seamless process for our faculty and administration, as well as our applicants.”
“We really enjoy that we can assign the video responses to our reviewers and they can evaluate them at their leisure,” explained Dr. Chang. “All they have to do is click the rubric answers to generate applicant scores, and that has been so helpful in making things as straightforward as possible for our reviewers.”
Discover how Purdue University eliminated reviewer fatigue in the admissions process
A twofold approach for optimal success
Receiving over 2000 applications per cycle, the Detroit Mercy Dental admissions team invites every applicant who meets their minimum cognitive thresholds to complete a Kira assessment. From there, the team invites between 33% and 40% of applicants to attend an in-person interview*.
“It’s a twofold approach,” explains Dr. Chang. “An applicant's Kira results are an important part of assessing the whole candidate before the actual interview.”
The value of a Kira assessment for Detroit Mercy Dental is in the structured review, consistency of process for applicants and reviewers, and bias-mitigating features like the inter-rater reliability metrics. With these safeguards in place, reviewers are able to interact more freely with their applicants when it comes time for their live interviews.
“As much as we may try to maintain consistency across applicants during live interviews, it’s virtually impossible,” shared Dr. Chang. “By having Kira already provide a structured, consistent and unbiased assessment of the applicants, our live interviews are able to be more open-ended.”
Once applicants are invited to a Kira assessment and interview, they’re assigned to reviewers who follow them throughout the process. Once assigned, reviewers receive the applicants’ written applications with the cognitive scores masked.
“By keeping the GPA and DAT scores hidden until after the live interview, we reduce the potential for Halo bias,” explained Dr. Chang. “We want to create a level playing field and ensure that we’re really measuring something different through these interviews.”
Learn more about Halo bias and the nine other types of admissions bias
By assessing their written application, followed by their Kira assessment and their live interview, the reviewers can establish a comprehensive view of the applicant. When it comes time to make a final decision, there are multidimensional data points on which the applicant is assessed.
A selling point rather than an obstacle
When Detroit Mercy Dental first implemented Kira, there was some concern that applicants would be deterred by an added step in the process. However, over the school’s five years with Kira, they have not only seen an increase in applications but see less than 1% of applicants fail to complete their Kira assessment.
“It’s certainly not a deterrent,” Dr. Chang shared.
“Applicants jump at the chance to express who they are as a candidate.”
“And that’s true now more than ever,” he continued. “Over the past year of this pandemic, students have become accustomed to speaking into a camera. Any barrier that it may have presented before, in terms of discomfort or nervousness, doesn’t exist now.”
Discover how Des Moines University is reaching a new generation of applicants
Finding the diamonds in the rough
Pulling competencies from the AAMC Anatomy of an Applicant, Detroit Mercy Dental worked with the Client Success team at Kira to develop a pool of questions that focused on measuring applicants’ non-cognitive skills.
“We encourage reviewers to think about whether this is a student they would be excited to see in the classroom. Is this a student they’d be excited to see on the clinic floor?” shared Dr. Chang.
“With Kira, we’ve found that there are so many candidates who didn’t have the greatest cognitive score on their application, but when we look at their Kira scores and their interview scores, they’re exceeding expectations.”
“These are the candidates who the faculty are excited to teach," Dr. Chang continued. "They’re excited to mould them and help them become great dentists because they can see that they have great potential.”
“We don’t want to miss good candidates that may not show up as well on paper,” shared Dr. Chang. “A big part of succeeding in dentistry is patient interactions. Do they have empathy? Do they understand what it’s like to interact with people who are different than they are? These are things you can’t assess through grades and standardized tests.”
Download Kira’s comprehensive guide to holistic review
“Not all students have to have a 4.0 GPA. Not everyone can and that’s okay,” shared Dr. Chang. “As long as they meet the threshold that tells us they can successfully navigate the curriculum, what we really care about is their passion and their soft skills. These are far better indicators of how successful they will be as a healthcare provider, and ultimately, our goal is to graduate the best dentists.”
*a previous version of this post erroneously reported 85% are interviewed.
As the largest parish day school in the country, Holy Innocents’ Episcopal School enjoys a robust and active community of students, parents and faculty. Serving students from PK3 through twelfth grade, the school fosters a diverse and inclusive environment in which students can develop a genuine love of learning.
Placing a strong emphasis on an applicant’s respect for self and others, and sense of service to the global community, Holy Innocents’ has long incorporated holistic practices into their admissions process. Through one-on-one interviews with each applicant, the admissions team ensured that incoming students truly exemplified their school values.
When faced with creating a completely virtual admissions process, Holy Innocents’ needed a platform that could deliver the same depth of insight into their prospective students, while being completely remote.
“We needed to interview our applicants in a safe manner that was not going to be cumbersome to families,” explained Dr. Beth-Sarah Wright, the Director of Enrollment Management at Holy Innocents’ Episcopal School. “Thankfully, we had heard about Kira Talent.”
Discover where Holy Innocents is seeing their success:
- A seamless transition to virtual interviews
- Getting in-depth applicant insights by asking the right questions
- Bringing out the best in applicants and providing peace of mind to parents
- A solution perfectly designed for busy families
- Redefining fairness in the admissions process
- Staying ahead of the curve through innovation
Making a memorable admissions experience for applicants and their parents
“Prior to Kira, we conducted everything in-person,” shared Enrollment Data Manager Katie Patrick.
Using Kira to assess incoming students for grades four and up, the admissions team created completely custom assessments suited to each age group. By crafting questions, competencies and rubrics that were designed to effectively and fairly evaluate prospective students, Holy Innocents’ was able to recreate their traditional process in a virtual format.
Traditionally, for middle and upper school applicants – students in grades six through twelve – the in-person, one-on-one interviews centred around the school’s mission statement. Applicants were asked to share their understanding of its meaning and give examples as to how they exemplified those traits in their day-to-day lives.
With the fourth and fifth graders, the admissions team took a slightly different approach. Applicants had a one-on-one sitdown with a teacher where they worked through a variety of homework-style problems. In addition to academic aptitude, these exercises were meant to showcase an applicant’s problem-solving and interpersonal skills.
With a combination of timed-video and timed-written questions completed on the Kira platform, Holy Innocents’ was able to effortlessly transition the assessments for their lower, middle and upper school applicants online.
“With Kira, we ask applicants some get-to-know-you questions, followed by some open-ended questions which help us assess their soft skills,” Patrick shared. “Our fourth and fifth graders are then also given some skill-testing math questions and reading passages to assess comprehension and fluency. The math and reading comprehension exercises are the same for all applicants, but the get-to-know-you and competency-based ones are randomly selected from a bank of questions that we created based on the student’s grade level.”
An added benefit of the Kira assessments is the opportunity for Holy Innocents’ to showcase what makes their school culture unique. Questions are presented in videos pre-recorded by teachers, allowing the applicants to feel like they’re being welcomed into the school’s community.
“In a traditional interview, applicants would only meet one interviewer and that would be their touchpoint to the school,” explained Andrew Mack, the Assistant Director of Admissions at Holy Innocents’ Episcopal School. “With Kira, applicants are introduced to four or five of the teachers who make up our school community. It gives them a much better sense of what it would be like to be a student here.”
See how more schools are using videos to attract applicants
“With Kira, we’ve not only replicated our process, we’ve enhanced it,”
“The overall experience is more engaging for our applicants, and Kira has helped us ask more focused and nuanced questions that, while still geared towards our mission statement, enable us to dig a little bit deeper with the students,” Patrick added.
Asking the right questions to find the best students
Backed by a decade of experience, the Kira team helped Holy Innocents Episcopal School elevate their interviewing process, starting with the questions they asked applicants.
“Kira taught us about the types of questions to ask,” explained Dr. Wright. “The team showed us how to ask a question in a way that really encourages the applicant to think critically, and to respond in a way that reflects their true feelings and authentic experiences.”
“The questions that Kira helped us create were exactly what we’ve always wanted to ask, but were never able to figure out how to phrase properly,” added Patrick.
“We didn’t know exactly what to expect at the beginning, but working with Kira and building these assessments was just a joy and a pleasure,” Dr. Wright continued. “Kira gives us a pretty good picture of our students, who they are, what kind of student they are, what kind of citizen they are, and whether or not they would be a good fit for our school. And that helps us make informed and insightful admissions decisions.”
Creating a secure environment for kids to put their best foot forward
“Kira brought peace of mind to the online interviewing experience that we couldn’t have achieved with Zoom,”
“We were thrilled that it wasn’t going to be just another Zoom experience for students," continued Patrick. "Our applicants are as young as third graders, and as we don’t allow parents to participate in the assessment, the children were doing this completely independently.”
Kira’s straightforward, user-friendly design made it easy for their young applicants to complete the assessment on their own, while the all-in-one-platform structure ensured that they were safe while doing so.
“The parents really treated it as a professional experience for the children,” explained Dr. Wright. “We quite explicitly said no help from parents outside of technical set up, and parents really responded to that. Kira helped us set the right tone for the interview, so parents were at ease putting their third or fourth graders in a room with their headphones on to complete the assessment.”
Kira’s structured process, including timed responses, automated transitions and simple, clear instructions, helps keep the assessment on track without requiring assistance from an adult.
By preemptively flagging applicants who require attention, Kira has also helped instill confidence in the integrity of the processes.
“Kira alleviated significant stress from the admissions team.”
“Knowing that the platform was built specifically for admissions helped tremendously. There were no loopholes, no potential cracks in the process that we had to fill, no way to finagle the system,” explained Patrick. “Everything just automatically works as it’s supposed to.”
By creating an engaging and inclusive environment where applicants feel empowered, Kira assessments help give the admissions team a more authentic glimpse of each student and their unique personality.
“Extraverted students still thrived in their Kira assessment, just as they would have in person. But for the introverted students, Kira made a world of difference,” shared Mack. “With the in-person interviews, students were meeting their interviewer for the first time and were immediately having to answer all kinds of questions. With Kira, they’re in their own homes, with familiar people. They’re more comfortable, more confident, and it shows.”
Offering flexibility and ease to busy families
"With any new technology, there’s always some nervousness around an unfamiliar process,” shared Mack. “But once they got into the platform and understood how flexible and easy it is to use, the families and staff members were able to appreciate how useful the platform is."
The on-demand format of Kira’s Asynchronous Assessment means families don’t have to rearrange their busy weekend schedules to accommodate an on-campus interview. Applicants can complete their Kira assessment wherever and whenever they're ready to focus and put their best foot forward.
“The feedback we got from the applicants and their families was overwhelmingly positive.”
“People really appreciated the latitude with which they could complete their assessments,” Patrick explained. “In-person interviews can be extremely difficult for families to get to with their other responsibilities. Whether it’s extracurricular activities, family engagements, or jobs, it’s tough to find the time, especially if you’re applying to multiple schools or juggling the schedules of several children. Families have really appreciated the flexibility that Kira offers.”
Fostering a fair and consistent process
In addition to bringing a new level of ease to applicants, Kira helps Holy Innocents Episcopal School create an inherently fairer process.
In previous years, the admissions team had noticed how susceptible applicants were to the external circumstances surrounding their interview. Whether it was the decorations in the classroom, the noises from outside, or the interviewer’s mannerisms, seemingly inconsequential factors could tangibly change whether the child opened up, or shied away. By reducing those external influences, Kira has helped create consistency.
“In an in-person interview, applicants may be inadvertently swayed or guided by an adult in the room,” Dr. Wright explained. “Kira lets them respond on their own, without the influence of another person’s reactions or expressions coaxing or hindering their answers.”
Through a structured review process that includes a predetermined set of competencies and rubrics, Kira Talent also brings a level of consistency to the review process that helps the team mitigate bias.
“Our in-person interviews were one-on-one, so we were relying solely on the feedback from that one reviewer,” Dr. Wright continued. “With Kira, we can get multiple eyes on each student, and all the evaluations are completed individually, so that helps us create a fair and consistent process that reduces biases.”
“Kira’s inter-rater reliability metrics allow us to see how our reviewers scored the applicants; who tended to score more harshly and who was being too generous,” added Mack. “With the help of Kira, we’re able to gauge what a true and fair evaluation for each student is.”
Creating an innovative process for a new generation
With technology being a part of their lives earlier than ever before, students today are more attuned to the experience of representing themselves in a digital format. From video games to social media, navigating online experiences is practically second nature.
Far from being a barrier, this generation of students often feels more comfortable sharing who they are through video. Going forward, Holy Innocents’ sees Kira as a prime opportunity to keep a new generation of applicants engaged throughout the admissions process.
Discover how The York School is leveraging innovative tech to attract applicants
“We’re looking at how we would combine the Kira assessment with an on-campus visit,” shared Patrick. “As Kira is constantly evolving and offering more capabilities, it’s a way for us to keep our admissions process innovative.”
“Kira was such a good investment for our school.”
“We still don’t know exactly what next year will look like," continued Dr. Wright. "But even as restrictions are lifted we plan to keep Kira as part of our process.”
With only 31 veterinary schools across the United States, seats in the top programs are highly coveted. Out of the 1600 applicants who apply to the College of Veterinary Medicine at Purdue University Doctorate of Veterinary Medicine each year, 84 talented students are offered a spot.
“Given our small class size and the high demand for a seat, it can be very difficult to make selections,” shared Lori Stout, the Director of Admissions and Recruitment at Purdue University’s College of Veterinary Medicine. “That’s why we try to use as many diverse criteria as possible to help make that decision.”
In years past, Purdue used on-campus interviews to evaluate applicants’ soft skills, traits, and passion for veterinary medicine. Taking place over a span of two days, applicants were awarded a 25-30 minute interview with a program faculty member. When admissions processes were forced online, Purdue knew that forgoing interviews would be detrimental to the integrity of their selection process.
With the help of Kira Talent, the college replicated their interviews virtually, bringing flexibility, consistency, and equity into the process. Looking forward, Purdue wants to build on this momentum, leveraging Kira as a way to continue strengthening admissions.
Discover where Purdue is seeing their success:
- Creating an applicant-first admissions process
- Effortlessly transitioning online
- Building a consistent and equitable review process
- Eliminating reviewer fatigue
- Leveraging inter-rater reliability data
- Getting applicant buy-in
An easy choice for admissions teams
“We met with our admissions committee, and with the value of what Kira brings to our process, it absolutely doesn’t make sense to give it up,” shared Stout. “We want to build on what we’ve learned using Kira and tweak things to make this year’s interviewing process even better.”
Although Stout and her team are expecting campuses to be open next year, by keeping their interviews online they aim to reduce barriers for applicants in the admissions process.
Discover why Des Moines University is also continuing with virtual interviewing
“We typically invite 300-350 applicants to interview for our program,” explained Stout. “Before, those applicants would have had to travel to campus for their interview day. Now, these applicants are invited to complete a Kira assessment.”
“In veterinary medicine, one of the biggest issues that we’re facing, on a national level, is the education debt load,” Stout continued. “If we, as a field, are so concerned about student debt load, why are we asking them to incur all the costs associated with these interviews?”
For many schools, on-campus interviews have a twofold task of assessing applicants while also showing off program facilities and campus culture. But with most applicants applying to programs across the country, interviews can easily stack up. The associated costs, from flights or gas to hotels to time off work, can easily reach thousands of dollars. And given the competitiveness of the field, it’s not uncommon for applicants to pay for a visit to a school they won’t get an offer from.
Instead, Purdue is inviting applicants to visit campus after they’ve received an offer. The “offered student day” is extended to applicants who would like to visit the college and the campus before accepting their seat in the program.
“Our goal is to try to save them money upfront,” said Stout. “Once they’ve received an offer, they can choose to come to campus and see everything we have to offer.”
Eliminating the risk of technical difficulties and logistical nightmares
Before Kira Talent, the work of coordinating and scheduling interviews fell on Stout.
“We would send applicants their invitation via email with a link to a website that we had built and ran internally,” explained Stout. “From there, they were directed to choose between AM or PM, and then we’d assign them a slot.”
Between monitoring the website for glitches, assisting applicants through the process and accommodating requests for interview changes, Stout was not only conscious of the time it took, she was also worried about the likelihood of something slipping through the cracks.
“I had this huge spreadsheet that organized interview scheduling,” she explained. “As I assigned slots and made edits, I’d have to be so careful not to accidentally double up or delete someone."
"The margin for error was greater than I wanted to have.”
When it was clear that interviews couldn’t happen on-campus, Stout’s greatest fear was how she was going to bring the whole process online.
“I was thinking we were going to have to use Zoom, and that sounded like a logistical nightmare!” she shared. “I was not only apprehensive about the level of planning and coordination that would take, I was also worried about the risk of technical difficulties.”
“Not everyone involved in the admission process is technologically savvy,” Stout continued. “If Zoom were to cause issues, I wouldn’t be able to fix it and I wouldn’t have anyone to call for assistance. We would have been at the mercy of something we had no control over.”
From inviting applicants to assigning reviewers and tracking their progress, Kira unified the team’s tools and processes into a single online platform.
“With Kira, it’s as simple as clicking a few buttons,” shared Stout. “Having the manual work done for me makes everything so much easier.”
Redefining success with the help of expert advice
Prior to Kira, interviews at Purdue’s College of Veterinary Medicine centered around a list of required and optional questions. Using a 15-point scale equated to letter grades (a 15 equaling an A+), interviewers assigned applicants an overall score. “It was very difficult to enforce consistency within that system,” said Stout. “We couldn’t be sure everyone was asking the same thing or defining and scoring the same way.”
When introduced to Kira, Stout was excited by the equity it would bring to their admissions process.
“The Kira team helped transform the entire process. They helped us come up with more specific competencies to assess for, and helped us build structured rubrics to reduce the potential for bias,” Stout explained. “Looking back, that’s what we were lacking in our old process. When you don’t have a structured process in place, you can’t have everyone on the same page.”
Kira helped Purdue’s admissions team establish a focused set of competencies that could be defined on a rubric so that each applicant can be evaluated by multiple reviewers independently and consistently.
“During our onboarding, we did an exercise with our Client Success Manager (CSM) to identify our program’s values and which competencies were most important in successful students,” shared Stout. “Our CSM took those and came back to us with an ‘interviewing mission statement’. It was exactly what we wanted to accomplish but hadn’t been able to put into words."
"To know that the Kira team truly understood what would help us succeed – that was an exciting moment.”
The process helped Purdue create clear definitions, which in turn helped reviewers understand what to look for in an applicant’s response and how to evaluate it.
“Everything was clearly defined, not arbitrary, not subject to interpretation.”
Read more about Kira's client experience
Creating a process reviewers love being a part of
The simplicity of the Kira platform improved the process for Purdue’s reviewers as well. Because they were reviewing recorded responses, reviewers were able to evaluate their assigned applicants on their own schedule.
“I was worried I might have to push reviewers to get their scores in before the deadline, but that wasn’t an issue at all,” shared Stout. “Everyone completed their applicant reviews before the deadline, and I think the process being so convenient played a large part in that.”
In fact, Stout expects to see a rise in the number of volunteers for next year’s reviewing cycle.
“People who volunteered this year talked about how easy and convenient it was,” she explained. “They could sit down, at home, and complete a couple each night. So for next year, potential volunteers know that they don’t have to sit through a whole day of back-to-back interviews.”
Prior to using Kira, Purdue’s interview days ran from 8 am - 5 pm. By the end of the day, reviewers were, understandably, exhausted.
“I’ve had some interviewees tell me that their interviewer was disengaged,” shared Stout. “And it’s not because they aren’t interested, but by the afternoon of the second straight day, they’ve been listening to similar responses over and over. As hard as they try to stay engaged, reviewer fatigue is a real and inevitable obstacle.”
Discover what applicants thought of their experience with Kira Talent
This year, Purdue was able to offer reviewers a far less demanding schedule. By giving a timeframe in which they had to complete their evaluations, Kira allowed reviewers to work when it suited them best. “They could do things on their own, at their own pace, and when they knew they were in the right headspace and could give the applicant their full attention,” explained Stout.
Training reviewers based on data, not opinions
In addition to reviewer training during the onboarding process, Kira provides inter-rater reliability data to help admissions teams identify if and when a reviewer deviates from the average.
“The inter-rater reliability reports gave us concrete guidance on who we needed to reach out to and what we needed to work on.”
“Before Kira, I could only rely on feedback from applicants or my own personal estimations," shared Stout. "I didn’t have a full picture of how we were doing as reviewers, or if we were being fair to everyone.”
The admissions team now shares these reports with their reviewers, giving them active and concrete feedback on their work.
“It gives them an opportunity to re-assess how they evaluate an applicant, if they missed something, or if they saw something one way and could look at it differently. Before it was all arbitrary. If the applicant was good, they gave them a good score, but what score that was exactly really depended on the reviewer.”
Keeping applicants' best interest top of mind
As Purdue moves into its second year with Kira Talent, Stout and her team are making a point to share with applicants why the college is continuing with Kira Talent over in-person interviews, and how that benefits them.
“Firstly, we’re trying to save them money,” said Stout. “The cost of veterinary school is high enough. We don’t need to add to that by having them travel to all these interviews.”
“Secondly, this process helps us form an unbiased view, and that ultimately benefits the applicant. Their success isn’t determined by getting an interviewer with whom they connect because they’re from the same hometown or they have a dog with the same name. With Kira, we have a holistic view of every applicant, which means everyone is assessed on an equal playing field.”
For medical schools, Multiple Mini Interviews (MMIs) are a staple of the admissions season. A reliable method of assessing soft skills such as communication and critical thinking, MMIs provide insight into how an applicant would interact with patients and colleagues as a healthcare professional. For the Psychiatry Residency program at Queen’s University’s School of Medicine, this assessment of soft skills is integral to the admissions process.
With approximately 30 residents spread across the 5-year residency program, Queen’s offers students a balance of hands-on in-hospital rotations and educational research opportunities.
“We pride ourselves on producing really well-rounded psychiatrists,” shared Dr. Eric Prost MD, FRCPC, an Assistant Professor at Queen’s School of Medicine. “Our students have both research and clinical experience. They’re just as comfortable tackling scholarly projects and they are attending to any patient who walks in the door.”
So when Queen’s shifted their admissions process online, they needed a platform that could deliver the effective and engaging experience they wanted for their applicants, as well as the support and guidance they needed for their admissions team. They found their solution with Kira's Live Interviewing.
Recognizing the value of a virtual MMI
“Even before the pandemic there was a push within the School of Medicine to transition to virtual,” explained Dr. Prost. “The travel cost related to in-person interviews, both in terms of the effect on the environment as well as the financial cost to applicants, was something we were keen to address in our process.”
Excited as they were by the prospect of providing an accessible and environmentally conscious interviewing experience, the admissions team disliked the impersonal, and often problematic, nature of many off-the-shelf solutions.
“We needed a platform that would perform flawlessly and a virtual interviewing experience that would help us attract and enroll high-quality candidates. With Kira, both of those things happened.”
“If we were going to do it virtually there were two key factors that needed to be addressed,” explained Dr. Prost.
The first was the support behind the platform. “We didn’t want to be dealing with a bunch of IT problems,” shared Dr. Prost. “It’s not only extraordinarily frustrating for both the admissions team and our applicants, but it’s unprofessional and it tarnishes an applicant’s impression of our school.”
The second was the opportunity to market the program. “Part of the appeal of our program is the warmth and camaraderie of our residence group,” Dr. Prost continued. “Candidates often comment on how close-knit and friendly our residents’ group is. So we needed to be able to convey that virtually. Kira Talent met and exceeded those needs.”
Enhancing admissions from application to interview
With full confidence conducting their MMIs in Kira, Queen’s saw an opportunity to gain deeper insights into their applicants’ soft skills and traits earlier in the admissions process. In conjunction with their traditional written application components, Queen’s had applicants complete an Asynchronous Assessment in Kira. To help reduce bias in the evaluation process, each reviewer was assigned to review applicant responses for specific competencies. This horizontal review process, helps reviewers become experts in the competency they're assigned to, thereby improving the accuracy and reliability of their scoring.
“We want to be as objective as possible throughout each stage of the process,” explained Dr. Susan Ilkov-Moor MD, FRCPC, an Assistant Professor at Queen’s School of Medicine. “Adding the Asynchronous Assessment allowed us to hear from each applicant before making any decisions. By combining information from both the written application and the Kira assessment we ensured that we were making well-informed decisions based on a holistic view of all our applicants.”
Following their Asynchronous Assessment, successful applicants were invited to an MMI in Kira Talent. With Live Interviewing, Kira took Queen’s traditional MMI process and replicated it online. Through custom in-platform competency-based rubrics and seamless scheduling, the streamlined process enabled Queen’s to interview more applicants per day.
“Live Interviewing mirrored our traditional MMI and it did a great job of recreating the process,” shared Dr. Ilkov-Moor. “The whole process went extremely smoothly.”
The advantage of expert advice
By incorporating pre-recorded videos into the Kira platform, the program was able to showcase not only shots of campus but the personality and spirit of the faculty and residents’ group.
“We got feedback from an incoming student that they loved the opening video we had with Dr. Prost introducing the program” shared Dr. Ahila Vithiananthan BHSc. Hons., MD, a current psychiatry resident at Queen’s School of Medicine. “They also specifically commented on the quality of questions that Queen’s asked in the Asynchronous Assessment and the MMI, compared to some other programs. That’s largely due to Kira’s role in the creation of our interviews.”
Read more applicant feedback in the 2020 Applicant Experience Report
Guided by a decade of experience in the higher ed admissions space, Kira brings that expertise into the onboarding process. Kira helped the Queen’s Psychiatry Residency Program overhaul the rubrics and questions used in both the Asynchronous Assessment and MMI.
“Kira helped us create stronger questions and rubrics.”
“They were okay before but the competency workshop with Kira helped us create better ones,” shared Dr. Prost.
A golden standard for support
With Kira Talent already powering virtual assessments for the Queen’s undergraduate medical program, the Psychiatry Residency Program knew that Kira’s award-winning client experience would be an added support for their admissions team.
Discover what admissions teams have to say about their experience with Kira Talent
“It was a significant benefit for us to have the help of the Kira team,” shared Dr. Vithiananthan. “Kira was amazing throughout the onboarding process, always there to answer any questions, and was available during all the Live Interviews, on the weekend, supporting both the admissions team and reviewers, as well as the applicants.”
“The reason that we chose Kira was because we knew it wasn’t going to be us doing it on our own,” shared Dr. Prost. “Kira gave us an excellent online experience with no bugs and no IT issues. We could count on 24/7 support and a team to keep us organized and on track. And that was not only during the onboarding but throughout the assessment period and day-of for the MMI. I could not be happier, I have absolutely no complaints.”