“How was your experience using our product?”
This is one of the most important questions a technology company can ask. Especially when the technology is helping students on the path towards higher education.
For the past three years at Kira, we have reported on the responses we’ve received from tens of thousands of applicants who have used the Kira platform.
In this year’s report, you can scan both quantitative and qualitative feedback from applicants, read more about how we use feedback, and why we report on it.
- Year in review
- By the numbers
- By the comments
- How we collect and use feedback
- Why publish this data?
- Past reports
Year in review
As more and more programs add Kira to their admissions process, we had another record-breaking year in terms of applicant volume. Overwhelmingly, applicants continue to rate their Kira experience as positive and leave great suggestions for us in the comments.
One thing last year’s data doesn’t show is that we implemented a significant change to Kira’s technology.
In late 2019, the Kira platform officially moved away from Flash video recording across all assessments. The move to HTML5 recording enabled Kira to become a more flexible and accessible platform for applicants and admissions teams.
Though Flash was a dominant platform for online multimedia content earlier in the decade, it is being phased out across the web in favor of the more modern HTML5. We also successfully launched new technical infrastructure to better support applicants in mainland China.
Given the new infrastructure and evolution of HTML5 in the fall of 2019, many applicants have yet to experience this improved version of Kira.
Looking ahead, next year’s applicant feedback will be our first full calendar year to report on the impact of dropping Flash from our platform.
By the numbers
Over 67,000 applicants left feedback on their Kira experience from January 1, 2019 to January 1, 2020.
While the number of applicants who complete a Kira assessment increases every year, fewer applicants are leaving feedback. Given the number of completed applicants has risen year-over-year, this number came as a surprise to us.
In 2020, we’re updating the feedback survey in the hopes of attracting more users to leave their feedback and comments.
82% of applicants rated their Kira experience either “Great” (4/5) or “Exceptional” (5/5)
This is a consistent rating from past years.
51% of applicants rated their Kira experience “Exceptional” (5/5)
This is a consistent rating from past years.
The average rating overall was slightly above “Great” (4.3/5)
This is a slight increase from the past two years (Kira applicants rated their Kira experience at an average of 4.2/5.
1.5% of applications rated their experience as “poor”
This is a slight increase from last year (1.4% of applicants rated their Kira experience poor). While we were not able to bring this number down in 2019, we are confident with our move away from Flash, we’re on track to reduce this rating in 2020.
6.8% of applicants opened a support ticket while completing a Kira assessment.
Great news! This number has dropped slightly from last year and the year prior, meaning once again, fewer applicants required help from our support services.
89% of applicants who left feedback on support rated their experience as great. The options were great, okay, and not good.
By the comments
Reading daily feedback reports has always been a highlight for the Kira team. Let’s open up the black box and see what we learned from last year’s comments:
Applicants appreciated a different type of test
“I love this platform. It pushes students to do something out of the ordinary.”
“For me, it was a very nice challenge after 6 years of exams.”
“Easy to use and extremely innovative. Would love to see other universities implement this program!”
“I believe this was a great way for [redacted school] to use this platform and video assessment. The holistic approach is without a doubt one of the best ways to evaluate a candidate.”
We hear from admissions teams that Kira assessments help show a different side of their applicants beyond grades and test scores. It’s rewarding to hear from the number of applicants who feel the same way.
Applicants didn’t love Flash
We heard this feedback loud and clear and made phasing out Flash our #1 priority last year.
“You guys should stop using flash”
“Having to use Flash is annoying”
“Trying to get adobe flash to work was painful.”
We’re so happy to be Flash-free in 2020. We’re already seeing a significant improvement in feedback from applicants since making this change in 2019.
Applicants gave mixed reviews related to stress & anxiety
“The practice “I will do great on this assignment” helps people to get confident and comfortable and is a good self-fulfilling prophecy.”
“I think this online interview also takes away a lot of the nerves accompanies with Grad school interviews, and I feel it allows interviewers to see a real picture of prospective students by removing some of the “white coat” syndrome entailed in the interview process.”
“Using Kira takes away much of the stress from the actual interviews!”
“It’s friendly, but I needed more friendliness, I was completely nervous while recording.”
“Make it less stressful, maybe?”
As we continue to learn from feedback about the experience, we hear from some applicants who find the experience stressful and others who are relieved in comparison to an in-person interview.
While there’s no perfect solution, we continue to keep an eye on feedback like this.
Applicants left helpful feedback on assessments for schools
“Some questions beg more than a minute for an answer but maybe that’s for the interviewers to set?”
“I would like to be provided with practice questions with topics that were more similar to the actual interview questions.”
We pass feedback like this along to the school. Every year, we share anonymized comments from applicants to help each program improve their assessment for next year.
Because applicants do not want to hurt their chances at entry, admissions teams rarely get honest feedback directly from applicants on their process.
Applicants left us design ideas, too!
“I think the home check-in page could use a redesign”
Sounds good. We’re actually releasing a new look for this page this summer!
And even the occasional, unusual feature request…
“The platform is perfect but if it could put some filter to make the people who use the platform look more pretty it would be better.”
We appreciate this feature request, but you look great as is! No filter needed.
How we collect and use feedback
How we collect feedback
At the end of a completed Kira assessment, we ask applicants to rate their experience on a scale of poor to exceptional. On top of a numeric rating, applicants have the opportunity to share their feedback and requests anonymously.
This feedback report is based on data collected over the 2019-2020 calendar year from January to January.
How we use feedback
The feedback applicants share at the end of their Kira assessment is extremely valuable in three ways:
For schools: At the end of each year in an Insights Report, we share a collection of the most commonly appearing or notable, anonymized student comments with the admissions team. Applicant feedback around question content, preparation and response time, and video quality, get passed along to the school.
For future students: The feedback section of the Kira assessment is an opportunity for applicants to improve the product experience for future users.
For Kira: We need your feedback to continue to develop an applicant-first product. We bring new ideas, suggestions, and criticisms directly from these reports into our product planning.
Why publish this data?
To commit to continuous improvement: In 2020, we are doubling down on accessibility and inclusion. We want to reduce barriers students may face when completing a Kira assessment. Our mission is to make admissions fair. A key component of living that mission is ensuring all applicants are able to convey the unique experiences they’ve had to post-secondary institutions.
To celebrate the success of our partner schools and their applicants: For the third year in a row, we’ve toppled our past levels of completed Kira assessments. And once again, the feedback has stayed consistently positive.
To be transparent with prospective Kira teams: Any technology vendor that your applicants interact with will reflect on your school. If you’re considering Kira, you should know what the latest cohort of applicants has to say, and we should be comfortable standing by that feedback.