Introduction

Having been developed and evaluated in medicine,1,2,3 multiple mini-interviews (MMI) are being used increasingly as part of the admissions process for undergraduate dental students. In 2012 one dental school (Cardiff) utilised multiple mini-interviews as part of its student selection procedures and it is noted that in 2013-14 at least six UK dental schools will be using multiple mini-interviews as part of their admissions proces;4 Belfast, Bristol, Cardiff, Dundee, Glasgow and King's College London. Multiple mini-interviews have also been introduced as part of the selection process for dental nurses and dental therapy and hygiene students in Cardiff. A thematic evaluation and analysis of the multiple mini-interview process and its acceptability to potential dental students identified a number of emergent themes (lack of control, anxiety and nervousness, comparison with conventional interviews and preparedness) and a usefulness for the MMI as a tool to aid student selection.5 A quantitative evaluation of applicants' responses to the multiple mini-interview process was perceived as possible by analysing the responses to a suitably designed post-interview evaluation questionnaire by applicants attending for the Cardiff (2012) multi mini-interviews. The aims of this study were to identify any new themes for consideration during the MMI process highlighted by a post-interview evaluation questionnaire in a new cohort of undergraduate dental students and undergraduate dental care professionals (hygienists and therapists) and to compare and contrast the findings between groups. In addition, an objective was the further detailing and quantification of applicants' perception towards the MMI as a selection tool by analysing the responses from the different applicant cohorts.

Methods

This research involved two groups of students who applied to study at Cardiff Dental School; candidates applying for admission to the five- or six-year BDS courses and candidates applying for admission to the therapy and hygiene course. All candidates who secured a multiple mini-interview following the screening and scoring of their applications were invited to complete a questionnaire designed following the results obtained from an evaluation of the previous year's multiple mini-interviews.5 Candidates were given printed and verbal explanation of the purpose of the questionnaire. The questionnaire was divided into sections and covered basic demographic information, the quantitative assessment of questions relating to themes identified as important from the previous year's analysis, the quantitative evaluation of the multiple mini-interview stations and a free comments area. With reference to the quantification assessment of the stations used in the MMI, applicants were invited to grade the perceived usefulness of the station on a scale of 1 to 5 where the scores were represented by the following; 1 = very useful, 2 = useful, 3 = no feeling, 4 = not very useful and 5 = useless. These scores were used to evaluate the 'usefulness' or otherwise of the stations between the two groups participating in the MMI. Candidates were made aware that they had the right to decline to complete the questionnaire, in full or in part, and that if they did so this would have no bearing on the outcome of the multiple mini-interviews. They were also made aware that while individual written responses may be published deductive disclosure would not be possible from the assembled data. Ethical approval for this project was obtained from the Cardiff Dental School Ethics Committee.

Results

From the 235 BDS undergraduate interviewees and 62 therapy and hygiene interviewees, 231 (98.3%) and 61 (96.7%) completed or partially completed questionnaire responses were transcribed and coded into an SPSS database to assist analysis. For ease of reading and reference the results are relayed with the use of sub-headings.

Demographic detail

The demographic data (Table 1) revealed that 61.9% (143) of the BDS interviewees and 86.7% (52) of the therapy and hygiene interviewees were female. The majority of interviewees for both courses came from schools in England with 69.4% (159) and 83.3% (50) being noted for the BDS and therapy and hygiene cohorts respectively. In both interview groups it was noted that the percentage reporting as having received a state education was approximately 70% (68% for BDS interviewees and 70% for therapy and hygiene) and while 69.6% of the BDS interviewees had experience of an interview before their MMI, this was down to 58.3% in the hygiene and therapy interviewees. While the information is presented it should be noted that no statistical analyses based on the demographic data was undertaken in this project.

Table 1 Demographic data for undergraduate dentistry interviewees (BDS) and Diploma in Therapy and Hygiene interviewees (H&T) participating in the multiple mini-interviews held in Cardiff
Full size table

Responses to previously identified themes

Table 2 details the responses resulting from completion of the questions relating to identified themes. The table details the raw data/responses for BDS and DCP students (hygiene and therapy) as well as the results of binomial statistical calculations and chi-squared tests of independence of categorical variables on nominally scaled data; statistically significant differences are indicated. Where a choice between four descriptors had to be made (questions one to four, seven and nine), comparisons between the two more positive and two more negative statements were made within and between the results recorded. Where comparisons between five descriptor statements had to be made, analyses between the two positive and two negative statements as well as their comparison to the neutral (middle) responses were also made within and between groups.

Table 2 Responses to questions relating to the themes of importance as identified from previous analysis
Full size table

The first four questions covered the theme of 'opportunity'. All questions revealed statistically significant differences within and between the groups except for question three, which showed no statistically significant difference in responses between groups; the actual reasons cannot be identified without further research and analyses.

When comparison to conventional interviews was explored in questions five and six (in comparison to feeling the same), it was observed that the dental student group was identified as being more anxious (p >0.001) but that both groups indicated that the multiple mini-interviews helped the applicants deal with such concerns in a positive way (p >0.001).

With reference to preparedness (questions seven to eleven), question seven revealed that all applicants felt that the pre-interview information had little bearing on their overall comfortableness with the MMI process, with significantly less applicants feeling that the information provided was detrimental to the performance (question eight: p >0.001). The results also indicated that while both groups reported preparedness as being significantly important in relation to performance (question nine: p >0.001) both groups felt their overall performance to be better than they expected for stations that they could or could not prepare for (questions 10 and 11: p >0.001 and p = 0.03), with significantly greater numbers being recorded for both groups with reference to better than expected performances on the stations that could be prepared for (question ten).

Overall, it was also noted that both groups of students were positive in their perception of the selection process as a whole through the utilisation of the multiple mini-interview (question 12: p >0.001).

Responses to usefulness of stations

Table 3 details the responses relating to how applicants rated the usefulness of the MMI stations; it should be noted that only seven of the ten stations were common to both cohorts and this explains why dual responses were not recorded for stations four, nine and ten. All stations were delivered similarly for both cohorts and scored in the same manner by the interviewers. As five descriptors of usefulness of stations were provided the results are based on comparisons both within and between groups as described above with more positive and more negative descriptors also being compared to what can be described as the more neutral option of no feeling.

Table 3 Responses to the usefulness of the stations used during the multiple mini-interviews
Full size table

This part of the analysis evaluated the stations completed and what students felt about them with respect to their usefulness. For all stations other than one and seven it was noted that they were perceived to be significantly more useful than useless by the applicants (p >0.001). Comparisons between useless with ambivalence (no feeling) were also noted as being similar to that as usefulness although the significance of the differences varied. Station one related to a station examining creativity and while the results showed that the dental applicants felt if difficult to discriminate in usefulness, this does not suggest a poor station. Further analysis of the performance of the station itself would be needed and the distribution of the results analysed before such a station should be considered unsuitable. Equally pertinent to the observation and comments above is the applicants' perception of station seven, which looked at reasoning. For this station the DCP group did not return a significant difference when useful versus useless responses were analysed. The p value of 0.06 must be considered as showing similarity when all other responses are viewed.

Analysis of free text

There were 54 separate free text comments recorded from the applicants (36 by the BDS applicants and 18 by the hygiene and therapy applicants), qualitative analysis did not identify any themes that had not been noted in the previous thematic analysis study. The four emergent themes noted in the 2011 thematic analysis being 'lack of control', 'anxiety and nervousness', 'comparison with conventional interviews', and 'preparedness',5 were the themes that informed the construction of the questionnaire. If new themes had been identified then a new questionnaire for future analyses would have to be produced.

Discussion

This research showed an overall similarity between the two groups participating in the evaluation process and it was shown that the previous year's thematic analysis had successfully identified candidates' areas of concern as no new areas were identified from the free comments by either group, despite the fact that this was the first time a group of dental care professionals had had the opportunity to comment on a MMI process. The results showing that the admissions team (being cognisant of the thematic analysis) had taken sufficient steps to try and address previously identified concerns for the forthcoming cycle. It should also be noted that the DCP cohort (due to the relatively small number involved) has been considered as a single group in this research and that no attempt was made to differentiate between hygienists or therapists although this may have potential for future analyses and research.

Although the authors had expected to observe differences between the two prospective student groups, it was believed that the group new to the MMI process (the DCPs) may have been more reticent towards their use as they had not received any dealings in this area before and had not been able to share their experiences as the dental candidates had done the previous year. The fact that statistical analyses showed such minor differences was, again, perhaps testimony to the attention paid to the previous year's thematic analysis. It is also entirely plausible that social media and 'chat room' sites used by prospective students had also gone some way to demystify the MMI process as an experience for applicants.

Only question five (on nervousness) highlighted a difference when examining the intra-cohort results; the DCP group showing a more equal distribution on the degree of nervousness when compared to a traditional interview. It may be that this is due to the DCP candidates having less experience of interview processes in general but only further analysis and questioning directly related to such a hypothesis could prove this.

While questions one, three and five showed significant differences between the cohorts it is proposed that these are due to the actual different experiences of the two groups and the proportionally greater percentage in positivity expressed for the MMIs by the DCP cohort.

When analysed it would appear that only stations one and seven exhibited minor differences between the groups and it may be that these stations require examination regarding their generalisability towards what they are hoping to evaluate during the MMI.

It was evident that the responses received from the new dental student cohort were similar to that previously received and that no new themes or areas for concern were identifiable. While this does not necessarily add to the previous research it is, however believed, invaluable that annual review/assessment of the MMI process should occur in order for its continued development and improvement. It is entirely possible that a single comment can prove infinitely valuable; the authors believe that giving the opportunity to voluntarily complete a questionnaire affords the ability to facilitate such comments. It is clearly evident to those who participate and involve themselves in the MMI process that it needs to be dynamic and that annual appraisal of the process is very necessary in order to ensure its role in admissions, predictive validity and student progression.

Conclusion

The purpose of this study was to gain and develop further insight into applicants' perceptions of the multiple mini-interview process and quantify these in order to further inform future research and analyses based on the prepared questionnaire. It would appear that overall the experience of the multiple mini-interview process is a positive one by applicants and indeed that little difference was noted in this research between undergraduate dental student interviewees and DCP (hygiene and therapy) interviewees despite the newness of the process to the hygiene and therapy cohort. It was noted that the experiences between both cohorts were similar (not only) to each other but to the previous group studied.