Download The Effectiveness of Structured vs. Unstructured Interviews in Predicting Job Performance and more Exercises Literature in PDF only on Docsity!
Structured vs. Unstructured Interview:
Improving Accuracy & Objectivity
Prepared by the Dept of Psych EDI committee, McGill: This document reviews the advantages of structured over unstructured interviews in giving rise to better hiring decisions. It is written for people who might wish to improve or standardize their interview process. Different techniques and what is known or unresolved about these techniques is summarized. Prior to the reviewing the literature, we first discuss some terms that are useful for understanding the available research summary. General Definitions and Description: Interviews – Within the academic setting interviews are commonly used in the context of graduate student and faculty member recruitment. Occasionally, undergraduate students will experience an interview process when applying for lab research. In general, an interview can be defined as: “An interpersonal interaction of limited duration between one or more interviewers and a job-seeker for the purpose of identifying interviewee knowledge, skills, abilities and behaviours that may be predictive of success in subsequent employment. The operational indicators of this success include criteria of job performance, training success, promotion and tenure.” (Wiesner & Cronshaw, 1988) While the actual goals of this process can be listed as, “...[to determine] operational indicators...[such as] criteria of job performance, training success, promotion and tenure.” (Wiesner & Cronshaw, 1988, pg.276) Unstructured Interviews – An interview process in which questions asked are not systematized across candidates, and the interviewer focuses on open discussion to evaluate candidates. Common traits characterizing Unstructured Interviews are a lack of pre-determined questions, rating scales and/or topic guidelines. Consequently, each interview varies in questions asked to which candidates. It is difficult to remember a full discussion line by line, and different conversations can lead in different directions, thus when Unstructured Interviews are utilized as an interview process, the questions will always vary. Structured Interviews – An interview process in which questions are pre-determined and asked consistently to all candidates. Additionally, Structured Interviews will commonly include a ranking scale associated with a candidate’s answers. Research suggests the most important attributes and/or dimensions of a Structured Interview are: “...job-relatedness of the interview...standardization of the process...and structured use of the data to evaluate the candidate” (Macan, 2009, pg. 205) Dilution Effect – We are not always efficient in discerning pertinent information when both relevant and extraneous facts are presented. The more data that is presented, the greater difficulty in focusing on the important information that pertains to the specific topic (Dana et al., 2013, pg.512).
Sensemaking – Can be described as “…the ability for interviewers to make sense of virtually anything the interviewee says…” (Dana et al., 2013, pg.512). This can be beneficial as it causes us to naturally seek connections between events. Yet these interconnections can sometimes be simply imposed by the interviewer. Similar–to–me–Bias – A bias in which a person with whom we can perceive a common ground (gender, ethnicity, hometown & hobbies for example) will be more favoured compared to someone who we have nothing in common with. Halo Effect – A cognitive bias which can also be called a stereotype of physical attractiveness. Studies have shown that people who were rated higher in physical attractiveness were hired more frequently and were less likely to be convicted in comparison with those who were not rated as highly attractive. Individuals who are higher on attractiveness are perceived as higher on other positively-valenced qualities, though attractiveness is not in fact correlated with all of those qualities.
Evidence Structured Interviews are more objective and accurate:
As stated earlier, the goal of an interview is to identify the skills, knowledge and behaviour of an interviewee and the subsequent success that person may achieve should they be employed. Research has shown Structured Interviews are better at predicting actual job performance when multiple candidates are interviewed (Levashina et al., 2013) “A major finding in interview research … is that interviewer judgments based on structured interviews are more predictive of job performance than those from unstructured interviews.” (Macan, 2009, pg. 204) This is because four main issues may arise when Unstructured Interviews are used:
- Low reliability: the candidate demonstrating the best potential in job performance may be passed over.
- Low validity: selection of a candidate tends to be somewhat random and is not strongly correlated with job performance. “… adding structure to the interview process can enhance the reliability and validity of interviewer evaluations…” (Macan, 2009, pg. 204).
- Susceptibility to biases: Decisions informed by Unstructured Interviews are more susceptible to a variety of biases such as the Halo Effect and implicit stereotyping biases pertaining to gender, ethnicity, sexual orientation and potential disabilities of the candidates Additionally, since Unstructured Interviews leave the structure of the interview to the interviewer, not only in terms of questions, but also choice of ranking scheme, decisions arising from Unstructured Interviews can be influenced by the interviewer’s idiosyncratic beliefs about job requirements and the ideal candidate, rather than closely relating to the actual job requirements. “Because interviewers conduct unstructured interviews in an idiosyncratic way and have discretion in what they ask and how they evaluate responses (Dipboye, Wooten, & Halverson, 2004), the content and evaluation process in unstructured interviews may be more reflective of the interviewers’ implicit theories of the job requirements than the actual job requirements.” (Levashina et al., 2013, pg.12)
information about questions specific to certain groups that might help recruit candidates of diverse backgrounds. Questioning and Interview Process Control of Ancillary Information Although there are many ways to question a candidate in a Structured Interview. A general guideline was proposed in the article The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature “... basing questions on a job analysis... asking the same questions of each applicant... limiting prompting, follow-up, and elaboration on questions... using better types of questions... using longer interviews or larger number of questions... controlling ancillary information; and not allowing questions from applicants until after the interview.” (Levashina et al., 2013, pg.4) An important point this quote touched on is the control of ancillary information. This can be avoided by tailoring the questions to specific aspects of the job while maintaining the flow of the interview. The candidate should not be allowed to elaborate upon a point, or question, more than the interviewer feels is necessary. This will help mitigate the Dilution Effect while concurrently preventing the interviewer from building up a case/argument as to why they deserve the job. One could argue the point of an interview, from the candidate’s perception, is to build a case as to why they deserve the job. However, when the interview is not focused on job specific questions; we tend see an increase in overall Interview bias due to the many aforementioned points. Beware of Rapport Building Another aspect of Structured Interviews which deviates from the standardized norm is the avoidance of rapport building, or ‘breaking the ice’, when beginning the interview (Levashina et al., 2014, pg.10). That being said, studies are conflicted in opinion as to the operationality of removing informal rapport building. Some researchers argue this supports the candidate as it gives them the chance to become acquainted with the interviewer, and that this actually fosters a more stress-free environment in which the candidate may perform to their full potential. Over time this can beneficial as it will develop the employer-employee relationship, consequently increasing overall productivity. Additionally, research has found that when rapport building is removed, the candidate is left with a more negative impression of the interviewer and the company as a whole. The main argument in opposition of that point pertains to Impression formation and the Confirmation bias. Beginning an interview with no predetermined format allows the interviewer to converse freely with the candidate. This situation allows several forms of interviewer bias to manifest. In order to circumvent this, research has provided a couple of suggestions. Above all, they recommend completely removing any form of rapport building. Yet if this, for various reasons, will not be done; research advises to have a predetermined script in which the interviewer follows precisely. This script would contain ‘ice- breaker’ questions that would be asked of each candidate in order to simultaneously put them at ease whilst preserving consistency and fairness across all candidates. To conclude, it does come down to the interviewer and whatever they feel comfortable with. While the interviewee will always appreciate some form of rapport building, it is important to be cognizant of the aforementioned points as these can influence the interviewer when deciding upon the best possible candidate.
Question Consistency It is important that each candidate is asked the same question, and that the phrasing of those questions remains consistent from interview to interview. Types of Questions Two different question forms have arisen as potentiates for effective Structured Interview questioning. The first being Situation Questions and the other being Personality Based Questions (Levashina et al., 2014, pg.18).
- Situation Questions can be further broken down into the two subcategories of goal- orientated questions and job-specific questions. a. Goal-orientated questions do not ask candidates about, for example, their five/ten- year plans. More so, they ask candidates about their future goals as if they already secured the job in which they are applying for. An example of this could be: “If you were made manager of this office, what would you like to have achieved, in terms of this company, within five years and how would this benefit the company and your fellow employees?” b. Meanwhile, job-specific questions are focused on testing a candidate’s cognitive abilities in relation to the job they are applying for while additionally challenging their professional judgement as if they were already in that role. Essentially, the idea is to place candidates in practice situations that closely resemble potential real-work experiences. An interviewer could ask, for example: “If you were a professor, how would you deal with a student making micro-aggressive comments in class?”
- The other format of purposed Structured Interview questioning is Personality Based Questions. These questions are poised to determine the personality of a candidate based off previous job situations and how they were handled. However, this brings up a few important points worth mentioning. a. The first being how does an interviewer accurately measure important psychological traits such as conscientiousness, empathy and self-discipline? One way would be to hire psychologists to interview the candidates, yet this will become heavily cost-intensive with large applicant pool. Another solution is to have potential candidates complete a personality exam. Yet these can be faked or influenced preventing proper data collection. b. Additionally, how do we decide which personality traits are ideal for each specific role? c. The third, and probably the most prominent, point is that Personality Based Questions may actually lower overall diversity. Once the ‘ideal personalities’ are somehow determined, these personalities will be sought after in comparison with others. This will likely decrease diversity as it is highly likely that specific groups of people demonstrate more of these qualities than others. On the other hand, Personality Based Question’s show greater correlations with predicting job performance in comparison with Situation Questions, especially as the complexity of the job increases making Situation Questions much more difficult to answer properly. Therefore, Personality Based Questions are good for predicting job
preparing before the interview. When provided with the questions, the candidates will have prepared for the interview and thus be more confident going into it. This is likely accounting for a higher approval rating in candidates who were provided the questions. Overall, research is inconclusive as to the actual effects of question transparency. More research is needed. In conducting Structured Interviews, the literature has been rather sparse. It mainly focuses on asking Situational Questions and Personality Based Questions, without alluding to the actual process of the interview. However, Kelly et al. (2018, pg.18) in A systematic review of stakeholder views of selection methods for medical schools admission put forth the concept of Multiple Mini Interviews, or MMIs. These are a series of structured interviews, taken in succession and split into multiple stations with varying interviewers. Each station/interview lasts only about five minutes, and there can be anywhere from seven to fourteen stations. Within each station candidates are presented a situation replicating real-work scenarios, in which they must act accordingly with their responses being evaluated. The stations should be varying in difficulty of cognitive requirements and professional judgement. The reason being it is not only the responses to individual stations that is measured, but also the relative response between easy and difficult stations is what evaluators place the most emphasis on. For this type of structured interview, it is advised that an absolute ranking system should be avoided as the evaluators focus on the relative response, not just the candidate’s performance in individual stations. Multiple Mini Interview’s are designed to limit the Similar-to-me bias by having multiple stations with many different evaluators. That being said, Multiple Mini Interview’s are much more time consuming and cost intensive compared to other forms of interviews. Evaluation Vertical and Horizontal Evaluation Process Two seemingly conflicting strategies on candidate evaluation were put forth by Bohet, (2016, pg.4) in How to Take the Bias Out of Interviews. The first entails a vertical evaluation process where candidates are scored immediately after a response is given. This functions to remove any reliance upon memory when scoring candidates as that can allow various forms of stereotypes to infiltrate the mind and corrupt their view of the candidate. According to Bohet, avoiding a reliance upon memory also seems to prevent implicit Group biases from affecting evaluators decision-making processes. Another strategy is called horizontal evaluation. This form of evaluation requires all of the interviews to have been completed before the evaluation process can begin. After the final interview, the candidates are compared in a linear fashion where, for example, each candidate is assessed for question A before moving onto the assessments for question B. The research has described this form of evaluation as being more impermeable to gender-based stereotypes and/or biases. This process will rely upon memory and consequently be susceptible to the aforementioned stereotypes. Yet, a solution was proposed in the form of note taking. Note-Taking Throughout the interview, the evaluator should be taking notes pertaining to any extraneous information they notice in the interviewer. This can present itself in the form of body language, vocal inflections, confidence level and overall flow and/or way of speaking. It is
important that evaluators know exactly what they should be looking for and how this might appear in a candidate. This data can then be used to potentially infer a candidate’s personality, and further, to determine if the job position is a good fit. (Levashina et al., 2014, pg.21). In order to prevent Interviewer bias from arising due to the subjectivity of each evaluator’s determination, it is important to have an established guideline of desired personality types and how they might be presented in an interview. Anchored Rating Scales Researchers presented a ranking system called Behavioral Anchored Rating Scales (BARS) in order to evaluate these personality cues (Levashina et al., 2014, pg.34). Essentially, for each question there is a corresponding list of answers the candidates may provide. Once an answer is given, the evaluator will then circle one of the predetermined answers that matches the candidate's given response the closest. The literature advises the difference in job performance accuracy was minimal when five or seven descriptions were provided for each question. The idea behind this process is to reduce interviewer bias while simultaneously providing quantitative data that can be analyzed later on. In practice, an evaluator will have a list of attributes they are looking for in each question. For example, when asking the candidate a question about their biggest weakness. The Behavioral Anchored Rating Scale could provide body language cues pertaining to honesty and level of arrogance, along with provided examples allowing the interviewer to make the most accurate discernment possible. For non-Personality Based Questions, the author proposed the idea of Anchored Rating Scales (ARS) (Levashina et al., 2014 , pg.33). These are essentially the same thing; except they focus on providing ranking scales for Situational Questions and job-specific questions, they have nothing to do with personality. Without an established rating scale for the candidates, the decisions made by the interviewer will be determined by qualitative cues presented by the candidate. In addition to quantitative facts committed to memory which, are vulnerable to various biases used by the brain to reinforce memory. This prevents interviewers from comparing candidates in a fair way based off tangible data and information.
