Download Cognitive Ability Tests in Personnel Selection: A Comprehensive Review and more Study notes Human Resource Management in PDF only on Docsity!
CHAPTER 7- Using Ability Tests in Selection: Fishing Industry Introduction General mental ability (GMA) and specific cognitive tests have been recognized as the most powerful predictors of overall job performance, task performance, academic performance, and training proficiency. Thus, cognitive ability tests occupy the most relevant place among the personnel selection procedures. There are at least six important reasons supporting the claim that cognitive ability tests play a key role in personnel selection procedures First, GMA tests have the highest validity and lowest application cost. Second, the empirical evidence on the validity of GMA measures for predicting job performance is stronger than that for any other method. Third, GMA has been shown to be the best predictor of job related learning. It is the best predictor of acquisition of job knowledge on the job and‐related learning. It is the best predictor of acquisition of job knowledge on the job and of performance in job training programmes. Fourth, the theoretical foundations of GMA are stronger than for any other personnel selection measure, and theories of intelligence have been developed and tested by psychol-ogists for over 90 years. Fifth, the incremental contribution of specific abilities to the prediction of performance or training outcomes may be minimal beyond g. Sixth, the findings on the validity of GMA and cognitive tests are the major contribution of industrial, work, and organizational (IWO) psychology to the study of GMA and its use in applied settings. This chapter reviews the literature on the use of ability tests in personnel selection, focusing on several relevant issues: 1) the definition of cognitive abilities and prevalence of use in personnel selection; 2) the main theoretical models of the psychometric structure of GMA and cognitive abilities; 3) the criterion validity and validity generalization of GMA and specific cognitive abilities, including their incremental validity over GMA validity; 4) issues of group differences, bias and fairness; 5) applicant reactions and justice perceptions; and 6) suggestions for future research. (Objectives of the report) The Definition of Cognitive Abilities and Prevalence of Use in Personnel Selection The cognitive ability test is a test intended to measure a candidate’s cognitive ability. Cognitive ability is the mental capacity to reason with others, solve problems, understand complex ideas, and learn quickly. It is the ability to comprehend the environment and our surroundings. There is a lot of research that links cognitive ability to job performance and leadership potential. Therefore, a lot of organizations include in their recruitment process, cognitive ability tests. What do Cognitive Ability Test Measure?
- Reasoning: the ability to think in a logical way
- Perception: the ability to interpret sensory information
- Memory: the ability to memorize and store information
- Verbal Ability: the ability to process verbal information
- Mathematical Ability: the ability to process mathematical information
- Problem Solving: the ability to find solutions to difficult problems How to read cognitive ability test answers? There are several ways to interpret a cognitive ability test. The most common way is to read the test based on the sum of the correct answers given (raw score) by the candidate. It is also common to measure speed and incorporate the time spend to complete the test in the results. The speed of performing the test can be considered a factor when reading the results. For example, if you have two candidates that scored the same in the test, you could use speed as another metric to get insight on
someone’s ability to think faster. Human Resources also use normal distribution (continuous probability distribution for a random variable) and compare the test results to a pre-tested group. Are cognitive Ability Test Reliable? There is strong evidence indicating that cognitive ability is linked to employee performance. These tests measure general mental capability and are one of the most accurate psychological assessments. But as any other test, the cognitive ability test should be well defined to be effective and to predict job performance. The Main Theoretical Models of The Psychometric Structure of GMA and Cognitive Abilities Spearman’s model of Cognitive Abilities (only show structure in PPT) The existence of general intelligence was proposed by Charles Spearman in 1904. General Intelligence is also known as g factor, but in simple terms, it can just be called intelligence. Spearman concluded that there is a single g-factor which represents an individual's general intelligence across multiple abilities, and that a second factor, s, refers to an individual's specific ability in one particular area. If every individual excels in certain areas, it will not be impossible for them to excel in other areas too. Thurstone’s Model of Primary Abilities (only show structure in PPT) Louis Leon Thurstone made significant contributions in many areas of psychology, including psychometrics, statistics, and the study of human intelligence. He developed methods for scaling psychological measures, assessing attitudes, and test theory, among many other influential contributions. He is best known for the development of new factor analytic techniques to determine the number and nature of latent constructs within a set of observed variables. The new statistical techniques developed by Thurstone provided the necessary tools for his most enduring contribution to psychology: The Theory of Primary Mental Abilities, a model of human intelligence that challenged Charles Spearman’s then-dominant paradigm of a unitary conception of intelligence. Spearman, using an earlier approach to factor analysis, found that scores on all mental tests (regardless of the domain or how it was tested) tend to load on one major factor. Spearman suggested that these disparate scores are fueled by a common metaphorical “pool” of mental energy. He named this pool the general factor, or g (Spearman, 1904). Thurstone argued that g was a statistical artifact resulting from the mathematical procedures used to study it. Using his new approach to factor analysis, Thurstone found that intelligent behavior does not arise from a general factor, but rather emerges from seven independent factors that he called primary abilities : word fluency, verbal comprehension, spatial visualization, number facility, associative memory, reasoning, and perceptual speed (Thurstone, 1938). Furthermore, when Thurstone analyzed mental test data from samples comprised of people with similar overall IQ scores, he found that they had different profiles of primary mental abilities, further supporting his model and suggesting that his work had more clinical utility than Spearman’s unitary theory. However, when Thurstone administered his tests to an intellectually heterogeneous group of children, he failed to find that the seven primary abilities were entirely separate; rather he found evidence of g. Thurstone managed an elegant mathematical solution that resolved these apparently contradictory results, and the final version of his theory was a compromise that accounted for the presence of both a general factor and the seven specific abilities. This compromise helped lay the groundwork for future researchers who proposed hierarchical theories and theories of multiple intelligences
The Berlin Intelligence Structure Model (BIS) of Jäger (1982; 1984 ) is an integrative, hierarchical and faceted model of intelligence. At the most general level, general intelligence ("g") is assumed as the integral part of all ability components, i.e. the positive manifold. On the second level, seven higher order abilities are suggested to belong to two facets (Guttman, 1957): the operational facet distinguishes abilities according to the cognitive processes involved, i.e. reasoning, perceptual speed, memory, and creativity. The content facet distinguishes abilities according to the material that is applied, i.e. verbal, numerical and figurative- spatial intelligence. The cross-classification of the four operational and three content-related components is divided into twelve cells on the third level. However, the cells do not have the status of specific abilities (e.g. numerical reasoning, or verbal creativity) as in Guilford's Structure-of-Intellect model (SOI model) (Guilford, 1967), who postulated an ability factor for every combination of the SOI facets. Instead, performances are assumed on this level indicating that every intellectual performance depends on at least two abilities, an operational and a content ability. Therefore, the third level can be used to classify performance measures (tasks). The well-known Raven matrices (Raven, 1965), for instance, need to be classified into figural-spatial reasoning, indicating that this test is measuring a combination of two specific abilities. The seven factors of the BIS are described as processing capacity, which corresponds exactly to reasoning (fluid intelligence); creativity , which is close to fluency and flexibility (divergent thinking); memory, which refers to the ability to recall lists and configurations of items after learning them (short- term memory); and perceptual speed , which refers to the quick and accurate performance on simple tasks (mental speed). The three content factors are verbal, numerical, and figural-spatial intelligence. The more general ideas behind the BIS were summarized by Jäger, Süß, and Beauducel (1997)^ as follows: (1) all intellectual abilities contribute to every intellectual performance, but with different weights. The variance of every intellectual performance can be decomposed according to these abilities; (2) intellectual performance and the ability constructs can be classified according to facets. Two facets, operations and contents, were specified (bimodality assumption); (3) intelligence constructs are structured hierarchically, i.e. they can be assigned to different levels of generality (hierarchy assumption). Carroll’s Three-Strata Model (only show structure in the ppt) The three-stratum theory is a theory of cognitive ability proposed by the American psychologist John Carroll in 1993. It is based on the factor analytic study of the correlation of individual difference variables from data such as psychological tests, school marks, and competence ratings from more than 460 datasets. These analyses suggested a three-layer model where each layer accounts for the variations in the correlations within the the previous layer. The three-layers (strata) are defined as representing narrow, broad, and general cognitive ability. The factors described stable and observable differences among individuals in the performance of tasks. Carroll argues further that they are not mere artifacts of a mathematical process, but likely reflect psychological factors explaining differences in ability. This does not alter the effectiveness of factor scores in accounting fro behavioral differences. Carroll proposes a taxonomic dimension in the distinction between level factors and speed factors. The tasks that contribute to the identification of level factors can be sorted by difficulty and individuals differentiated by whether they have acquired the skill to perform tasks. Tasks that contribute to speed the factors are distinguished by the relative speech with which individuals can complete them. Carroll
suggests that the distinction between between level and speed factors may be the broadest taxonomy of cognitive tasks that can be offered. Carroll distinguishes his hierarchical approach from taxonomic approaches such as Guilford’s structure of intellect model (three-dimensional model with contents, operations, and products). Johnson-Bouchard VPR Model of Intelligence (only show structure in the ppt) The g-VPR model is a model of human intelligence published in 2005 by psychology professors Wendy Johnson and Thomas J. Bouchard. They developed the model by analyzing Gf-Gc theory, John Carroll’s three-stratum theory, and Vernon’s verbal-perceptual model. The g-VPR model is a four-stratum model:
- First stratum: Primary traits (openness, conscientiousness, extra-version, agreeableness, neuroticism)
- Second Stratum: Broader than stratum 1, but still narrow abilities.
- Third Stratum: Verbal, Perceptual, and rotation factors
- Fourth Stratum: g factor Johnson and Bouchard made comparisons between Gf-Gc, three-stratum, and verbal-spatial model. The found that Vernon’s verbal perceptual model got better modeling-fit results than the other two, but still did not fit very well. Then, based on the verbal-spatial model, Johnson and Bouchard began by adding a memory factor (labeled content memory to distinguish it from the memory factor in the fluid- crystallized model). They also made some extra changes to improve model fit. According to results form CAB (comprehensive ability battery), they added/moved loading to adjust new mode. Then it came into a new “visual-perceptual-memory” model. However, Johnson and Boucher found that isolation of the tests involved resulted in identification of an additional third-stratum factor for image rotation, which also eliminated the contradictory cross-loadings. Therefore , finally, the new model was named as verbal-perceptual-rotation (VPR) model. The Mutualism and Extended Mutualism Model (only show structure in the ppt) The mutualism theory of intelligence posits that the positive manifold arises because of mutually beneficial interactions between originally orthogonal cognitive abilities, rather than because of a genuine general construct. Gignac (2014) reported that the strength of intelligence was largely constant from the ages of 2.5 to 90 years, which was interpreted as an indirect failure to confirm the mutualism theory of intelligence. In this investigation, a second indirect test of the mutualism theory of intelligence was performed. Specifically, it was hypothesized that, if the extended mutualism of G is plausible, then there should be some consistent, positive associations between group-level factors, controlling for the effects of a general factor. To examine this possibility,the associates between cognitive ability group- level factors were estimated across series pf seven relatively large and relatively represented samples of intelligence battery data. Next, seven single-factor models were estimated against the seven group-level inter-correlation matrices. The corresponding single-factor residual correlations matrices were observed to yield an approximately equal number of positive and negative residual correlations and an overall mean of zero. Furthermore, the only moderately consistent residual effect was a negative association between crystallized intelligence and processing speed. Although only an indirect test, the results are interpreted to be more supportive of intelligence factor theory than mutualism.
Organizational justice is concerned with the fairness of the distribution of organizational outcomes (outcome fairness) and the fairness of procedures used to distribute these outcomes (procedural justice). Gilliland adapted the basic principles of organizational justice to provide a comprehensive model of how applicants perceive and react to selection procedures. This model has received considerable support. Glliland’s model suggests that selection systems and tests are viewed favorably by applicants (i.e., are considered fair) to the extent they comply with or violate procedural and distributive justice rules. These procedural and distributive justice rules are standards that applicants hold for how they expect to be treated and how selection procedures should be administered and used. These justice rules determine perceptions of process and outcome fairness, such that when the rules are satisfied, the selection process and outcome are perceived as fair, but when they are violated, the selection process and outcome are perceived as unfair. As will be discussed, applicant perceptions of the fairness of a selection process can influence a number of important individual and organizational outcomes. It should be noted that according to Gilliland’s model, justice rules would not directly relate to applicant intentions or behavior, but would do so indirectly through process fairness perceptions. For example, perceived job relatedness is an example of a procedural justice rule. Perceived job relatedness refers to the extent to which the applicant perceives that the content of a test reflects the content of the job (e.g., the knowledge, skills, and abilities required by the job). Perceived job relatedness has been recognized as the most important procedural justice rule because it consistently influences fairness perceptions and, through fairness perceptions, test performance. Conclusion In The Nature of Intelligence , Thurstone (1924, p. xiv) wrote: ‘there is considerable difference of opinion as to what intelligence really is, but, even if we do not know just what intelligence is, we can still use the test as long as they are demonstrably satisfactory for definite practical ends.’ Ninety years later, we can conclude that intelligence and cognitive tests have demonstrated they are excellent procedures for the practical purposes of personnel selection and that, although opinions differ on what intelligence is persist, important advances on the theoretical account of cognitive ability have been made. There is no consensus about the nature and existence of a general factor of intelligence, but the vast majority of researchers now agree that a general factor can be found when a large battery of cognitive tests is factor analysed, and the majority of the psychometric models of cognitive abilities include a general factor (Horn’s model is the exception). There is less agreement regarding the number of levels in the hierarchy and the number and type of medium and narrower abilities. Though survey data show that GMA tests are frequently used in personnel selection across the world, this is not a sufficiently good reason for using the procedure for decision making. For example, graphology is very popular in some countries (e.g., Brazil, France and Israel), but the empirical evidence shows that its validity for predicting job proficiency is zero. In other words, if the validity of a procedure is zero, it would be the same as using a table of random numbers to choose an applicant. The empirical evidence cited in previous sections suggests that, in a rapidly changing world of work, GMA is the best predictor of the future adaptability to new tasks and functions. Succinctly, the state of art suggests that: 1) the validity of‐related learning. It is the best predictor of acquisition of job knowledge on the job and ‐related learning. It is the best predictor of acquisition of job knowledge on the job and cognitive ability tests are generalizable across occupations and situations, and moderated by job complexity, so that operational validity is 0.40 or higher; 2) the relationship between GMA and task
performance is linear and its effects are primarily indirect thorough job knowledge; 3) GMA predicts moderately OCB, but not CWB; 4) there are ethnic and group differences in both GMA and the specific cognitive abilities, and the standardized differences are greater for lower job complexity levels and for crystallized ability; 5) although there is some evidence of differential validity, there is no differential prediction (bias) for African Americans. Finally, the findings underscore that cognitive ability tests may‐related learning. It is the best predictor of acquisition of job knowledge on the job and be valuable, cost saving instruments for companies by ensuring high standards of individual job‐related learning. It is the best predictor of acquisition of job knowledge on the job and performance, which in turn raise productivity (Scherbaum et al., 2012).
