The Assessors

Assessors can be professional psychologists or line managers in the organization, the latter usually two levels above the position that is being considered. The advantage of using line managers as assessors is that they are familiar with the organization and the behavior required in the job. However, some research has shown that validities of ACs are higher when professional psychologists instead of line managers are used as assessors (Gaugler, Rosenthal, Thornton, & Bentson, 1987).

The focus of an AC is to gain information from various exercises to evaluate the concerned behavioral dimensions. It is very important that the assessors are well- trained both on making observations of each behavioral dimension and on how to evaluate and score the candidates' performance based on the observations made throughout the AC. If professional psychologists are hired to be the assessors, they must be formally briefed on the objective of the AC and the behavioral dimensions that are needed for the job that is being considered. If the assessors are line managers, although they are very knowledgeable about the job behaviors, they typically are not familiar with making observations on behaviors representative of each dimension and to give ratings based on the behavioral evidence collected from each exercise. Moreover, previous research has found that assessors seem to form an overall impression of candidates' performance first and then rate the behavioral dimensions based on that initial overall impression (Lance, Forster, Gentry, & Thoresen, 2004). This results in inaccurate ratings. Thus, it is essential to train line managers on how to make observations and how to evaluate candidates' performance.

Assessor training should help to develop assessors on the six abilities noted below (Gatewood, Feild, & Barrick, 2008):

  1. Understanding the behavioral dimensions

    First of all, assessors should have a thorough understanding of the behavioral dimensions that are of concern in the AC and that are important for the job in question. Usually a definition with a detailed description of each behavioral dimension is given to the assessors. Assessors must be familiarized with the dimensions and all assessors should have a common understanding of each dimension so they know what behaviors they should look for under that particular dimension.

  2. Observing the behavior of participants

    After they have understood the behavioral dimensions, assessors should learn what behaviors to observe and record for later scoring. In most cases (as noted above) assessors are easily driven by their initial impression of a candidate and this first impression will affect the scoring. The first thing to do here is to help assessors to differentiate recording behaviors and making judgments. To do this, usually assessors are given different statements and are asked to answer which statements are judgments and which reflect behavior. For example, "confident in resolving the conflict" is a judgment and "came up with alternatives during discussion" is a statement which reflects a behavior. In training, usually after the assessors have a sense of the differences between judgments and behaviors, they will watch a video and practice recording behavior.

  3. Categorizing participant behavior into appropriate behavioral dimensions

    This phase merges the first two phases. The assessors are trained to record behaviors under each behavioral dimension. Usually a behavioral dimension and its description, as well as the behaviors representing each dimension are given to the assessors. The assessors are then asked to put each behavioral statement under each behavioral dimension. In training, after they are familiarized with the behavior under the corresponding dimensions, they will watch a video and try to observe and record the behavior under each relevant dimension.

  4. Judging the quality of participant behavior

    The aim of this part of the training is to develop common ground among all assessors in using the rating scales. Usually each behavioral dimension is rated on a 5-point scale ranging from 1 (no opportunity existed for this dimension to be shown) to 5 (excellent, with a great deal of the dimension being shown). Assessors must build a common frame of reference so that they base their scoring on the same scale.

    In training, assessors are usually given a list of behavioral statements and asked rate each statement based on the 5-point scale. A discussion will follow to bring out any differences in the ratings; thus any dissimilarity or disagreements can be clarified to align the assessors.

  5. Determining the rating of participants on each behavioral dimension across the exercises

    In this part of the training, the assessors will be taught to draw conclusions on the score for each behavioral dimension by combing the rating points of each dimension across different exercises. Similar to the last phase, this phase is also based on group discussion and it is about how to come up with a frame of reference when combining the scores. Successful assessors will be able to combine the ratings of different assessors across different exercises to come up with themes regarding the behaviors.

  6. Determining the overall evaluation of participants across all behavioral dimensions

    The assessors will be trained on how to come up with an overall rating against all the behavioral dimensions, which in turn are based on the job analysis results. The job analysis helps determine which dimensions are more important to the job in question (for selection ACs) and these dimensions are to be weighted more heavily. Again, the assessors will go through a mock assessment and a discussion will follow to solve any question or problem each assessor may have.