Selecting and Hiring Psychologically Fit Probation Officers: A Focused Examination of the PEPQ/PSR Plus

Selecting and hiring the most psychologically fit probation officers is of utmost importance to the judiciary, court administration and the public good. This study examined the predictive validity of the PsychEval Personality Questionnaire/Protective Service Report Plus (PEPQ/PSR Plus) in its ability to predict job performance in a combined cohort of pre-employment and incumbent probation officer candidates. Analyses revealed a statistically significant ability to predict performance problems, demonstrating that the PEPQ/PRS Plus is a valid and clinically useful psychological screening tool for the assessment and selection of probation officer candidates.


Introduction
Court administered probation services are expanding exponentially throughout the world. In the United States, there are more than 2,000 separate probation agencies that fall under the jurisdiction of the courts 2 and the employment of probation officers is expected to increase by 11% between 2006 and 2016 3 . In Europe, there has been a crucial development of probation services over the past 10 to 15 years in several countries that previously had none (e.g., Romania, Bulgaria, Moldova, Slovakia, and the Czech Republic). Moreover, other countries that have had well established probation services have also experienced a marked expansion of their systems (e.g., England and Wales, Finland, and the Netherlands) 4 .
With considerable growth and development of probation services, a burgeoning area of practice for police and public safety psychologists is the provision of psychological evaluation services for probation departments. Specifically, conducting psychological evaluations of current and prospective employees is a human resource service that has been used for decades in police agencies, but has only recently begun to emerge within probation departments. Three of the most common types of psychological evaluations performed with law enforcement agency personnel include; preemployment psychological evaluations (conducted to assist agencies with making informed hiring decisions), fitness for duty psychological evaluations (conducted with an incumbent employee whose behavior has given cause for concern regarding psychological suitability for work), and special duty psychological evaluations (conducted with an incumbent employee who is being considered for a duty assignment requiring specialized psychological traits or characteristics). While such psychological evaluations within probation departments are becoming increasingly common, very little scientific work has been done in this area, and it has been largely ignored by those who work within the realm of police and public safety psychology. For example, an EBSCO Host query of the PsychInfo, PsychArticles and Medline databases conducted in January 2014 for Boolean terms "probation," "psychological" and "selection" yielded zero on-point hits regarding psychological evaluation and selection of probation officers. While the psychological literature is replete with empirical studies assessing the efficacy of selection tools for predicting police officer performance 5 , the literature is barren regarding the psychological assessment and selection of probation officer candidates. Nevertheless, the consequences of failing to "select out" psychologically ill-equipped candidates for probation officer positions can be extreme, and can put probation departments, courts and the general community at-risk.

Similarities and Differences Between Probation and Police Officers
Of importance, psychological evaluations of probation officer candidates should not be construed as being simply synonymous with police psychological evaluations since there are a number of important and noteworthy distinctions between these groups. Namely, there are important differences in essential job functions, in the philosophical orientation of departments, and in the background and training requirements of police versus probation officers. Moreover, in most jurisdictions in the United States, police officers are required to be certified under a state's Peace Officer Standards & Training Board (P.O.S.T.), while probation officers may or may not be required to be P.O.S.T. certified. Additionally, a police officer's primary job function is usually defined as "protecting and serving the community," whereas probation officers' duties typically entail providing both enforcement and rehabilitative services under the auspices of the courts. Van Kalmthout & Durnescu (2008) note that, in Europe, most probation department mission statements reflect the traditional ethos of protecting the public by effectively enforcing community sanctions, but some jurisdictions have begun to include the victim as a probation service beneficiary (e.g., Austria, Bulgaria, Catalonia, Romania and Scotland). They also note that others have begun to embrace a restorative justice paradigm (e.g., the Czech Republic, Hungary, Slovakia). Nevertheless, the end result is that each probation department finds itself in the unique position of articulating whether "rehabilitation," "enforcement," or some combination thereof is emphasized in its agency's primary mission statement 6 .
Although important distinctions do exist across job functions, there are also a number of important similarities between police and probation officers. Chief among them is the growing trend within the United States for probation officers to carry firearms 7 , with many probation departments empowering their officers with full arrest authority. As reported by Small & Torres (2001), all but 11 of the 94 federal judicial districts in the United States currently permit U.S. probation officers to carry firearms. As an outgrowth of such changes, probation agencies have found themselves having to adapt by providing firearms tactical training to their officers, and by establishing policies and procedures for the certification and/or decertification of officers who carry firearms. With these enhanced functions come increased risks, not only to probation officers themselves, but also to probation departments and the general community. While a comprehensive review of both similarities and differences between probation and police personnel is beyond the scope of this article, the interested reader is referred to Herrmann & Broderick (2011) 8 for a thorough analysis and review of this topic.

Purpose of Study and Research Question
As a result of the multifaceted nature of probation work, and because the consequence of failing to select out unsuitable probation officer candidates can be great, it is imperative that only the best and most aptly suited individuals are selected and hired into probation officer positions. By extension, it is also of utmost importance to know whether psychological screening instruments and procedures historically used and validated with police officer candidates are valid and useful for the screening and selection of probation officer candidates in the 21 st century. Because important distinctions exist between these vocational cohorts, it cannot be assumed that assessment devices will have equivalent validity with police and probation officer candidates. Predictive validity must be established and cannot be assumed. The purpose of the present study was therefore to conduct a focused predictive validity analysis of one psychological screening instrument, the PsychEval Personality Questionnaire / Protective Services Report Plus [PEPQ / PSR Plus] (IPAT, 2003), for the assessment and selection of probation office candidates. Specifically, this study sought to determine whether PEPQ / PSR Plus scale scores and/or composite factor scores could be used to predict inadequate performance among a combined sample of new-hire probation officers candidates and veteran "incumbent" probation officers. The specific research question addressed in this study was as follows:

Participants
Participants in this study were a convenience sample of 202 pre-employment and 60 incumbent probation officer candidates who were administered the PEPQ / PSR Plus as part of a psychological screening and selection process. The 202 pre-employment candidates were seeking initial employment as probation officers, while the 60 incumbent officers were seeking departmental authorization to carry a firearm in the commission of their job duties.

Measures
The PEPQ / PSR Plus is a computer generated report produced by the Institute for Personality and Ability Testing, Inc. (IPAT), and is based upon a test taker's responses to the 325 item PEPQ test. Part I of the PEPQ contains all 185 "normal" personality items that are derived from the 16PF Fifth Edition 9 . Part II of the PEPQ contains 140 items that tap into "pathology oriented" constructs, some of which were drawn from Raymond Cattell's early work in the 1960s on the Clinical Analysis Questionnaire (CAQ), and some of which were drafted more recently. The PEPQ / PSR Plus report is intended to provide a global view of an individual by assessing 16 "normal" personality dimensions as well as 12 "pathology oriented" dimensions. The PEPQ / PSR Plus report specifically contains information about a test taker's response style, an interpretive section regarding four Protective Services Dimensions, a profile summary of five global factor scales, 16 primary factor scales, 12 pathology oriented scale scores, and a pathology oriented index providing a snapshot of a test taker's psychological health in four critical composite dimensions. A summary of the various PEPQ / PSR-Plus test dimensions is presented in Table 1 and Table 2. Because the PEPQ / PSR Plus report includes an assessment of pathology oriented constructs that may convey disability or impairment related information, the PEPQ / PSR Plus is only applicable for use in pre-employment screening contexts in the United States where a "conditional offer" letter of employment has been tendered (see EEOC, 1995EEOC, , 1997EEOC, , 2000.

Procedure
Prior to being evaluated, all probation officer candidates read and signed an informed consent document detailing the components of their psychological evaluation, and granted authorization to use data generated from their evaluation process for anonymous and aggregate research purposes. Permission to conduct research was also granted by the adult and juvenile chief probation officers for whom third-party psychological evaluations were being conducted. All probation officer candidates were administered a battery of written psychological tests which included the PEPQ / PSR Plus 10 .
Candidates also participated in an approximate 50 minute face-to-face interview with a licensed psychologist as part of their evaluation process. The clinical interview covered 10 topical areas thought to be relevant in the assessment of those who are interested in probation work. The interview was scored and an applicant's score was combined with information from other sections of the overall psychological assessment battery. As is typical in law enforcement screenings, candidates were not given any direct feedback regarding their scores or interview performance, which was a blind evaluation process to the candidates. While a multimodal assessment battery was used in the evaluation and selection process, only scores generated from the PEPQ / PSR Plus report were examined for the purpose of this study since it was judged to be the most robust component of the assessment battery, and was designed to be a comprehensive, "stand-alone" psychological screening tool.

Design and Analysis
For each officer in this study, critical incidents were collected as part of the day to day operations of the department. These critical incidents served as the outcome of interest in the present study (i.e., the criterion variable). Critical incidents were defined as negative incidents in the officer's work record that could result in extending an initial probationary period, being released prior to completing a probationary period, or incidents of gross misconduct by an officer. Most officers in the sample were not involved in any critical incident and of those that were, even fewer were involved in more than one critical incident. As a result, the critical incidence criterion was coded as a dichotomous variable with a base rate of involvement in one or more critical incidents of 14.9%.
The data for the applicants and incumbents were combined for the current study as the sample size for the incumbent group was too small to allow for meaningful analyses of the relationship between the PEPQ / PSR dimensions and inappropriate behaviors 11 . A total of 262 officers were included in the analyses. A logistic regression model was utilized as the criterion was dichotomous and the scales from the PEPQ / PSR Plus are continuous in nature. Logistic regression is considered more appropriate than linear regression for modeling outcomes with discrete categories. The PEPQ / PSR dimension scales and the 12 pathology scales were entered into the model as the predictor variables and a backward elimination method was used to refine the model. Backward elimination starts with the most complex model, known as the saturated model, and eliminates terms in the model starting with the highest order interactions one at a time based on a comparison of the saturated model to the less complex model 12 . This process continues until the comparison of the model chi-squares are not significantly different. The initial saturated models for critical incidents were comprised of the four PEPQ / PSR dimensions, the 12 pathology oriented scales, and all possible two-way and three-way interactions.
Higher-order interactions were not considered both because there was no theoretical rational to include them and the model would have been unnecessarily complex given the current date set. Also, the PEPQ / PSR dimensions were used rather than the 16 primary factors as they were the point of focus in the current study, and using these four composite dimensions rather than the 16 primary factors greatly simplified the saturated model.

Results
A qualitative description of each of the 16 primary factors, four protective services dimensions and the 12 pathology oriented scales are presented in Table 2. Means and standard deviations for each are presented in Table 3. The final model for critical incidents criterion consisted of an interaction for Interpersonal Relations and Intellectual Efficiency PEPQ / PSR dimensions along with their associated main effects and a main effect of the Alienation and Perceptual Distortion scale. The weights for the model are presented in Table 4. As the weights in Table 4 indicate, both of the PEPQ / PSR dimensions were negatively related to the probability of a critical incident, and the Alienation and Perceptual Distortion scale was positively related to the probability of a critical incident. In other words, as scores on the Interpersonal Relations and Intellectual Efficiency PEPQ / PSR dimensions increased, the likelihood of a critical incident statistically decreased. Conversely, as scores on the Alienation and Perceptual Distortion scale increased, the probability of a critical incident statistically increased.
Interpreting a logistic function, such as the results of a logistic regression model, tends not to be an intuitive process. As a result, the logistic regression model is typically translated into either a probability or an odds ratio. This is done by rescaling the weights for the predictors and the constant by the inverse of the logarithmic function. Both the original and the re-scaled values for predictors in the model are shown in Table 4. The third column in Table 4 (labeled "Exp (B)") displays the re-scaled weights, while the last two columns display the lower and upper confidence intervals for the re-scaled weights, respectively.
Interpreting the re-scaled weights revealed that, 1) a one-unit increase on the PEPQ / PSR Plus Alienation/Perceptual Distortion scale resulted in the probability of a critical incident increasing by a 36%, 2) a one-unit increase on the Interpersonal Relations composite resulted in a decrease in the probability of a critical incident by 62%, and 3) a one-unit increase on the Intellectual Efficiency composite resulted in a decrease in the probability of a critical incident by 50%. Similar to multiple linear regressions, the interpretation of a single predictor is only viable when all other variables in the model are held constant.    Figure 1 to demonstrate the effect of one-unit score increases while holding other predictors in the model constant. The Y axis in the figure is the probability of a critical incident given a specific score profile on the predictors in the mode (i.e., Interpersonal Relations, Intellectual Efficiency, and Alienation and Perceptual Distortion). The X axis reflects scores on the Alienation and Perceptual Distortion Scale. The lines in the figure represent a one unit increase in the Intellectual Efficiency dimension. As Figure 1 shows, when a score on the Intellectual Efficiency dimension is equal to one (line labeled as Sten = 1), and the score on the Alienation and Perceptual Distortion scale is also equal to one, there is a probability of approximately .70 that an applicant will experience a critical incident during the probationary period. This probability increases with each one unit increase for scores on the Alienation and Perceptual Distortion scale until the probability plateaus around .99 for a Sten score of 10. Setting the Intellectual Efficiency score to 5 (holding the Interpersonal Relations dimension score equal to one) we can trace the probability of a critical incident as the Alienation and Perceptual Distortion scale increases by following the line labeled "Sten = 5" in the figure. In this example, when the Alienation and Perceptual Distortion scale score is equal to one, the probability of a critical incident is approximately zero. As the score on the Alienation and Perceptual Distortion scale increases, so does the probability of a critical incident. With an Alienation and Perception scale score of 5, the probability increases to almost .10. Increasing the Alienation and Perceptual Distortion scale score to 10 results in a probability of approximately .40. Lines labeled as "Sten = 9" and "Sten = 10" in Figure 1 represent Intellectual Efficiency dimension scores of nine and 10 respectively. As can be seen in Figure 1, when scores on this dimension are high, the probability of a critical incident increases very little, even as the Alienation and Perceptual Distortion scale score increases.

Discussion
As probation services continue to expand throughout the world, practice opportunities are also increasing exponentially for police and public safety psychologists to provide employee screenings and assessment services for probation departments. As previously noted, hiring the best and most aptly suited individuals to work as probation officers is of prime importance, and is an area of risk management that should be on the "radar screen" of every probation department and court administrator. Failure to select out unsuitable probation officer candidates can have grave consequences, especially now that many probation officers carry firearms in certain jurisdictions. While there are some obvious similarities between police and probation officer candidates, these vocational cohorts should not be construed as being simply synonymous with one another because important similarities and differences exist between them. 13 By extension, psychologists who conduct evaluations on probation officer candidates should not assume that psychological instruments shown to be valid and useful with police candidates will have equivalent validity with probation officer candidates. Psychologists should select and use only those assessment tools that have been demonstrated to have predictive validity with a specific population of interest.
The purpose of the present study was to determine the predictive validity of one psychological instrument, the PEPQ / PSR Plus, as a selection tool for use with probation officer candidates. This study specifically sought to determine whether PEPQ / PSR Plus scale scores and/or composite factor scores could be used to predict inadequate performance among a 13 see Herrmann & Broderick, 2011 Intellectual Efficiency Scores combined sample of new-hire and incumbent probation officers candidates. This study found empirical support indicating that the PEPQ / PSR Plus does indeed possess the ability to make meaningful predictions about probation officer candidate performance. Namely, the Alienation/Perceptual Distortion scale, the Interpersonal Relations composite scale, and the Intellectual Efficiency composite scale were all found to be sensitive predictors of future performance problems in a combined cohort of pre-employment and incumbent probation officers. Therefore, the answer to this study's research question is "yes," the PEPQ / PSR Plus does appear to be a valid selection tool for use with probation officer candidates. Because of the blended nature of our sample, the present results suggest that the PEPQ / PSR Plus may have applicability as both a pre-employment screening tool for new-hire probation officer candidates and also as a "fitness for duty" assessment tool for use with incumbent probation officers. However, further analysis is needed with both a larger sample size and a non-blended cohort (i.e., discrete populations of interests) in order to conclusively make this determination. Additionally, replication studies also need to be conducted outside of the United States in order to generalize these findings and, as such, we sound the call for such international replication studies to be initiated. Nevertheless, at present the PEPQ / PSR Plus is the only selection tool to date which has been shown to have predictive validity with probation officer candidates. For this reason, the PEPQ / PSR Plus should be given serious consideration for inclusion in any assessment battery by police and public safety psychologists charged with conducting psychological evaluations of probation officer candidates.