Schizophrenia Bulletin Advance Access originally published online on February 16, 2005
Schizophrenia Bulletin 2005 31(1):67-72; doi:10.1093/schbul/sbi007
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Learning Potential and the Prediction of Work Skill Acquisition in Schizophrenia
Assistant Professor, Department of Psychology, California State University, Northridge, Northridge, CA; and VA Greater Los Angeles Healthcare System, Los Angeles, CA
Associate Research Psychologist, Department of Psychiatry and Biobehavioral Sciences, Geffen School of Medicine, University of California, Los Angeles, VA Greater Los Angeles Healthcare System, Los Angeles, CA
Professor in Residence, Department of Psychiatry and Biobehavioral Sciences, Geffen School of Medicine, University of California, Los Angeles, and VA Greater Los Angeles Healthcare System
Professor in Residence, Department of Psychiatry and Biobehavioral Sciences, Geffen School of Medicine, University of California, Los Angeles, and VA Greater Los Angeles Healthcare System
Send reprint requests to Dr. M.J. Sergi, Department of Psychology, California State University, 18111 Nordhoff Street, Northridge, CA 91330–8255; e-mail: mark.sergi{at}csun.edu.
 |
Abstract |
|---|
 Top Abstract MethodResultsDiscussionReferences |
|---|
This study examined whether a measure of learning potential could predict work skill acquisition in schizophrenia beyond the prediction offered by a single administration assessment. Fifty-seven outpatients with schizophrenia or schizoaffective disorder completed a test-train-test version of the Wisconsin Card Sorting Test as a measure of their learning potential. The outpatients were randomly assigned to training by errorless learning or conventional instruction on two work skills (index card filing and toilet tank assembly). Work skills were assessed both immediately and 3 months after training. Generally, patients with high learning potential performed better on the work skill tasks. Learning potential explained an additional 15 percent of variance beyond single administration assessment in participants' accuracy immediately after work skill training and an additional 13 percent of variance in participants' accuracy 3 months after training. These findings indicate that measures of learning potential contribute to the prediction of work skill acquisition, going beyond the predictive power of single administration assessments.
Keywords: Schizophrenia / learning potential / dynamic testing / neurocognition / work skills / vocational skills
Neurocognitive impairments are associated with the daily functioning of persons with schizophrenia (Green 1996
; Green et al. 2000). These associations are, to a large extent, face valid. Deficits in verbal memory and vigilance may affect an individual's ability to converse with friends, family members, and employers. Problems in planning and self-monitoring may influence an individual's ability to catch the right bus or attend work or social events (Green et al. 1999).
Traditional neurocognitive assessments, those involving a single or static administration of neurocognitive measures, provide moderately good prediction of skill acquisition in schizophrenia (Mueser et al. 1991; Smith et al. 1999). Alternative testing approaches involving neurocognitive measures may yield even better prediction. Learning potential assessments, also called dynamic assessments, have been shown to predict the rehabilitation outcome of persons with schizophrenia, and their prediction may be superior to that of static assessments (Wiedl 1999; Wiedl et al. 2001).
Learning potential assessments use a test-train-test approach that frequently involves multiple administrations of commonly used neurocognitive measures. Between the two standard administrations of the measure, the participant may receive verbal instruction as well as feedback designed to increase performance on the measure. Learning potential is determined by the difference between the participant's pretraining and posttraining performances.
Because learning potential assessments involve multiple administrations, they require more time to conduct than single administration assessments. Clearly, they are worth administering only if they provide information above and beyond that of a single administration assessment. The primary aim of the present study was to determine whether a measure of learning potential could predict work skill acquisition in schizophrenia beyond the prediction offered by a single administration assessment. We hypothesized that a learning potential assessment involving multiple administrations of the Wisconsin Card Sorting Test (WCST; Heaton et al. 1993) would predict work skill acquisition when the prediction offered by a single administration of the WCST was also considered.
The present study was conducted within the context of a work skill acquisition study (Kern et al. 2002). After completing the learning potential assessment, patients completed 1 hour of training on two work skills (index card filing and toilet tank assembly) in a simulated workshop. Patients were randomly assigned to one of two forms of instruction (either errorless learning or conventional instruction). The effectiveness of these brief interventions was assessed both immediately after training and 3 months after training. The errorless learning method putatively minimizes reliance on episodic memory and yields higher levels of posttraining performance than the conventional approach. The work skill acquisition study provided an opportunity to examine the value of learning potential assessments across the two different types of training. A secondary aim of the present study was to examine whether learning potential assessments differentially predict work skill acquisition in one type of training approach versus the other. We hypothesized that, because errorless learning relies less on the cognitive capacities of patients, learning potential's ability to predict work skill acquisition would be attenuated in patients receiving errorless learning training.
|
Method |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
Participants.
Fifty-seven outpatients with schizophrenia (32 males and 8 females) or schizoaffective disorder (9 males and 8 females), diagnosed by interview with the Structured Clinical Interview for DSM–IV (First et al. 1997), participated in the present study after providing their written informed consent. Eight of the 65 outpatients who participated in the larger work skill acquisition study (Kern et al. 2002) did not complete the learning potential assessment, leaving 57 outpatients for the present study. All participants were recruited from the Department of Veterans Affairs Greater Los Angeles Healthcare System and the San Fernando Mental Health Center (Los Angeles County Mental Health). All participants were clinically stable outpatients who had not been hospitalized in the prior 6 months, had been maintained on the same medication for at least 3 months, and had been unemployed during the past year. Table 1 describes the demographic and clinical characteristics of the participants divided by training received. All comparisons were not significant. Participants who returned for the 3-month followup assessment and those who did not were also compared on the characteristics of table 1. Only one comparison yielded a significant difference: females more frequently failed to return for the 3-month followup assessment than their male counterparts.
|
Design.
Prior to training, all participants completed assessments of their learning potential, their psychiatric symptoms over the past 2 weeks (Brief Psychiatric Rating Scale; Ventura et al. 1993), and their reading ability (Wide Range Achievement Test—Revised; Jastak and Wilkinson 1984). All participants, in groups of two to four persons, completed approximately 1 hour of training on two work skills (index card filing and toilet tank assembly) in a simulated workshop. The form of instruction received by each participant (either errorless learning or conventional instruction) was determined by random assignment. The order of training and assessment of work skill tasks was counterbalanced across groups. The effectiveness of these brief interventions was assessed both immediately after training and 3 months after training. A more extensive description of the work skill acquisition study is available in Kern et al. (2002).
Learning Potential Assessment.
The assessment of learning potential involved a test-train-test version of the WCST (Heaton et al. 1993). The WCST is a measure of executive functioning and problem-solving skill that requires participants to match individually presented stimulus cards to one of four key cards. The cards can be matched according to the shape, number, or color of the symbols on each card. The 64-card computerized version of the WCST was administered three times in the present study. The first and third administrations held to the standard procedures for the WCST (i.e., correct-incorrect feedback only, no disclosure of categories). Following the procedures of Goldberg et al. (1987) and Green et al. (1992), participants were told of the three categories (shape, number, and color) prior to the second administration of the WCST and received verbal feedback during the second administration of the WCST (e.g., "That's right, we're matching to color," "That's not correct, we're not matching to shape, we must be matching to either number or color").
While prior studies of learning potential have used a categorical approach to identify participants as "high scorers," "learners," or "nonlearners" (e.g., Wiedl 1999), we used a dimensional approach involving gain scores to provide greater statistical power. Gain scores were calculated by dividing actual gain (posttraining performance – pretraining performance) by potential gain (perfect performance – pretraining performance). Pretraining performance was the number of cards correctly sorted in the first administration of the measure, and posttraining performance was the number of cards correctly sorted in the third administration of the measure. Because the WCST involves six changes in category that occur without warning, sorting 58 cards correctly was considered perfect performance.
Assessment of Work Skill Acquisition.
Two work skill tasks, index card filing and toilet tank assembly, were used to assess work skill acquisition at two time points: immediately after training and 3 months after training. These work tasks have standardized scoring procedures and good construct validity (Massel et al. 1990; Zarate et al. 1998). Participants were asked to sort as many index cards as they could and to assemble as many 31-piece toilet tanks as they could, each in a period of 55 minutes (order counterbalanced across training groups). By design, there was no limit to the number of cards that could be filed or tanks that could be assembled during the assessment period. Two indexes of work skill (accuracy and a performance index) were used for each task. Accuracy for each task was defined as the number of correct responses divided by total responses. The performance index for each task was defined as the number of correct minus incorrect responses. The latter index provides an overall measure of performance proficiency that reflects both speed and accuracy.
Statistical Analyses.
The data from both work skill tasks were standardized and combined with equal weighting to yield two dependent variables: accuracy and a performance index involving accuracy and speed. Two regression analyses (general linear mixed regression model with repeated measurement; SAS Institute 1990) were conducted to examine the utility of learning potential as a predictor of work skill acquisition and whether this predictive validity differed across the two training interventions. One regression analysis used accuracy as the dependent variable, and the other used the performance index as the dependent variable. The independent variables in each of the regression analyses were (1) learning potential gain score (assessed with the test-train-test version of the WCST), (2) training strategy (errorless learning or conventional), and (3) time of evaluation (immediately after training and 3 months after training). Learning potential gain score was entered as a continuous variable. To examine the predictive utility of learning potential beyond that of a single administration assessment, pretraining performance on the WCST (first administration of the WCST) was used as a covariate in each regression. Transformations were used to normalize skewed distributions. When learning potential interacted with other variables, a median split of learning potential was used to facilitate the interpretation of the interaction. Two stepwise multiple regression analyses were employed to determine the percentage of variance in work skill accuracy that was predicted by the measure of learning potential. In each multiple regression, the single administration assessment was entered first and the measure of learning potential was entered second. In essence, these are simple analyses of the main effect of learning potential controlling for the single administration assessment assessed across both training conditions.
|
Results |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
Table 2 displays zero order correlations between the measure of learning potential, the single administration WCST, and the work skill tasks immediately after training and 3 months after training. WCST-based learning potential and the single administration WCST are clearly independent. Also, the measure of learning potential is related to the accuracy and performance index measures of work skill both immediately after training and 3 months after training.
|
Learning Potential's Prediction of Work Skill
Accuracy. Learning potential predicted the accuracy of patient performance on the work skill tasks (main effect for learning potential, F(1, 52) = 9.69, p < 0.005). The single administration of the WCST was related to accuracy (F(1, 52) = 6.87, p < 0.02). Although the two-way interaction between learning potential and work skill training was not significant (F(1, 52) = 0.10, nonsignificant [ns]), a three-way interaction between learning potential, work skill training, and time of work skill assessment was significant (F(1, 41) = 4.95, p < 0.05). To examine the nature of the three-way interaction, we divided subjects into high and low learning potential based on a median split. Among participants with low learning potential (figure 1), those who received errorless learning training outperformed those who received conventional training at the immediate assessment (t(41) = 4.11, p < 0.0005) but not at the 3-month followup assessment (t(41) = 0.03, ns). The pattern of results was different among participants with high learning potential. Patients with high learning potential (figure 1) who received errorless learning training tended to outperform those who received conventional training both at the immediate assessment (trend level finding, t(41) = 1.82, p = 0.076) and at the 3-month followup assessment (trend level finding, t(41) = 1.99, p = 0.053). Despite this complex three-way interaction, the main effect for learning potential was quite strong. Those with high learning potential outperformed those with low learning potential in every combination of time x training intervention. This strong main effect is explored further in the regression analyses described below.
|
Performance index. Learning potential predicted the performance index of work skill (main effect for learning potential, F(1, 52) = 9.96, p < 0.005). The single administration of the WCST was not related to the performance index of work skill (F(1, 52) = 2.85, ns). Interactions between the independent variables were not observed. Notably, training strategy employed did not influence learning potential's prediction of the performance index of work skill (training strategy x learning potential, F(1, 52) = 0.09, ns).
Contribution of Learning Potential Versus Single Administration Assessment. Thirteen percent of the variance in the participants' accuracy on the work skill tasks immediately after training was explained by the single administration of the WCST. The measure of learning potential explained an additional 15 percent of the variance (p < 0.002). Three months after training, 6 percent of the variance in the participants' accuracy on the work skill tasks was explained by the single administration of the WCST, and the measure of learning potential explained an additional 13 percent of the variance (p < 0.01). This general summary of the main effect of learning potential must be viewed in the context of the significant three-way interaction described above (involving learning potential, time, and training strategy). Although higher learning potential was generally associated with better work skill acquisition (over and above single administration assessment), its effects were reduced at the immediate assessment among those trained with errorless learning. In other words, at least in the short term, errorless learning appeared to "equalize" performance without regard to learning potential; those effects were not, however, enduring.
|
Discussion |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
The current findings suggest that assessments of learning potential predict work skill acquisition in schizophrenia patients. A learning potential assessment based on the WCST predicted competence on work skill tasks measured by accuracy and a performance index based on speed and accuracy. In general, patients with high learning potential performed better on the work skill tasks than those with low learning potential. When accuracy was used as the outcome measure, prediction of performance involved interactions with training condition and time of assessment. Specifically, patients with low learning potential benefited from the errorless learning training at the immediate assessment but not at the 3-month followup assessment. Patients with high learning potential experienced a smaller benefit from errorless learning training that was consistent across time.
Consistent with our primary hypothesis, the WCST-based learning potential assessment had strong predictive utility for work skill acquisition even when controlling for the predictive utility of the single administration version of the WCST (i.e., the single administration version of the WCST was entered first in the analyses). Examination of the accuracy data indicated that the learning potential assessment explained an additional 15 percent of variance in work skill immediately after training and an additional 13 percent of variance in work skill 3 months after training.
Our second hypothesis received mixed support at best. We proposed that, because of the decreased cognitive demand of errorless learning training, learning potential would offer greater prediction of work skill acquisition for those participants who had received conventional training. In support of this hypothesis, learning potential influenced the participants' accuracy on the work skill tasks immediately after training for those who received conventional training but not those who received errorless learning training. However, effects involving learning potential and training strategy employed were not observed for accuracy at the 3-month assessment or for the performance index at either assessment. Considered collectively, the findings suggest that learning potential's prediction of work skill acquisition was not influenced by the training strategy employed.
The findings of the present study warrant continued investigation of learning potential's ability to predict important outcomes in schizophrenia. Whereas the current study found support for learning potential as a predictor of work skill acquisition from a single session of training, future studies should investigate the ability of learning potential to predict benefit from longer-term interventions. Another relevant issue concerns whether the WCST is the best test on which to determine learning potential. Because many studies of learning potential in schizophrenia have involved test-train-test versions of the WCST, we know much less about the predictive value of learning potential assessments based on other neurocognitive measures. The present findings support the extra time needed to administer dynamic measures (the dynamic WCST takes three times longer to administer than the static WCST). More stringent designs would compare the predictive utility of a dynamic measure to the predictive utility of a short battery of single administration measures from different neurocognitive domains, equated for time of administration.
|
Acknowledgments |
|---|
The present study was supported by the University of California, Los Angeles, Clinical Research Center for the Study of Schizophrenia (Robert P. Liberman, principal investigator); the Department of Veterans Affairs VISN–22 Mental Illness Research, Education, and Clinical Center (MIRECC); and National Institute of Mental Health grant MH–30911. The authors thank Sun Hwang, M.S., for her assistance with the statistical analyses.
|
References |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
-
First, MB, Spitzer, RL, Gibbon, M, Williams, JBW. The Structured Clinical Interview for DSM–IV Axis I Disorders—Patient Edition New York, NY Biometrics Research1997.
Goldberg, TE, Weinberger, DR, Berman, KF, Pliskin, NH, Podd, MH. Further evidence of dementia of the prefrontal type in schizophrenia? A controlled study of teaching the Wisconsin Card Sorting Test. Archives of General Psychiatry 44: pp. 1008–10141987.
Green, MF. What are the functional consequences of neurocognitive deficits in schizophrenia? American Journal of Psychiatry 153:321–3301996.
Green, MF, Kern, RS, Braff, DL, Mintz, J. Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the "right stuff"? Schizophrenia Bulletin 26:119–1362000.
Green, MF, Kern, RS, Sergi, MJ. An introduction to neurocognitive dysfunction in schizophrenia: Why should we care? In Keefe, RSE (Ed.). Improving Cognitive Function in the Schizophrenic Patient London, U.K. Science Press pp. 1–111999.
Green, MF, Satz, P, Ganzell, S, Vaclav, JF. Wisconsin Card Sorting Test performance in schizophrenia: Remediation of a stubborn deficit. American Journal of Psychiatry 149:62–671992.
Heaton, RK, Chelune, GJ, Talley, JL, Kay, GG, Curtiss, G. Wisconsin Card Sorting Test (WCST) Manual—Revised and Expanded Odessa, FL Psychological Assessment Resources1993.
Jastak, S and Wilkinson, GS. WRAT–R: Wide Range Achievement Test Administration Manual Los Angeles, CA Western Psychological Services1984.
Kern, RS, Liberman, RP, Kopelowicz, A, Mintz, J, Green, MF. Applications of errorless learning for improving work performance in persons with schizophrenia. American Journal of Psychiatry 159: pp. 1921–19262002.
Massel, HK, Liberman, RP, Mintz, J, Jacobs, HE, Rush, T, Giannini, C, Zarate, R. Evaluating the capacity to work of the mentally ill. Psychiatry 53:31–441990.[Web of Science][Medline]
Mueser, KT, Bellack, AS, Douglas, MS, Wade, JH. Prediction of social skill acquisition in schizophrenic and major affective disorder patients from memory and symptomatology. Psychiatry Research 37:281–2961991.[CrossRef][Web of Science][Medline]
SAS Institute. SAS, Version 6.12 Cary, NC SAS Institute1990.
Smith, TE, Hull, JW, Romanelli, S, Fertuck, E, Weiss, KA. Symptoms and neurocognition as rate limiters in skills training for psychotic patients. American Journal of Psychiatry 156: pp. 1817–18181999.
Ventura, J, Lukoff, D, Neuchterlein, KH, Liberman, RP, Green, MF, Shaner, A. Manual for the expanded brief psychiatric rating scale. International Journal of Methods in Psychiatric Research 3:227–2441993.
Wiedl, KH. Assessing cognitive modifiability as a supplement to readiness for rehabilitation in schizophrenic patients. Psychiatric Services 50:1411–14191999.
Wiedl, KH, Schoettke, H, Garcia, MDC. Dynamic assessment of cognitive rehabilitation potential in schizophrenic persons and in elders with and without dementia. European Journal of Psychological Assessment 17:112–1192001.[CrossRef]
Zarate, R, Liberman, RP, Mintz, J, Massel, HK. Validation of a work capacity evaluation in individuals with psychiatric disorders. Journal of Rehabilitation 64:28–341998.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Aubin, E. Stip, I. Gelinas, C. Rainville, and C. Chapparo Daily Functioning and Information-Processing Skills Among Persons With Schizophrenia Psychiatr Serv, June 1, 2009; 60(6): 817 - 822. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Watzke, P. Brieger, O. Kuss, H. Schoettke, and K. H. Wiedl A Longitudinal Study of Learning Potential and Rehabilitation Outcome in Schizophrenia Psychiatr Serv, March 1, 2008; 59(3): 248 - 255. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. W. Buchanan, D. C. Javitt, S. R. Marder, N. R. Schooler, J. M. Gold, R. P. McMahon, U. Heresco-Levy, and W. T. Carpenter The Cognitive and Negative Symptoms in Schizophrenia Trial (CONSIST): The Efficacy of Glutamatergic Agents for Negative Symptoms and Cognitive Impairments Am J Psychiatry, October 1, 2007; 164(10): 1593 - 1602. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. L. Hake, E. Hamera, and M. Rempfer Self-report of Cognitive Functioning and Learning in Schizophrenia Journal of the American Psychiatric Nurses Association, April 1, 2007; 13(2): 93 - 100. [Abstract] [PDF]
|
|
|
|
 |
|
 D. Koren, L. J Seidman, M. Goldsmith, and P. D Harvey Real-World Cognitive--and Metacognitive--Dysfunction in Schizophrenia: A New Approach for Measuring (and Remediating) More "Right Stuff" Schizophr Bull, April 1, 2006; 32(2): 310 - 326. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||