Schizophrenia Bulletin Advance Access originally published online on August 17, 2005
Schizophrenia Bulletin 2005 31(4):810-815; doi:10.1093/schbul/sbi046
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Applying New Approaches From Cognitive Neuroscience to Enhance Drug Development for the Treatment of Impaired Cognition in Schizophrenia
Department of Psychiatry, University of California at Davis, Imaging Research Center, 4701 X Street Sacramento, CA 95817
| Abstract |
|---|
New approaches to the measurement of cognition in schizophrenia include the use of tasks from experimental cognitive psychology to examine the integrity of specific cognitive systems and the application of these tasks in noninvasive neuroimaging (e.g., functional magnetic resonance imaging [fMRI]) studies that directly measure the effects of drugs on cognition-related brain activity. These approaches offer many advantages, including the isolation of specific cognitive systems that may be conserved across species; controlling for the confounding effects of generalized performance deficits such as poor motivation, sedation, and so on; and providing a direct translational bridge from studies using animal models of cognition to patient-based research using fMRI. These developments have the potential to transform the early human phases of drug development and streamline the decision making at this critical point in the process. As was the case for the Measurement and Treatment Research to Improve Cognition in Schizophrenia initiative, optimizing the application of cognitive neuroscience to new drug development will require a major commitment by multiple investigators to task development and a thorough psychometric evaluation of both behavioral and neuroimaging measures.
Keywords: Cognitive psychology / cognitive neuroscience / neuropsychology / schizophrenia / cognitive enhancement
| Introduction |
|---|
With the growing recognition of the disabling role that cognitive deficits play in schizophrenia1 and of the relatively limited effects of currently available antipsychotic therapies on this aspect of the illness, the National Institute of Mental Health (NIMH) sponsored the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. Drawing upon a decade of research measuring drug effects on cognition in schizophrenia in clinical trials of second-generation antipsychotics, and several decades of experience using clinical neuropsychological methods, a battery of tasks was selected, norming studies have been completed, and the plans for the first Treatment Units for Research on Neurocognition and Schizophrenia network studies are well advanced. With renewed interest from the pharmaceutical industry and the support of the Food and Drug Administration (FDA), this remarkable effort, taking less than 2 years, may well lead to a small-scale "Manhattan Project" focusing of the development of novel therapies for impaired cognition in schizophrenia. This effort has already contributed to a broader reconceptualization of the treatment of schizophrenia that goes beyond relapse prevention and symptom control to focus on reducing disability and improving functional outcome.
The timely development of the MATRICS battery drew upon recent experience studying the treatment effects on cognition in schizophrenia and hence represents a best current practice. Throughout the process there was much discussion of the contribution that could be made by more novel approaches to studying cognition, including the use of experimental cognitive tasks and noninvasive neuroimaging methods. This article will discuss the advantages and limitations of these approaches together with an analysis of how they may contribute to the process of drug discovery and at what stages of the process they are likely to be most helpful.
| Two Distinct Approaches to Measuring Cognition in Schizophrenia |
|---|
The history of the investigation and measurement of cognition in schizophrenia has been undertaken using 2 distinct approaches, 1 based in clinical neuropsychology, and a second based in experimental cognitive psychology. Clinical neuropsychology uses batteries of standardized tests with generally well-established psychometric properties to measure cognitive abilities. Individual test scores are often combined to make composite scores based upon factor analyses, and cognitive domains, such as attention, executive functions, memory, and so on, are identified using this purely empirical approach. In contrast to the standardized tests used in clinical neuropsychology, the experimental cognitive approach uses highly refined tasks that have been developed to examine the function of specific cognitive systems. These theoretical model systems have been conceptualized, specified in mechanistic terms, and validated using exhaustive experimentation in which task parameters are varied and performance is measured to test predictions arising from the models. This approach has been used by experimental psychologists to study human cognition for over 60 years, which is reflected in the fact that it is rare to find a psychology undergraduate who has not, for course credit, contributed several hours of her or his time to this endeavor. Since this experimental approach involves constantly designing and modifying tasks, there is no standardization of task design or administration. Except in a few areas of individual differences research, relatively little has been done to understand the measurement properties of experimental cognitive tasks.
The majority of the tasks (with the exception of the vigilance, problem-solving/reasoning, and social cognition tasks) included in the MATRICS battery are clinical neuropsychological measures that have been used for decades in clinical practice and research and also quite widely in clinical trials of second-generation antipsychotic effects on cognition in schizophrenia. These tests have many desirable properties, including excellent measurement properties reflected in stable test-retest reliability and a relative lack of floor and ceiling effects, critical for measuring treatment effects. The tests are also selected from a range of distinct domains of cognitive function that have emerged in previous factor analytic analyses and are assumed to reflect distinct functional domains.
Given the experience that the field has with clinical neuropsychological tasks and their desirable psychometric properties, why should we be considering a shift to a more cognitively mechanistic approach as we look to move the field of drug discovery for the treatment of cognitive deficits in schizophrenia forward in the future? This article will argue that we must commit to what may amount to a paradigm shift, in the context of also establishing a dialogue between cognitive and clinical neuropsychological approaches, for 2 critical reasons. The first is that the experimental cognitive approach allows us to examine the function of specific cognitive systems. By this I mean that it is possible, with this approach, to distinguish the function of 1 specific cognitive system (e.g., working memory, long-term relational memory, or different kinds of attention such as vigilance, focused attention, and selective attention) from another and to distinguish between these specific cognitive deficits and the generalized deficits that schizophrenia patients manifest as a result of reduced motivation, sedation from medications, general inattentiveness, and poor test-taking skills. The second reason is that studying specific cognitive mechanisms in this way is the approach that both animal modelers and human cognitive and affective neuroscientists use. If we are to develop a translational neuropharmacology of schizophrenia that involves bridging cognitive and neuroimaging studies of human subjects, including patients with schizophrenia, and drug development that relies on animal models, it is not a matter of whether we need to bring these 2 fields into closer contact but, rather, how.
| The Measurement of Specific Cognitive Functions |
|---|
A limitation of clinical neuropsychological tests is that in every case they are highly complex in their structure and engage a range of cognitive mechanisms. If we consider the Wisconsin Card Sorting Task (WCST; see figure 1), this complexity is readily apparent. Subjects are presented with a set of cards with 3 different features (shape, color, and number) and are instructed to sort the cards without being told the relevant feature. They must learn this by trial and error based upon feedback from the tester. Once they have established the correct principle the rule is switched unannounced, and subjects must learn the new principle based upon feedback. The WCST is 1 of the most reliable neuropsychological tasks for measuring differences between schizophrenia patients and healthy subjects, making it a sensitive measure of cognitive deficits in schizophrenia. However, an analysis of the component cognitive mechanisms engaged by this task reveals a wide range of cognitive functions that must be engaged in order to successfully perform. Subjects must attend to and discriminate among all relevant features; they must process the feedback they receive and learn from their errors; and they must select the appropriate sorting response, shift sets when the rules change, inhibit an established response tendency when the rule changes, and maintain a representation of their recent behavior in working memory.
|
The complexity of standardized clinical neuropsychological tests has direct implications for their ability to provide sensitive measurement of drug effects on cognition in schizophrenia. For example, if a drug has been developed in an animal model of working memory, it might actually improve this system but not show an effect on a complex task such as the WCST because the other cognitive systems that also support task performance are unaffected by the drug. Variance from cognitive mechanisms unaffected by the drug but engaged by a task may limit sensitivity to drug effects on a specific cognitive system. The corollary to this is that if a task can provide a relatively factor-pure measure of a specific cognitive system, this is likely to increase the fidelity of the signal originating from the effects of a drug on its biological substrate.2
| Distinguishing Specific Cognitive Deficits From Generalized Performance Deficits |
|---|
In 1973 psychologists Loren and Jean Chapman pointed out a fundamental flaw in most studies of cognition in schizophrenia, which they refer to as "the psychometric confound."34 They note that cognitive tests differ in their sensitivity to individual differences, a property referred to as discriminating power. This psychometric property in turn is a function of the test's difficulty (or more strictly, effect size) as well as its reliability. Highly reliable tests producing large effect sizes have high discriminating power. They also point out that tests with high discriminating power are highly sensitive to generalized performance deficits that arise from what are mostly considered to be nuisance factors such as poor motivation, sedation or fatigue, general inattentiveness, or poor test-taking skills.
This issue is best illustrated by a couple of examples. First we have an anecdotal one. Consider that we are interested in measuring a person's ability at golf. We could construct a test that compares simple motor skills to advanced golf ability by first having subjects sink a 3-inch putt and then requiring them to shoot for the green from 180 yards using a 5 iron. A poor golfer is likely to sink the putt but do badly off the fairway; a good golfer will do well at both. However, consider that a good golfer is very tired, has poor coordination because he or she was recruited from the nineteenth hole, or is a reluctant or unmotivated participant in the test. That individual will also likely be able to sink the putt but miss the fairway shot, because the greater discriminating power of this aspect of the test will be more sensitive to his or her generalized performance deficit.
For a more concrete example, in a study of spatial working memory, unmedicated schizophrenia patients were required to maintain the location of a dot appearing randomly at a location on a circumference and, after either a very brief or a several-second delay, identify the location by naming which letter in a random array around the circumference marked the location of the dot.5 Healthy control subjects showed a small but significant decrement in performance across the delay; patients showed a significantly larger delay-related performance decrement. The more cognitively interesting interpretation of this result is that the significant interaction between group and delay was indicative of a spatial working memory deficit in schizophrenia. However, this claim needed to be tempered in the discussion of this result, because in healthy subjects performance was worse on the long than the short condition, suggesting that this condition had greater discriminating power and hence increased sensitivity to a generalized deficit (see figure 2). Interestingly, the performance deficit shown by patients in this study was correlated with their level of negative symptoms, lending some credence to the possibility that the deficit being measured reflected poor motivation.
|
Differences in discriminating power across tasks confound the interpretation of many cognitive studies of schizophrenia as well as studies of treatment effects in schizophrenia because our most sensitive and psychometrically stable tests are highly sensitive to generalized deficits. This can undermine our ability to measure the integrity and the effects of treatments on cognitive processes in schizophrenia. The traditional practice in neuropsychology of using standardized scores and creating "profiles" does not address this problem. This is because while standardized scores represent performance on all tasks on the same metric, they do not change the fact that different tasks or groups of tasks differ in their discriminating power; and this, rather than a deficit in a specific domain, may result in an apparent profile of differential deficits across ability domains that is actually reflecting the presence of a generalized performance deficit.
It is important to recognize that this problem is not unique to neuropsychological studies, since the example above uses a theoretically motivated experimental cognitive task, and many studies using this approach are also confounded by this problem, despite the work of the Chapmans and others6 to raise our awareness of this confound as well as of potential solutions. When experimental cognitive tasks are designed to be used to measure cognition in schizophrenia and other cognitive disorders, due consideration needs to be given to building in controls for this confound. However, because of the task standardization inherent in clinical neuropsychology this is a particularly difficult problem to address within this framework. Most solutions to the problem of distinguishing between specific and generalized deficits involve either adjusting the parameters of the different tasks being used to measure various cognitive domains so that they have equivalent discriminating power or developing new measures with an internal structure that will provide performance indexes that can distinguish between the 2 kinds of deficit.67 The experimental cognitive approach, with its flexible and theoretically motivated approach to task design and implementation, provides a straightforward framework for the development of measurement approaches that can distinguish between a generalized and a specific deficit. Finally, it is also important to note that what we consider to be generalized deficits, such as negative symptoms, may also be isolated and measured using a cognitive experimental approach, so there is no need to "throw out the baby with the bathwater," to the degree that these deficits are relevant for understanding disability in schizophrenia and serving as treatment targets in their own right.
| Developing a Translational Neuropharmacology of Cognition in Schizophrenia |
|---|
Perhaps the most compelling reason for taking the experimental cognitive approach to investigate and measure cognition in schizophrenia is that these methods and constructs are the building blocks for animal pharmacology and for cognitive and affective neuroscience. Noninvasive imaging studies can allow us to measure drug effects on cognitive and emotional processes and on neural substrates.8 Functional magnetic resonance imaging (fMRI) may be used to measure brain activity in circuits supporting specific cognitive functions with a relatively high degree of anatomical specificity, while other methods such as event-related potentials and magneto-encephalography can index the time course and magnitude of cognitive processing in the brain. Furthermore, new methods have recently been developed to measure the presence of cortical oscillations in specific frequency bands associated with cognitive processing.910
A useful example of how cognitive neuroimaging may be used to establish connections across species and from animal models of disease to studies of ill humans may be taken from work involving cholinergic modulation of cognitive impairments in mild cognitive impairment (MCI) and Alzheimer's disease (AD). A substantial body of data from animal models suggests that neural systems involved in normal working memory are supported in part via cholinergic neurotransmission.1112 Given that working memory is an important cognitive system that is impaired in patients with AD and that some animal models of AD involve compromised cholinergic function,13 one might hypothesize that some of the positive effects through which cholinesterase inhibitors might benefit cognition in individuals with MCI and AD might be associated with enhanced function in working memory circuits. Indeed, numerous studies have shown that these agents improve working memory performance and enhance activity in working memoryrelated cortical circuitry in healthy subjects.1415 Furthermore, studies of patients with MCI16 and AD17 have confirmed that treatment with these agents does lead to enhanced activation (increased fMRI blood oxygen leveldependent signal) in frontal and parietal systems supporting working memory in association with improved working memory performance.
From this example it should be seen that it is increasingly possible to embark on translational research from the "top down" using human volunteers and cognitively impaired patients in pharmaco-fMRI studies, as well as from the "bottom up" using animal models of specific cognitive functions and potentially using the same kinds of hemodynamic and electrophysiological measures to complement and inform measures of behavioral performance.
| Challenges to Developing Technologies From Cognitive Science and Cognitive Neuroscience Into Useful Tools for Psychopharmacology |
|---|
Despite the case that may be made for moving toward an experimental cognitive and cognitive neuroscientific approach to measuring drug effects on cognition in schizophrenia, there are a number of challenges that must be overcome. For example, the flexibility of the experimental cognitive approach also means that there are no standard forms of any task. Having standard versions of tasks together with consistent administration will be critical for clinical studies in which patients will most likely be tested at multiple sites. Task selection, optimization, and standardization for these purposes will require as much care and consensus as went into the MATRICS process. A second major issue is psychometrics. Relatively little is known about such properties as test-retest reliability for most experimental cognitive tasks, even those that have been widely used in cognitive psychology for decades. Many such tasks may be of little use in psychopharmacological studies because there may be ceiling effects or other undesirable test properties. Many of these issues are empirical questions that can only be addressed by conducting the kinds of norming studies, where relatively large numbers of subjects undergo repeated testing, undertaken during the development of the MATRICS battery. It is likely that experimental cognitive measures may never be as stable as traditional neuropsychological measures. It will be important, therefore, to involve psychometricians in the process of conducting this basic work on understanding and optimizing the measurement of specific cognitive processes and establishing minimal criteria for stability based upon established standards from the social sciences.
A second set of challenges arises when experimental cognitive tasks are incorporated into noninvasive neuroimaging studies of the neural correlates of task performance. While significant progress has already been made using this approach,8 and its potential for facilitating translational research is compelling, relatively little formal information is available about the reliability of these brain-based measures. This suggests the need for basic "neurometric" research in which effect sizes and reliability data are measured at the voxel and region level in noninvasive imaging studies, using the same general approach that is undertaken to evaluate promising cognitive measures.
Understanding the measurement properties of cognitive neurosciencebased measures will involve fairly large-scale studies both within and across sites. Substantial progress has been made in developing technical solutions to problems associated with pooling data acquired on different MRI scanners,18 making this approach feasible at this point. However, such studies will require significant cognitive and neuroimaging expertise as well as a great deal of time-consuming effort. In a recent review Honey and Bullmore describe the results of over 50 pharmaco-fMRI studies that have investigated the effects on brain activity of a range of different neuromodulatory drugs across a range of cognitive domains in both healthy subjects and patient groups.8 Compelling solutions have been developed to address interpretive issues related to the effects of drugs on task performance and the potential for drugs to affect the coupling between neuronal activity and hemodynamic-based signals. However, as noted above, in order to gain the confidence of industry and other key groups in the use of these measures it will be important to obtain more basic data on the reliability of these tools.
| GoNo Go: Cognitive Neuroscience and Accelerated Drug Discovery for Cognition in Schizophrenia |
|---|
MATRICS has successfully developed a battery of tasks that can be used for phase II and III studies of potential promising therapies for cognition in schizophrenia. The application of a cognitive neuroscientific approach of the kind described above offers unprecedented new opportunities for refining and accelerating drug discovery for impaired cognition in schizophrenia. For example, a critical and potentially costly and time-consuming juncture in the development of a new therapy occurs 1 step earlier in the process than the clinical trial phase, when companies must decide whether to take a drug forward into costly human clinical trials based upon preclinical data and preliminary human toxicology. Pharmacological fMRI may provide a very helpful tool at this juncture. Based upon the example described above for cholinesterase inhibitor effects on working memory systems in the healthy and dementing human brain, if a drug performs well in an animal model and is safe in humans but has no impact on the human circuits supporting the targeted cognitive system, this should raise concerns regarding its likely performance in clinical trials. The integration of cognitive imaging with more refined animal models has the potential to transform decision making in the early phases of drug development by providing concrete data on drug effects on cognition-related brain circuitry in schizophrenia.
| Conclusions |
|---|
Significant progress has been made over the past several years in recognizing the functional significance of cognitive impairments in schizophrenia and the importance of developing treatments for this disabling aspect of the illness. Through the MATRICS process a critical alliance has been forged among academia, industry, and the FDA and a set of tools has been developed for use in clinical trials of therapies. As the field moves forward we will have the opportunity to incorporate new approaches to the measurement of cognition in schizophrenia, building on the "cognitive revolution" of experimental psychology and the new neuroscience that has been born out of this. We will have the opportunity to refine our measurement of cognition in schizophrenia to focus on more precisely specified and measured cognitive mechanisms and to distinguish between disturbances in these functions and generalized deficits such as diminished motivation, drug-induced sedation, and the like. This new approach will also allow us to measure, directly, drug effects on the altered neural activity that underlies impaired cognition in patients with schizophrenia using methods such as fMRI.
Considerable development is needed in order to realize these possibilities, most importantly an ongoing collaboration between cognitive neuroscientists and psychometricians. It may well be that it is through such a collaboration that a new, interdisciplinary approach to measuring cognition in schizophrenia will emerge that will combine the specificity of cognitive psychological measures and their growing links to neural systems with optimal psychometric properties. The potential reward for this effort will be a more targeted and streamlined process of drug discovery and development that will address, more effectively, the challenge of developing effective therapies for impaired cognition in schizophrenia.
| Footnotes |
|---|
To whom correspondence should be addressed; tel: 916-734-7783, fax: 916-734-0750, e-mail: cameron.carter{at}ucdmc.ucdavis.edu.
| Acknowledgments |
|---|
This work was supported by Burroughs Wellcome Translational Clinical Scientist Award 1002274 and National Institute of Mental Health grants 5R01MH059883, 7R01MH066629, and 7K02MH064190.
| References |
|---|
- Green, MF. What are the functional consequences of neurocognitive deficits in schizophrenia? Am J Psychiat 1996; 153:321330.
[Abstract/Free Full Text] - MacDonald, AW III and Carter, CS. Cognitive experimental approaches to investigating impaired cognition in schizophrenia: a paradigm shift. J Clin Exp Neuropsychopharm 2002; 24:873882.
- Chapman, LJ and Chapman, JP. Problems in the measurement of cognitive deficit. Psychol Bull 1973; 79:380385.[CrossRef][Web of Science][Medline]
- Chapman, LJ and Chapman, JP. The measurement of differential deficit. J Psychiat Res 1978; 14:303311.[CrossRef][Web of Science][Medline]
- Carter, CS, Robertson, LC, Nordahl, TE, Kraft, L, Chaderjian, M, Oshora-Celaya, L. Spatial working memory deficits and their relationship to negative symptoms in unmedicated schizophrenia patients. Biol Psychiat 1996; 40:930932.[CrossRef][Web of Science][Medline]
- Strauss, ME. Demonstrating specific cognitive deficits: a psychometric perspective. J Abnorm Psychol 2001; 110:614.[Medline]
- Knight, RA and Silverstein, SM. A process-oriented approach for averting confounds resulting from general performance deficiencies in schizophrenia. J Abnorm Psychol 2001; 110:1530.[Medline]
- Honey, G and Bullmore, E. Human pharmacological MRI. Trends Pharmacol Sci 2004; 25:366374.[CrossRef][Medline]
- Green, MF and Nuechterlein, KH. Cortical oscillations in schizophrenia: timing is of the essence. Arch Gen Psychiat 1999; 56:10071008.
[Free Full Text] - Tallon-Beaudry, BO and Peronnet, FPJ. Induced gamma band activity during the delay of a visual short term memory tasks in humans. J Neurosci 1998; 18:42444254.
[Abstract/Free Full Text] - Dougherty, KD, Turchin, PI, Walsh, TJ. Septocingulate and septohippocampal cholinergic pathways: involvement in working/episodic memory. Brain Res 1998; 9:5971 12.
- Barch, DM. Pharmacological manipulation of human working memory. Psychopharmacology 2004; 174:12635.[Medline]
- Fisher, A, Brandeis, R, Chapman, S, Pittel, Z. Michaelson DM. M1 muscarinic agonist treatment reverses cognitive and cholinergic impairments of apolipoprotein E-deficient mice. J Neurochem 1998; 70:19911997.[Medline]
- Furey, ML, Pietrini, P, Haxby, JV. Cholinergic enhancement and increased selectivity of perceptual processing during working memory. Science 2000; 22:23152319 5500.
- Bentley, P, Husain, M, Dolan, RJ. Effects of cholinergic enhancement on visual stimulation, spatial attention, and spatial working memory. Neuron 2004; 25:969982 6.
- Saykin, AJ, Wishart, HA, Rabin, LA, et al. Cholinergic enhancement of frontal lobe activity in mild cognitive impairment. Brain 2004; 127:15741583.
[Abstract/Free Full Text] - Rombouts, SA, Barkhof, F, Van Meel, CS, Scheltens, P. Alterations in brain activation during cholinergic enhancement with rivastigmine in Alzheimer's disease. J Neurol Neurosur Ps 2002; 73:665671.
[Abstract/Free Full Text] - Turner, J, Potkin, S, Brown, GG, et al. Multi-center fMRI methods and design: function BIRN. Schizophrenia Bull 2005; 31:437.
This article has been cited by other articles:
![]() |
D. Dickinson and P. D. Harvey Systemic Hypotheses for Generalized Cognitive Deficits in Schizophrenia: A New Take on An Old Problem Schizophr Bull, March 1, 2009; 35(2): 403 - 414. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Lewis, R. Y. Cho, C. S. Carter, K. Eklund, S. Forster, M. A. Kelly, and D. Montrose Subunit-Selective Modulation of GABA Type A Receptor Neurotransmission and Cognition in Schizophrenia Am J Psychiatry, December 1, 2008; 165(12): 1585 - 1593. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Dickinson and J. M. Gold Less Unique Variance Than Meets the Eye: Overlap Among Traditional Neuropsychological Dimensions in Schizophrenia Schizophr Bull, May 1, 2008; 34(3): 423 - 434. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. S. Carter and D. M. Barch Cognitive Neuroscience-Based Approaches to Measuring and Improving Treatment Effects on Cognition in Schizophrenia: The CNTRICS Initiative Schizophr Bull, September 1, 2007; 33(5): 1131 - 1137. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Stover, S. Marder, and W. Carpenter Progress on NIMH Initiatives (Memorial Theme for Wayne Fenton, MD) Schizophr Bull, September 1, 2007; 33(5): 1084 - 1085. [Full Text] [PDF] |
||||
![]() |
E. L. Stover, L. Brady, and S. R. Marder New Paradigms for Treatment Development Schizophr Bull, September 1, 2007; 33(5): 1093 - 1099. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. S. CARTER Understanding the Glass Ceiling for Functional Outcome in Schizophrenia Am J Psychiatry, March 1, 2006; 163(3): 356 - 358. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||



