The NIMH initiative titled "Measurement and Treatment Research to Improve Cognition in Schizophrenia" (MATRICS) was designed to support the development of pharmacological agents for improving the neurocognitive impairments that are a core feature of schizophrenia. These cognitive deficits affect nearly every individual with schizophrenia. Moreover, the severity of these cognitive deficits is closely related to the severity of functional impairments that result in poor community adaptation and work disability for the large majority of individuals affected by this illness (see Appendix C). Although available pharmacological treatments are effective for reducing the severity of the psychotic symptoms of schizophrenia, they are relatively ineffective in reducing the severity of neurocognitive deficits and associated functional impairments.
The goal of MATRICS was to advance the development of drugs for treating these impairments in schizophrenia by addressing important obstacles that have interfered with drug development (see Appendix F). Among these barriers were:
The MATRICS deliverables included in this summary represent the work of experts from academia, NIMH, FDA, and the pharmaceutical industry. This summary includes these deliverables in brief form and the accompanying materials provide detailed support for the recommendations.
|1. Reliable and valid assessment of cognition at the level of all individual major cognitive domains|
|2. Inclusion of the following cognitive domains: Speed of Processing, Attention/Vigilance, Working Memory, Verbal Learning and Memory, Visual Learning and Memory, Reasoning and Problem Solving, and Social Cognition|
|3. High test-retest reliability|
|4. High utility as a repeated measure|
|5. Demonstrated relationship to functional outcome|
|6. Demonstrated tolerability and practicality|
The assessment of cognition at the level of cognitive domains was also seen as necessary because schizophrenia is characterized by a particular pattern of deficits across cognitive domains that distinguishes the illness from dementia, major depression, and bipolar disorder. It was recognized that it will be necessary to evaluate the effects of potential cognition-enhancing drugs across domains because there may be differential effects across domains. Indeed, it is possible that a pharmacological agent will improve some cognitive domains and impair others, and this possibility would need to be evaluated in any FDA submission. An additional reason for assessing cognitive domains is that some effective agents may target a particular subset of domains rather than all aspects of cognition. Thus, a battery involving reliable, valid assessment of individual cognitive domains would allow an option of more specific cognitive endpoints, for example, if preliminary data showed that a drug had specific effects on memory.
This strong recommendation to assess cognition at the level of individual cognitive domains does not prevent companies from using a summary score derived from the seven domains as its outcome measure. It does mean that the cognitive battery should always assess all of the relevant domains in clinical trials, even if a company is predicting effects on only a subset of domains (e.g., the memory domains). If an overall cognition summary score is to be used, it should be based on equal weighting of the seven cognitive domains.
To delineate the major dimensions of cognitive deficit in schizophrenia, Keith Nuechterlein, Co-Chair of the Neurocognition Committee, chaired a subgroup of Committee members whose task was to review the literature and to recommend key separable cognitive domains. The following principles were emphasized:
The Neurocognition Committee initially considered an additional criterion for test selection: the likelihood of measuring change by psychopharmacological agents. This criterion was dropped during the RAND Panel Evaluation because tests for the cognitive consensus battery will be used with innovative drugs that may work by novel mechanisms. The RAND Panel was reluctant to estimate a test's future sensitivity to effects of new drugs based solely on the limited range of drugs that have been examined so far.
The steps for reaching a consensus cognitive battery are shown in the Figure and discussed in Appendix D. Following the MATRICS meeting in April, 2003, the MATRICS Neurocognition Committee discussed each of the roughly 90 neurocognitive tests that were nominated as measures within one of the seven cognitive domains. Through consideration of what the Committee already knew about the tests in relationship to the test selection criteria, it narrowed the number of tests to six or fewer per domain (a total of 36 tests). MATRICS staff systematically conducted a comprehensive review of the literature bearing on the performance of each of these 36 tests across each of the five criteria for test selection (materials provided at www.matrics.ucla.edu). This comprehensive review constituted the database for a RAND Panel meeting held in September 2003. At the RAND Panel meeting, 14 experts with various areas of expertise were selected by the Neurocognition Committee and used the RAND Panel Method to rate each of the candidate tests on each of the essential criteria. These ratings of the panelists are listed in Appendix I. The ratings from the RAND Panel process were provided to the MATRICS Neurocognition Committee and led to the selection of a "beta" version of the consensus cognitive battery. This beta version of the battery was intentionally over-inclusive and included the 20 most promising tests, requiring roughly twice as long to administer as the final battery. A 5-site MATRICS Psychometrics and Standardization Study (MATRICS-PASS) directly compared the tests on their psychometric properties, their relationships to functional outcome, and practicality / tolerability to select the tests in each domain for the final NIMH consensus cognitive battery for clinical trials. The psychometric data for the cognitive tests from MATRICS-PASS are included in Appendix J. Based on careful consideration of these data, the MATRICS Neurocognition Committee selected and endorsed The MATRICS Consensus Cognitive Battery, which is listed below in Table 2. It is the MATRICS recommendation that this battery be the standard instrument for all clinical trials of cognition-enhancing agents in schizophrenia. The final step in this process is the collection of data on a community standardization sample. This phase is ongoing and is expected to be completed in the summer of 2005.
The MATRICS approach to selecting the neurocognitive battery emphasized the selection of cognitive tests with psychometric properties that were best suited for clinical trials. These measures needed to have demonstrated their usefulness for measuring impairment in schizophrenia through documented high test-retest reliability, utility as repeated measures, relationship to functional outcome, and practicality and tolerability. The current selection of measures that best demonstrated these characteristics should not in any way discourage vigorous pursuit of advances in cognitive science and cognitive neuroscience that will lead to refined neurocognitive measures in the future. Indeed, developments in these fields are expected to lead to improved cognitive measures for a future version of a consensus cognitive battery. As was emphasized in the MATRICS New Approaches Conference, an increased emphasis on the psychometric properties of refined measures arising from cognitive science and cognitive neuroscience will aid incorporation of such measures. Including promising new measures in studies of cognition in schizophrenia and in clinical trials will help to demonstrate instances in which these newer measures have advantages as compared to the instruments in the MATRICS consensus battery. This, in turn, will provide data that will be vital for updating the battery in years to come.
|The final composition of the battery will depend on reaching appropriate agreements for licensing and distribution of tests with the respective test developers and publishers. Here is a listing of the tests that we anticipate will comprise the final battery.|
|Speed of Processing||Admin Time (min)|
|Brief Assessment of Cognition in Schizophrenia (BACS) - Symbol-Coding||3.0|
|Trail Making A||2.1||Attention/Vigilance|
|Continuous Performance Test - Identical Pairs (CPT-IP)||13.4|
|University of Maryland - Letter-Number Span||5.9|
|Wechsler Memory Scale (WMS) - III Spatial Span||5.1|
|Hopkins Verbal Learning Test (HVLT) - Revised||4.1|
|Brief Visuospatial Memory Test (BVMT) - Revised||4.7|
|Reasoning and Problem Solving|
|Neuropsychological Assessment Battery (NAB) - Mazes||11.2|
|Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) - Managing Emotions||12.0|
|Estimated Administration Time of Cognitive Battery:||63.5|
The MATRICS Initiative did not originally include the need to develop recommendations for co-primary measures for use in clinical trials in addition to a cognitive performance measure. The issue of co-primary measures came up in discussions with Thomas Laughren and Robert Levin from the FDA starting with the first MATRICS consensus meeting in April 2003. In these discussions, Drs. Laughren and Levin raised serious questions about the likelihood that changes in cognitive performance tests would be sufficient evidence alone to support a claim for the efficacy of a pharmacological agent. At the same time, they recognized that changes in community-based functional outcomes, such as work or social activity, were unlikely to occur during a clinical trial of three to six months. These community-based functional changes often lag behind cognitive performance change and can also be affected by community factors beyond the scope of pharmacological intervention (e.g., employment opportunities). Soon after the first MATRICS consensus conference, the MATRICS leadership appointed a task force on outcomes that was chaired by Alan Bellack to review potential functional and co-primary measures.
This task force on outcomes focused on two approaches for selecting co-primary measures for clinical trials. One approach is to use standardized tests of "functional capacity," which specifically assess whether an individual has the capacity for performing key tasks of daily living (e.g., preparing a meal, taking mass transportation, managing medications, demonstrating social competence). Assessments of functional capacity are simulated activities, conducted in the clinic, not the community (Appendix G). While such assessments do not guarantee that an individual actually performs the tasks in the community, they are less influenced by intervening variables and may be appropriate for clinical trials of cognitive enhancing agents. The second approach to finding a co-primary measure involves an interview-based assessment of cognition in which patients rate their own cognitive abilities, or caregivers provide their impressions of a subject's cognitive level. Both of these types of potential co-primary measures were considered to have high face validity for patient improvement and could possibly change in a reasonably short time in the presence of a potent cognition enhancer.
The characteristics of a good co-primary measure for use in clinical trials were discussed at the MATRICS FDA-NIMH advisory meeting (see Appendix A). It was decided that a co-primary measure should have the following characteristics: a) good face validity for patient improvement; b) high likelihood of change in close temporal proximity to changes on cognitive performance measures; c) absence of strong dependence on range of rehabilitation opportunities and level of social support; d) practicality (from the perspective of the measure administrator) and tolerability (from the perspective of the patient); and e) adequate reliability and validity. To demonstrate adequate reliability and validity, the co-primary measure should meet the following conditions: a) good test-retest reliability; b) demonstrated associations with cognitive performance measures; and c) demonstrated associations with community functional status.
Because there were insufficient data to establish validity on the measures suggested by the task force on outcomes, potential co-primary measures were added to MATRICS-PASS to evaluate their psychometric properties. Specifically, two measures of functional capacity were added: Maryland Assessment of Social Competence (MASC, developed by A. Bellack and colleagues), and the UCSD Performance-Based Skills Assessment (UPSA, developed by T. Patterson and colleagues). The MASC assesses social competence and the UPSA assesses the capacity to carry out daily activities. Appendix G contains a description of functional capacity measures. Similarly, two interview-based measures were added to MATRICS-PASS: the Schizophrenia Cognition Rating Scale (SCoRS, developed by R. Keefe and colleagues) and the Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS, developed by R. Bilder and colleagues). In MATRICS-PASS, the test-retest reliability for each measure was evaluated at a 4- week interval. To evaluate validity, summary scores from each potential co-primary measure were correlated with cognitive performance based on the MATRICS Consensus Cognitive Battery, and with self-reported community functional status, based on an interview consisting of items from the Social Functioning Scale (developed by M. Birchwood and colleagues) and the Social Adjustment Scale (developed by M. Weissman and colleagues).
Summary data from the potential co-primary measures are presented in Appendix K. In a review of the data that included the members of the MATRICS Neurocognition Committee and the developers of the measures, there was agreement on several points: First, all four measures had adequate test-retest reliability. Second, most tests had adequate range, but a slight ceiling effect and practice effect was noted for one of the one functional capacity measure (UPSA) for subset of patients. Third, all measures had modest relationships to functional status that were somewhat lower than expected. Fourth, the relationships to cognitive performance were notably higher for functional capacity measures than for the interview-based measures of cognition. Fifth, missing data were more frequently observed for the interview-based measures due to absence of caregivers who could serve as informants.
Given these data, the MATRICS Neurocognition Committee concluded that all of the potential co-primary measures performed reasonably well. Although the Committee remained enthusiastic about these two general approaches to co-primary measures, it was reluctant to strongly recommend any one potential co-primary measure over the others. The view was that the measurement in these two areas is still relatively early in development and that further refinement of such measures is expected in the future. Thus, while all tested co-primary measures were considered acceptable, it was judged premature to endorse a single universal standard measure, unlike the situation for the cognitive performance battery.
An important goal of the MATRICS initiative was to develop a consensus regarding the design of clinical trials that would demonstrate the effectiveness of a cognition-enhancing drug for schizophrenia. To meet this goal, a steering committee comprised of NIMH, FDA, and MATRICS scientists selected experts in these areas: cognitive impairments in schizophrenia, neurocognition, neuropharmacology, clinical trial methodology, and biostatistics. The experts were asked to address specific questions relating to clinical trial design of adjunctive/co-treatment and broad spectrum agents. Questions were grouped under three topic areas: 1) inclusion criteria; 2) outcome measures; and 3) other design and statistical issues. After initial discussions, an FDA/NIMH consensus meeting chaired by Robert W. Buchanan was convened at NIMH. At the meeting, experts reviewed relevant evidence before offering the discussion panel their proposed guidelines for a given subset of questions. The discussion panel, which consisted of the expert presenters, steering committee members, and representatives from FDA, academia, and industry, deliberated to reach consensus on suggested guidelines. When evidence was insufficient, suggested guidelines represent the opinion of a diverse cross-section of experts. The recommendations are summarized in the following section. Because the recommendations for outcome measures are discussed in the previous two sections, this section will present only the recommendations for inclusion criteria and other design and statistical issues. A more detailed discussion of the background was recently published in Schizophrenia Bulletin and is included in Appendix A.
Question 1: DIAGNOSIS. Among the major psychotic disorders, what is the evidence that schizophrenia and related disorders are characterized by a unique pattern of cognitive impairments?
Suggested Guideline: Schizophrenia and schizoaffective disorder share a similar pattern of cognitive impairments, which is distinct from patterns in major depression, bipolar disorder, and Alzheimer's dementia. However, because the current approach of the FDA to approval/labeling reflects greater diagnostic specificity of the targeted population, studies of potential cognitive-enhancers should initially include only patients with schizophrenia.
Question 2: CLINICAL STATE AND SYMPTOM SEVERITY. What design approaches should be used to isolate change in neurocognitive domains from changes in other symptom domains?
Suggested Guideline: To isolate change in cognitive function from change in symptoms and other clinical features, include subjects who: a) have been clinically stable and in the residual (non-acute) phase of their illness for a specified period of time (e.g., 8-12 weeks); b) have been maintained on current antipsychotic and other concomitant psychotropic medications for a specified period of time sufficient to minimize potential complications of assessment of cognitive status (e.g., 6-8 weeks) and on current dose for a specified time period (e.g., 2-4 weeks); c) have no more than a "moderate" severity rating on hallucinations and delusions (e.g., Brief Psychiatric Rating Scale (BPRS) Hallucinatory Behavior or Unusual Thought Content item score ? 4); d) have no more than a "moderate" severity rating on positive formal thought disorder (e.g., BPRS Conceptual Disorganization item score ? 4); e) have no more than "moderate" severity rating on negative symptoms (e.g., all Scale for the Assessment of Negative Symptoms global items ? 3 or Positive and Negative Syndrome Scale - negative syndrome total score ? 15); and e) have a minimal level of extrapyramidal symptoms (e.g., Simpson-Angus Scale total score ? 6) and depressive symptoms (e.g., Calgary Depression Scale total score ? 10). These suggested guidelines apply to studies of either adjunctive/co-treatment or broad spectrum agents.
Question 3: ANTIPSYCHOTIC MEDICATIONS. Which antipsychotic(s) should be allowed in studies of an adjunctive/co-treatment agent?
Suggested Guideline: Select an antipsychotic(s) depending on the adjunctive/co-treatment agent and the stage of study. In stage I, avoid pharmacodynamic or pharmacokinetic interactions between the adjunctive/co-treatment agent and the antipsychotic. In stage II, evaluate the impact of potential pharmacodynamic or pharmacokinetic interactions with a larger, stratified sample, in an all-comers design, with few if any restrictions on allowed antipsychotics.
Question 4: POLYPHARMACY. Should antipsychotic polypharmacy be allowed in studies of an adjunctive/co-treatment agent?
Suggested Guideline: Exclude subjects who are taking more than one antipsychotic. Antipsychotic combinations are an unnecessary complication in studying an adjunctive/co-treatment agent.
Question 5: CONCOMITANT MEDICATIONS. Should concomitant medications be allowed?
Suggested Guideline: In stage I, avoid pharmacokinetic and pharmacodynamic interactions between the adjunctive/co-treatment agent and any concomitant medications (e.g., SSRIs). In stage II, examine potential pharmacokinetic and pharmacodynamic interactions on an agent-specific basis.
Question 6: MAXIMUM LEVEL OF IMPAIRMENT. In order to detect a therapeutic effect, should a maximum level of cognitive impairment be specified?
Suggested Guideline: Exclude patients from a trial only if their cognitive impairment severity compromises the validity of the cognitive outcome measures. This determination may be made using a combination of clinical judgment and objective data, such as premorbid IQ scores and premorbid IQ estimates from current reading level.
Question 7: MINIMUM LEVEL OF IMPAIRMENT. In order to detect a therapeutic effect, should a minimum level of cognitive impairment be specified in the inclusion criteria?
Suggested Guideline: Exclude subjects from a trial if their level of cognitive functioning is so high that they perform at or near ceiling and therefore cannot demonstrate improvement. With a properly constructed cognitive test battery, this level of performance will be very rare.
Question 8: SCREENING ASSESSMENTS. If patients are screened for inclusion based upon their level of cognitive impairment, how should screening assessments be conducted?
Suggested Guideline: If a screening assessment must be used, then use an assessment that is different from the measure used to assess cognitive outcome during the trial. The primary exception would be if a multiple baseline strategy is employed.
Question 9: CHOICE OF PRIMARY MEASURE. What are issues regarding the choice of primary cognitive outcome measure(s)?
Suggested Guideline: Pre-specify a single reliable and valid primary cognitive outcome measure, either global or domain-specific, based on its psychometric properties and results of pilot studies.
Question 10: TESTING OCCASIONS. What are the issues regarding the number of testing occasions?
Suggested Guideline: Use more rather than fewer testing occasions to reduce the impact of attrition and to capture change in symptom severity over time. Use data analytic procedures that incorporate repeated measures of the outcome.
Question 11: HETEROGENEITY OF SEVERITY AND RESPONSE. In light of the expected heterogeneity of severity and response within and across different cognitive domains, what approaches to design and analysis should be used to detect a therapeutic effect?
Suggested Guideline: In order to reduce baseline within-group heterogeneity and to increase the chance of detecting a therapeutic effect, include subjects in the residual (non-acute) phase of their illness and use one primary efficacy measure.
Question 12: CONCURRENT CHANGE IN SYMPTOMS. What approaches to design and analysis should be used to control for potentially concurrent changes in other symptom domains?
Suggested Guideline: Statistical approaches cannot be used to rule out pseudospecificity (i.e., an artificially narrow claim of cognitive enhancement that could result from post-baseline confounding, such as reduction in other aspects of the illness). Pseudospecificity is best dealt with by restricting symptom severity prior to randomization.
Question 13: COMPARISON GROUP. What are the appropriate comparator agents for adjunctive/co-treatment agents and broad spectrum agents?
Suggested Guideline: To study an adjunctive/co-treatment agent, use placebo as the comparator. The choice of comparator for a broad spectrum agent poses a more substantial challenge, but should be, at worst, cognitively neutral.Question 14: TRIAL DURATION. What is the optimal clinical trial duration of an agent targeting cognition in schizophrenia? Suggested Guideline: The trial needs to be of sufficient duration to show an enduring effect on cognition (i.e., at least 6 months). Longer duration studies should use multiple testing occasions, which require the existence of parallel forms of the outcome measure. Statistical procedures should be used that incorporate data from multiple assessment times. ______________________________ The recommendations in this report represent the work of individuals from industry, academia, as well as government. We are particularly grateful to Thomas Laughren and Robert Levin from FDA who provided substantial guidance to the group. All of the participants share the hope that this initiative will result in the development of more effective treatments that will improve the lives of individuals with schizophrenia.
A) Buchanan, R.W., Davis, M., Goff, D., Green, M.F., Keefe, R.S.E, Leon, A.C., Nuechterlein, K.H., Laughren, T., Levin, R., Stover, E., Fenton, W., Marder, S.R. (2005) A Summary of the FDA-NIMH-MATRICS Workshop on Clinical Trial Design for Neurocognitive Drugs for Schizophrenia. Schizophrenia Bulletin, 31, 5-19.
B) Gold, J.M. (2004). Cognitive deficits as treatment targets in schizophrenia. Schizophrenia Research, 72, 21-28.
C) Green, M.F., Kern, R.S., & Heaton, R.K. (2004). Longitudinal studies of cognition and functional outcome in schizophrenia: Implications for MATRICS. Schizophrenia Research, 72, 41-51.
D) Green, M.F., Nuechterlein, K.H., Gold, J.M., Barch, D., Cohen, J., Essock, S., Fenton, W.S., Frese, F., Goldberg, T.E., Heaton, R.K., Keefe, R.S.E., Kern, R.S., Kraemer, H., Stover, E., Weinberger, D.R., Zalcman, S., & Marder, S.R. (2004). Approaching a consensus cognitive battery for clinical trials in schizophrenia: The NIMH-MATRICS conference to select cognitive domains and test criteria. Biological Psychiatry, 56, 301-307.
E) Kern, R.S., Green, M.F., Nuechterlein, K.H., & Deng, B.H. (2004). NIMH-MATRICS survey on assessment of neurocognition in schizophrenia. Schizophrenia Research, 72, 11-19.
F) Marder, S.R., Fenton, W. (2004). Measurement and treatment research to improve cognition in schizophrenia: NIMH MATRICS Initiative to support the development of agents for improving cognition in schizophrenia. Schizophrenia Research, 72, 5-10.
G) McKibbin, C.L., Brekke, J.S., Sires, D., Jeste, D.V., & Patterson, T.L. (2004). Direct assessment of functional abilities: Relevance to persons with schizophrenia. Schizophrenia Research, 72, 53-67.
H) Nuechterlein, K.H., Barch, D.M., Gold, J.M., Goldberg, T.E., Green, M.F., & Heaton, R.K. (2004). Identification of separable cognitive factors in schizophrenia. Schizophrenia Research, 72, 29-39.
I) Data Table of Ratings for Tests from the RAND Panel
J) Data Tables on Psychometrics of Cognitive Measures from MATRICS-PASS Phase I
K) Data Tables on Psychometrics of Potential Co-primary Measures from MATRICS-PASS Phase I
L) Membership Listings for the Key MATRICS Committees
Contact email@example.com for questions or comments about the MATRICS program and see our website - www.matricsinc.org - for details regarding the MATRICS Consensus Cognitive Battery (MCCB).
Contact firstname.lastname@example.org for questions or comments about with this web site.