The “Improved” DSMIV? 

There is no available evidence indicating that the procedures of DSMIV have reduced the subjective biases associated with the diagnostic work of mental health professionals.  Diagnoses too often reflect the ethnic and social class prejudices of diagnosticians and the social stereotypes they associate with a particular disorder.i At PsychLaw.net we take for example, black patients are diagnosed as schizophrenic far more frequently than white patients.ii  This outcome corresponds more to the unfamiliarity of white diagnosticians with black culture.  Diagnostic labels also involve simple considerations of whether diagnosticians experience an affinity for their patients.  Patients who direct negative attitudes toward diagnosticians are labeled more seriously disturbed than patients who express deferential attitudes.iii  

The concept of “inter-rater reliability” was described in previous posts. The critically important question of interrater reliability asks: If two or more mental health professionals evaluate the same client, to what extent will they agree in their diagnostic conclusions?  Low levels of interrater reliability related to any classification procedure indicates that the procedure frequently leads to mistaken findings.   Surprising as it may seem, there are no interrater reliability data to be found in DSMIV.  DSMIII contained interrater reliability data for its various diagnostic categories, but the supposedly improved DSMIV neglected to report this information. The unavailability of interrater reliability data for the many diagnostic classifications of DSMIV profoundly undermines its evidentiary value. 

The 1999 Standards for Educational and Psychological Testingiv published by the American Psychological Association, clearly define the necessity of reporting interrater reliability data for any procedure relying on clinical opinion.  In particular, Standard 2.10 of these Standards states: 

“When subjective judgment enters into test scoring, evidence should be provided on both interrater consistency in scoring and withinexaminee consistency over repeated measurements.”v 

 

At PsychLaw.net we note that the use of DSMIV relies almost entirely on judgmental processes.  Neglecting to report interrater reliability data for its diagnostic categories therefore amounts to an egregious error.  Though DSMIV is not a psychological test per se, Standard 2.10 is nonetheless applicable to it.  The 1999 Standards specifically state:  

“The applicability of the Standards to an evaluation device or method is not altered by the label applied to it (e.g., test, assessment, scale, inventory).”vi 

 

DSMIV is an assessment technique designed explicitly for the assessment and classification of psycho-pathology. 

Despite the overwhelming importance of interrater reliability to diagnostic classification, most mental health professionals neglect to think about this issue carefully. At PsychLaw.net we consider, for example, how a doctorallevel psychologist responded to the following crossexamination. 

Attny: Do you know the research concerning the reliability of diagnoses? 

Psych: I don’t know what research you’re referring to. 

Attny: I’m referring to the scientific literature concerning the accuracy of diagnoses. 

Psych: I don’t know what you’re referring to, but if you could tell me, I’d be interested. 

Attny: Do you know what interrater reliability is? 

Psych: Yes, I do. 

Attny: Are you familiar with the research on the interrater reliability of DSMIV diagnoses? 

Psych: I am not. 

 

From the time of its first edition published in 1952, through its most recent 1994 fourth edition, each DSM has progressively increased the number of diagnosable disorders.vii The 1952 edition specified 60 categories of mental disorder.  The second edition, published in 1968, contained 145 diagnoses.  DSMIII, published in 1980, expanded to 230 disorders.  DSMIIIR, published in 1980, listed more than 300 diagnostic categories.  Most recently, the 1994 DSMIV has grown to more than 400 disorders.  Rather than lead to improved levels of interrater reliability, this increasing number of diagnosable disorders can only reduce the extent of diagnostic agreement between clinicians.  Increasing the number of category choices inevitably reduces the level of classification agreement between two or more raters. 

In addition to its many diagnostic classifications, DSMIV also uses a “Global Assessment of Functioning Scale” (GAF) to assess how effectively clients function in their daytoday lives.  Curiously, research data demonstrate no significant relationship between patient ratings of their own psychological symptoms, and clinical assessments of those patients using the GAF scale.viii  In other words, this 1995 study by Piersma and Boes, published in the Journal of Clinical Psychology,  demonstrated that GAF ratings are unrelated to how patients assess themselves. We at PsychLaw.net feel that quite obviously, this research sorely undermines the evidentiary value of GAF as a method for assessing the adequacy of a patient’s general functioning. 

Psychologists and laypeople alike typically assume that diagnosis is a necessary prerequisite to treatment.  Nevertheless, the relevant data fail to support this assumption.  In particular, two critics of the various DSM editions have sharply criticized its role in graduate training: 

“As clinicians, students may be given misinformation in graduate school.  They will be taught DSMIIIR or the fourth edition of the DSM as if it were science and not politics; but such diagnoses as are found in these texts predict only 5% to 10% of the outcome of psychotherapy.”ix (p. 216). 

 

CrossExamining The “Improved” DSMIV 

1. Diagnoses too often reflect the ethnic and social class prejudices ofdiagnosticians  Correct? 
2. BlackAmericansare diagnosed as schizophrenic more often than WhiteAmericans  Correct? 

3.But this outcome corresponds more to the unfamiliarity of whiteAmerican diagnosticians with blackAmerican culture  Correct? 

4. Diagnostic labels can also involve considerations of whether diagnosticians experience an affinity for theirpatients  Correct? 
5. The issue of interrater reliability is also critically important to any procedure for diagnosticclassification  Correct? 
6. The critically important issue of interrater reliability asks: 

– [read] – 

“If two or more psychologists evaluate the same client, to what extent will they agree in their diagnostic conclusions”  Correct? 

7. Low levels of interrater reliability related to any procedure for diagnostic classification indicates that the procedure frequently leads to mistakenfindings  Correct? 
8. DSMIII, published in 1980, contained interrater reliability data for itsdiagnostic categories  Correct? 
9. But the supposedly improved DSMIV neglects to report any interrater reliability for its diagnosticcategories  Correct? 
10. Therefore, we do not know how often DSMIV leads to mistakenconclusions  Correct? 
11. In other words, we do not know what the error rate is for any DSMIV diagnosticcategory  Correct? 
12. And mental health professionals cannot claim that DSMIV is generallyaccepted  Correct? 
13. Claiming that DSMIV is generally accepted necessitates that mental health professionals agree in their diagnostic conclusions when usingit  Correct? 
14. But, we do not have interrater reliability data for the diagnostic categories ofDSMIV  Correct? 
15. Mental health professionals cannot claim that DSMIV is generally accepted without knowing the extent to which theyagree  or disagree  in their diagnostic conclusions premised on DSMIV  Correct? 
16. The 1999Standards for Educational and Psychological Testing were published as a cooperative undertaking between the American Psychological Association, the American Educational Research Association, and the National Council on Measurement in Education  Correct? 
17. The 1999Standards for Educational and Psychological Testing define standards for a broad range of assessment procedures  Correct? 
18. And the 1999Standards for Educational and Psychological Testing are generally recognized and accepted as defining appropriate standards of practice regarding a broad range of assessment procedures  Correct? 
19. The 1999Standards for Educational and Psychological Testing indicate they can be applied to the following: 

– [ read ] – 

    “The applicability of the Standards to an evaluation device or method is not altered by the label applied to it (e.g., test, assessment, scale, inventory).” 

Now my question:  DSMIV is explicitly designed as Aa evaluation device” for the assessment and classification of psychopathology  Correct? 

20.Please consider what Standard 2.10 of the Standards for Educational and Psychological Testing state regarding interrater reliability: 

– [ read ] – 

“When subjective judgment enters into test scoring, evidence should be provided on both interrater consistency in scoring and withinexaminee consistency over repeated measurements.” 

Now my question:  The Standards for Educational and Psychological Testing therefore clearly define the necessity for interrater reliability data  Correct? 

21.DSMIV’s failure to report interrater reliability data consequently amounts to neglect of Standard 2.10  Correct? 

22.DSMIV also uses a “Global Assessment of Functioning Scale” (GAF)  Correct? 

23.And the GAF scale relies in part on what the patient tells the mental health professional  Correct? 

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS 

IF YOU HAVE NOT YET DONE SO] 

24.The Journal of Clinical Psychology is a generally recognized and accepted, peerreviewed journal in your field  Correct? 

25. And a 1995 article byPiersma and Boes published in the Journal of Clinical Psychology  titled “Agreement Between Patient SelfReport and Clinician Rating: Concurrence Between the BSI and GAF Among Psychiatric Patient”   might be related to your opinions in this case  Correct? 
26. And in their 1995 study,Piersma and Boes found no significant relationship between patient ratings of their own psychological symptoms and clinical assessments of those patients using the GAF  Correct? 
27. Have you published any data in apeerreviewed journal necessitating that we reconsider Piersma and Boes’ 1995 study? 
28. Can you cite any data published in apeerreviewed journal necessitating that we reconsider Piersma and Boes’ 1995 study? 

29.Without any data necessitating reconsideration of Piersma and Boes’ 1995 study, their results  published in a peerreviewed journal  should be generally recognized and accepted by your profession  Correct? 

30. In other words, your profession generally recognizes and accepts that there is no significant relationship between patient ratings of their own psychological symptoms and clinical assessments of those patients using theGAF  Correct?