Assessing the Readability of Capital Pattern Jury Instructions

Data obtained from the National Assessment of Adult Literacy (2003) indicate that 29% of adults have a Basic level of prose literacy. Prose literacy refers to the knowledge and skills needed to comprehend and use information from continuous texts, such as editorials, news stories, and instructional materials. In the current study we examined each state’s capital sentencing instructions for their readability using a highly reliable word-analysis tool. We found that reading levels of sentencing instructions significantly surpassed the reading comprehension abilities of American adults. All but three states’ instructions were above the twelfth grade reading level. In addition, legal principles embedded within the weighing language instruction were more difficult to comprehend compared to the aggravating or mitigating factors components of the instructions. By measuring the readability of instruction language and comparing it national literacy levels of adults, we provide an important explanation for poor comprehension of legal principles.

Overview

In the Gregg v. Georgia (1976) decision, the Court envisioned jury instructions as both a legal safeguard and a discriminating factor in determining the fairness of sentencing proceedings. With this in mind, instructions are intended to define the jury’s role and communicate applicable laws as well as other legally relevant principles (Lieberman & Sales, 2000, 1997). Pattern instructions in particular are designed to streamline the selection of appropriate case-specific jury instructions and diminish the tenuousness of their application. The primary goals of pattern jury instructions are to increase the legal accuracy of instructions, eliminate argumentative language, and improve comprehensibility (Chilton & Henley, 1996). However, some authors have observed that instructions almost always create confusion rather than comprehension, contributing very little in terms of juror understanding of legal concepts (Strawn & Buchanan, 1976). As a result, researchers have identified remedies for misunderstanding, such as refining instructions to address linguistic and comprehensibility issues (Charrow & Charrow, 1979; Elwork, Sales, & Elfini, 1977). The current study continues the research addressing instruction comprehension by examining the readability of capital pattern jury instructions using a highly reliable word analysis tool (Flesch, 1950). Our goal was to assess the reading comprehension and level of difficulty of each state’s pattern sentencing instructions and compare it to national levels of prose literacy in American adults.

Instruction Comprehension

Given that the question of life or death is an ultimate one, juries entrusted with capital sentencing deserve clear and comprehensive guidance. If jurors do not understand legal instructions, they may not be able to apply the law. To address this issue, Strawn and Buchanan (1976) tested the effectiveness of Florida pattern instructions for comprehension and applicability. They found that participants provided with jury instructions prior to completing a true/false test based on the law, incorrectly responded to nearly one-third of the test questions. This result was only marginally better than the group given no instructions prior to taking the test. In addition, only 50% of the instructed group was able to demonstrate a correct understanding of the presumption of innocence, a legal concept stating that a defendant is not required to produce any evidence of innocence. As such, a critical objective in improving the jury instructions should be to preserve their legal accuracy without sacrificing comprehensibility.

Juror ability to comprehend and correctly apply the law has also been examined with a focus on the linguistic composition of jury instructions. For example in one study, Charrow and Charrow (1979) rephrased California’s civil jury instructions eliminating some of the legal language and complex sentence structures. Juror comprehension was significantly improved with the revised instructions compared to the original instructions. Elwork, Sales, and Alfini (1977) conducted a similar study in which Michigan’s instructions were rewritten with basic linguistic rules in mind. Comprehension was significantly improved for participants who received the rewritten instructions compared to the original Michigan pattern instructions. Additionally, Severance and Loftus (1982) provided evidence demonstrating that knowledge of linguistics could be applied to developing a comprehensible set of instructions. In a series of studies, these researchers were able to identify and correct ‘problem’ areas in pattern jury instructions, leading to increased understanding of instructions. This is not always the case however, as demonstrated by Weiner, Pritchard, and Weston (1995). When testing juror comprehension of both the original and revised set of Missouri’s capital jury instructions, the researchers found that revisions made little improvement on jurors’ understanding. Furthermore, miscomprehension was found to be strongly related to willingness to impose the death penalty.

Finally, studies addressing the deliberation process find that juries often reach improper verdicts when a misunderstanding of the judge’s instructions misguides the group’s discussion of legally relevant principles. For example, in one study jurors did not exhibit increased comprehension of instruction language during the deliberation process (Ellsworth, 1989). Similarly in a subsequent study, researchers found improvements in instruction comprehension during deliberations, but only when a significant majority of jurors entered the deliberation process with a correct understanding of legal principles (Diamond & Levi, 1996). These findings highlight the notion that a jury verdict decided on the basis of a misunderstanding of legal principles greatly increases the likelihood of rendering a verdict that is incompatible with the law.

Juror Literacy

Considering the results of social science research, which consistently demonstrates low levels of instruction comprehension, it has become increasingly important for researchers in this area to understand the literacy skills of jury-eligible adults. Literacy is defined as “using printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential” (National Assessment of Adult Literacy; NAAL, 2003, Section 1.2, p.1-2). Literacy types are identified as Prose, Document, and Quantitative Literacy. Literacy levels are rated according to the following performance levels: Below basic, no more than the most simple literacy skills; Basic, the skills necessary to perform everyday reading tasks; Intermediate, the skills required to perform moderately challenging tasks; and, Proficient, the skills needed for complex reading tasks. Prose literacy, which measures the skills needed to understand and use information from continuous texts, is the form most applicable to jurors’ abilities to comprehend and apply sentencing instructions. On average, prose literacy level of adults is identified as Basic – possessing the skills necessary to perform everyday reading tasks. Specifically, NAAL’s (2003) survey found that 29% of adults possess a basic level of prose literacy. Additionally, adults over the age of 65 were found to be more likely to receive a below basic score on the prose literacy tasks compared to other age groups. Based on this finding, it is likely that below basic levels of prose literacy are present in a substantial portion of venire persons retained for jury service.

Similar comparisons have been drawn between NAAL’s data on literacy levels and comprehension with different populations. Particularly, Rogers, Harrison, Shuman, Sewell, and Hazelwood (2007) examined NAAL’s literacy data andMiranda warning comprehension of an incarcerated population (Miranda v. Arizona, 1966). After obtaining hundreds of Miranda warnings from jurisdictions across the United States, the researchers designed five components in order to organize content and assess comprehension levels between states and among the five organized components. This methodology provided a means for identifying particular strengths and weaknesses of the Miranda warning material. Similar to Rogers (2007) we have identified three components within the pattern capital instructions that appear fundamental to a complete and clear instruction. These components are aggravating factors, mitigating factors, and deliberative or weighing language. For our analysis, only material that is purely descriptive of aggravating and mitigating factors has been placed in those categories. Any language describing burdens of proof, and any other legal principles or mechanisms of considering aggravation and mitigation have been placed in the third category, weighing language.

The current study advances the research on readability of capital jury instructions in several ways. First, we apply well-established reading comprehension tools (Flesch-Kincaid reading level scores) to establish reading levels associated with instructions. Second, rather than sampling an individual set of instructions (see Elwork, et al., 1997; Weiner, et al, 1995) we attempted to exhaustively sample capital jury instructions in the U.S. Third, we compare our results with data on literacy levels from U.S. citizens.

Method

Sources of Pattern Capital Jury Instructions

Web-based versions of each state’s pattern jury instruction for the sentencing phase of a capital trial were obtained for 32 of the 33 states that currently allow the death penalty. Both general search engines and academic databases were utilized to search for these pattern instructions. In cases in which a direct listing of aggravating or mitigating factors was not contained within the instruction, state government web sites were used to locate this information.

Instrument

We used the Flesch-Kincaid reading ease test to assess the readability of capital pattern jury instructions. Developed by Rudolph Flesch (1950), the Flesch Reading Ease Formula is considered one of the oldest and most accurate readability formulas. The Flesch-Kincaid is a highly regarded tool (Dubay, 2004) that is widely used by many U.S. Government agencies (Berndt, Schwartz, & Kaiser, 1983). The formula combines sentence length with the average number of syllables per word to produce an estimated grade level necessary to comprehend a written passage. Flesch-Kincaid scores are strongly correlated with standardized reading tests. This word analysis tool operates within Microsoft Word. Scores range from 0 to 100, higher scores indicating easier text. Flesch-Kincaid scores of 0 – 59 are considered difficult to very difficult; scores of 60 – 100 are considered standard to easy. In addition to the reading ease index, Flesch-Kincaid computes a grade level of selected text, with a cut-off of twelfth grade.

Procedure

After collecting pattern jury instructions for this analysis, we reviewed and formatted each state’s instruction to fit a general template. This ensured uniformity of the pattern instructions, making it easier to discern the precise content included or missing from each set of instructions. We deleted any irrelevant content that did not instruct the jury, such as legal notes meant to inform trial judges. We then coded instruction language in accordance with our three working components: aggravating factors, mitigating factors, and weighing language. In the process of standardizing and coding the language for the purpose of the Flesch-Kincaid reading ease analysis, it became clear that some variations in the instructions were statutory. To retain internal and ecological validity, we did not change substantive or procedural differences between each state’s pattern instructions. An overall Flesch-Kincaid reading ease score and grade level were computed for each state’s capital pattern instruction. Readability index scores were then drawn individually for a componential analysis of each instruction, allowing us to accurately compare the readability of three domains within each instruction.

Results

Table 1 displays the distribution of readability scores for 32 states.

Table 1. Readability Scores as a Function of Component

  As can be seen in Table 1, all instructions would be categorized as difficult to very difficult according to Flesch-Kincaid scoring guidelines. An analysis of Flesch-Kincaid grade levels of the pattern jury instructions indicated that only 3 states’ capital pattern jury instructions were below the twelfth grade reading level: California, Kentucky, and Pennsylvania. Approximately 25% of the states yielded very difficult Flesch-Kincaid scores (0-30), the remaining 75% had scores indicating difficult comprehension levels (30-59). The Flesch-Kincaid readability tool also provides the frequency of passive sentences within a text as a percentage. The percentage of passive sentences within the pattern instructions ranged from 4 to 40%, M = 22%. We also conducted a series of analyses to examine whether readability differed by federal circuit. Mean scores on the instructions as a whole as well as the individual components did not differ significantly when we compared the 11 circuits or grouped these circuits into 5 by proximity. Table 2 displays this finding. We also grouped states by frequency of execution, but these analyses also did not reveal significant differences in instruction readability.

Table 2. Regional Distribution of States with the Death Penalty, N=33

  The following examples represent two of the more disparate ways in which jurors are instructed on how to consider mitigating factors.

Mitigating or extenuating facts or circumstances are those that you, the jury, find do not constitute a justification or excuse for the offense in question but that, in fairness and mercy, may be considered as extenuating or reducing the degree of moral culpability or blame.

A mitigating factor is any fact or circumstance, relating to the crime or to the defendant’s state of mind or condition at the time of the crime, or to [his/her] character, background or record, that tends to suggest that a sentence other than death should be imposed.

Weighing language was found to be significantly more difficult to read compared to language used to describe aggravating factors. In addition, this language had the lowest readability score (most difficult) compared to the other types of instructional language. The following examples demonstrate differences in weighing language:

If you unanimously find that one or more of the aggravating circumstances existed beyond a reasonable doubt, the death penalty shall not be imposed unless you also unanimously find that any such aggravating circumstance or circumstances outweigh the finding of one or more mitigating circumstances. Even if you find that the aggravating circumstance(s) outweigh(s) the mitigating circumstance(s), you may impose a sentence of imprisonment for life with the possibility of parole or imprisonment for life without the possibility of parole.

The specific findings as to any particular aggravating circumstance must be unanimous. All of you must agree that the Commonwealth has proven it beyond a reasonable doubt. That is not true for any mitigating circumstance. This different treatment of aggravating and mitigating circumstances is one of the law’s safeguards against unjust death sentences. It gives the defendant the full benefit of any mitigating circumstances.

Conclusion

Overall, our study offers an important explanation for poor comprehension of legal principles. Simply put, the reading levels of instructions are frequently at or above the twelfth grade, a result that is inconsistent with the average reading level of the American adult. Considering that less than fifty percent of adults possess the basic skills and knowledge necessary to read and comprehend moderately difficult reading passages, it’s not likely they are able to synthesize the complex language present in jury instructions. Our findings suggest that instructions should be reworded or reconfigured to match the literacy levels of American jurors.

Our study also highlights the challenge facing committees tasked with rewriting instructions. According to a comprehensive Impact Study (Dubay, 2004), individuals with basic and below basic levels of literacy represent the most significant problem as users of technical documents. The more technical the information, the greater the need for increased readability. For those who think that a measure of readability does not capture how well one listens, our results actually provide an important framework for understanding the relation between listening and reading comprehension in explaining comprehension difficulty of jury instructions. According to research, listening and reading comprehension involve similar syntactic and semantic processes, both of which are central to comprehension (Hausfeld, 1981). In fact, listening comprehension of longer passages surpasses reading comprehension until sixth grade, when reading becomes superior to listening (Durrell, 1969). As a result, it’s likely that jurors would experience greater difficulty with listening comprehension compared to reading comprehension.

The difficulty jurors experience with the technical language of instructions presents a formidable challenge for attorneys and consultants when preparing for trial. To address this challenge, consider utilizing instructions as a ‘road map’ when preparing trial strategy. Well-prepared instructions help to frame the critical case issues. With this in mind, it’s important to familiarize the jury with instructional language with each advantageous chance – for example, during voir dire and opening and closing arguments. Effective and innovative uses of instructions will lead to a more focused presentation to the jury and a more thoughtful deliberation process.

---

Rachel Small is a 2011 graduate of Roger Williams University M.A. Program in Forensic Psychology. She is currently a Ph.D. student in legal psychology at The University of Texas at El Paso. Her research interests address social justice issues in the context of due process. Professionally, she is interested in pursuing a career in trial consulting after graduation.

Judith Platania is an Associate Professor of Psychology at Roger Williams University. Her current research program investigates the role of non-statutory mitigating circumstances in capital sentencing decisions. The results of her most recent empirical investigations were presented at the 2012 meeting of the American Psychology-Law Society in San Juan, PR. She teaches undergraduate and graduate research methods and statistics. www.judithplatania.com

Brian Cutler is Professor of the Faculty of Social Science and Humanities at the University of Ontario Institute of Technology. He is past editor of Law and Human Behavior and past President of the American Psychology-Law Society (Division 41, American Psychological Association).

---

References

Berndt, D. J., Schwartz, S., & Kaiser, C. F. (1983). Readability of self-report depression inventories. Journal of Consulting and Clinical Psychology, 51,627-628.

Charrow, R., & Charrow, V. (1979). Making legal language understandable: A psycholinguistic study of jury instructions, Columbia Law Review 79,1306-1320.

Chilton, E., & Henley, P. (1996). Improving the jury system, jury instructions: Helping jurors understand the evidence and the law, University of California, Hastings College of the Law, Public Law Research Institute.

Diamond, S. S., & Levi, J. N. (1996). Improving decisions on death by revising and testing jury instructions. Judicature, 79, 224–232. DuBay, W. H. (2004). The principles of readability.

Durrell, D. D. (1969). Listening comprehension versus reading comprehension. Journal of Reading, 12,455-460.

Ellsworth, P. (1989). Are twelve heads better than one? Law & Contemporary Problems, 52, 205–224.

Elwork, A., Sales, B. D., & Alfini, J. (1982). Making jury instructions understandable. Charlottesville, VA: Michie.

Ellsworth, P. (1989). Are twelve heads better than one? Law & Contemporary Problems, 52, 205–224.

Flesch, R. (1950). Measuring the level of abstraction. Journal of Applied Psychology, 34, 384 390.

Gregg v. Georgia, 428 U.S. 253 (1976).

Hausfeld, S. (1981). Speeded reading and listening comprehension for easy and difficult materials. Journal of Educational Psychology, 73,312-319.

Lieberman, J., & Sales, B. (2000). Jury instructions: Past, present, and future. Psychology, Public Policy, and Law, 6, 587 – 590.

Lieberman, J., & Sales, B. (1997). What social science teaches us about the jury instruction process. Psychology, Public Policy, and Law, 3, 589-644.

Miranda v. Arizona, 384 U.S. 436 (1966).

Rogers, R., Harrison, K., Shuman, D., Sewell, K., & Hazelwood, L. (2007). An analysis of Miranda warnings and waivers: Comprehension and coverage. Law and Human Behavior, 31, 177 – 192.

Severance, L., & Loftus, E. (1982). Improving the ability of jurors to comprehend and apply criminal jury instructions. Law and Society Review, 17, 153-197.

Strawn, D., & Buchanan, R. (1976). Jury confusion: A threat to justice. Judicature, 59, 478 – 483. U.S. Department of Education. Washington, DC: National Center for Education Statistics. Literacy in everyday life: Results from the 2003 National Assessment of Adult Literacy (NCES 2007–480).

Wiener, R., Pritchard, C., Weston, M. (1995). Comprehensibility of approved jury instructions in capital murder cases. Journal of Applied Psychology, 80, 455 -467.