Fall 2005 - Volume 8 Number 4
Graduate Student Attitudes toward Grading Systems Abstract
This study examined graduate student attitudes towards letter and pass/fail grading systems in the Law School and the School of Education in a selective university in the United States. Fifty-four students completed a questionnaire on goal orientations (ability comparison vs. mastery), amount of effort and stress in each of the two grading environments. Students reported higher orientation towards ability comparison and higher levels of effort and stress in letter-graded classes. Gender, school, and mastery orientation differences were not significant.Introduction
In a dramatic step, professors at the University of California at Santa Cruz voted yesterday to require letter grading for the first time in the campus’ 35-year history. (Schevitz, San Francisco Chronicle, 2000)
The decision by the UC Santa Cruz faculty in 2000 to reinstate letter grades was a controversial one. Many saw the shift to grades as necessary, in order to heighten students’ chances for competitive post-graduate fellowships and high status jobs. In this sense the policy change reflected broader trends in the United States towards “outcomes” and “accountability” in public education. To others the policy change signaled the end of the school’s alternative atmosphere, which emphasized learning and narrative feedback over competition for grades. According to one disappointed UCSC student, “the [pass/fail] system is at the core of a tradition of emphasizing learning over grades” (Schevitz, 2000).
Of particular interest here are the questions of student motivation and learning that the debate touched on. This relationship between learning and grading systems has both theoretical and practical significance: Motivation researchers are interested in how different incentive systems and school environments influence different forms of student motivation. Instructors want to know what the optimal grading systems are that motivate students to learn.
This study examines the influence of grading systems on student attitudes towards learning at the Law School (LS) and the School of Education (SE) in a middle-size, highly selective university. Do different forms of assessment in higher education influence student attitudes towards learning, and how? Does the grading system (letter grade or pass/fail) in a specific class influence student attitudes towards learning? Do attitudes towards learning differ among students in graduate programs that have different grading policies?Theoretical Background
From a theoretical standpoint, the above research questions build on findings reported by Roeser, Midgley, and Urdan (1996) about young adolescents’ achievement motivation. They argue that the achievement norms of a school will have a significant impact on the types of goal orientations of students in that school. They found that students in schools with “relative ability” structures, such as honor rolls and other forms of social comparison, tend to adopt learning goals that reflect this orientation. These students are more likely to evaluate their learning in terms of how they perform relative to their peers. For many students, such social comparisons are detrimental to their motivation to learn. In contrast, students in schools with “task mastery” norms, which emphasize personal improvement and the mastery of knowledge, tend to adopt learning goals that reflect this orientation. They are more likely to accept mistakes as part of the learning process and to focus on individual learning rather than social comparison.
The university context of this study provides an interesting opportunity to extend this particular theoretical framework in a few unexplored directions. Roeser et al. (1996) did not specifically isolate the impact of pass/fail grades in pursuing their study--typically middle school students receive letter grades. This study presents an opportunity to see whether or not the letter grade vs. pass/fail distinction plays a role in determining either a relative ability or task mastery orientation. When a teacher enforces either a mandatory letter grade or pass/fail system, does this convey a message to students that is then reflected in their own goal orientations?
In addition, the population examinedgraduate students in Law and Educationis unique. While students in these programs share a similar record of high achievement and academic performance, the differences in these schools suggest an interesting comparison. In letter graded classes at the LS, students are graded on a “mandatory mean”. This means that the professor is required to give half of the students’ grades below the median, and half the students’ grades above the median, which would likely promote implicit competition among students for the higher grades. Furthermore, we hypothesize that students in the LS are likely to be more competitive because of the way clerkships and other jobs are allottedclass rank and grade point average (GPA), among other factors, figure prominently in such decisions. In contrast, students in Education are not graded on a mandatory mean, nor is their class ranking as critical. GPA and transcripts are usually less significant in determining future employment than the quality and type of research completed by the student. Therefore, these two schools are interesting sites for exploring whether or not the motivations and learning goals of graduate students reflect “relative ability” and “task mastery” orientations, and whether this varies between schools.
Such research should shed also light on the influence of different assessment methods on student motivation. As Schevitz’s article (2000) in The San Francisco Chronicle suggests, the implications of different grading systems are not solely of theoretical interest. Debates about the impact of grades on student motivation have been common since the 1960’s, as universities have tried to decide what system to put in place (Quann, 1984; Weller, 1983). Pass/fail grading was proposed as a non-competitive alternative to the traditional letter grading in the mid-1960s. It was thought of as a way of reducing anxiety and pressure and of encouraging students to explore other disciplines without the fear of lowering their GPA. The pass/fail model was implemented in many institutions in the years that followed and can still be found in many universities, although in most cases it is a limited option and remains less common than letter grading.
Since then debates have continued in the education literature about the relative merits of each system. On one side are those who are strongly in favor of grades because they are purported to be the primary source of motivation (Bell, 1994; Ebel, 1980). Critics of pass/fail systems argue that they weaken student motivation and effort in courses. Also, it has been found that students end up spending more time on their graded courses and de-emphasizing their pass/fail courses (Weller, 1983). On the other side are arguments critical of grades because they offer little information to the student (Sadler, 1983) or because they promote competition and unproductive pressure among students (Stallings, 1970). Educational researchers have also argued that students experiment freely in new subjects, focus more on individual learning than comparison (Gage & Berliner, 1998; Milton et al., 1986) and plan their own learning (Winter, 1993), when pass/fail assessment is in place.
One limitation of the above studies, in terms of their relevance to this study, is that they concentrate on attitudes among undergraduates. Graduate education is structured differently from earlier levels and has other goals, and thus findings are not necessarily generalizable. However, the coursework is generally assessed in similar ways, and the populations, while different in age, are comparable in other respects. A second limitation is that a good deal of the published information on this topic is based more on untested assumptions than empirical evidence. Debates about grading often invoke disparate philosophies about education, and these invariably become part of the discussion. Nevertheless, the debates are useful in highlighting the key variables of interest, such as effort, stress, competition, and intrinsic learning, to name just a few.
We hope to shed light on these issues by researching student attitudes towards learning in classrooms with different grading methods using a questionnaire, through the following research questions: Do students adopt different goal orientations (task mastery vs. relative ability) depending on the grading system of a course? Do students experience different levels of effort and stress depending on the grading system of a course? Do students at a LS and a SE differ in their goal orientations (task mastery vs. relative ability)? Do students at a LS and a SE differ in their levels of effort and stress?
We hypothesized that (1) graduate students are more oriented towards relative ability goals, put more effort, and feel more stress in letter-graded classes, (2) are more oriented towards mastery goals in pass/fail classes, and (3) that LS graduate students are more oriented towards relative ability goals than their SE peers.Method
To investigate the research questions posed, a 29-item questionnaire was constructed. About half of these items were adapted from an instrument utilized by Roeser et al. (1996). In its original form the instrument measures perceptions of “school goal structures” and “personal achievement goals”. Responses were given on a 5-point Likert Scale, where 1 stands for “I strongly disagree” and 5 for “I strongly agree”. The items were slightly changed to be more appropriate for graduate level students, as it was originally constructed for adolescents.
In the first part of the questionnaire, there were 15 items that require two responses each: one pertaining to letter graded and one to pass/fail classes (these items appear on Table 1). The items belonged to one of two dimensions: school goal structures and personal achievement goals. These two dimensions would allow an examination of their interrelationship and whether it varies under different grading systems. School goal structures were defined in terms of their orientation towards either task mastery (SCM) emphasizing improvement, mastery, and intellectual development, or relative ability comparisons (SCA) emphasizing social comparison and relative ability. Personal achievement goals were measured along the same dimensions, (student mastery orientation [STM] and student ability orientation [STA]). The four groups of items are presented in Table 1.
The second part of the questionnaire included items on “effort” and “stress”. These items were phrased comparatively between the two grading systems, e.g. “I worry more about the professor’s evaluation of my work in a pass/fail class than in a letter-graded class”. Students were asked to respond on a Likert Scale.
Additional items were: gender, years in graduate program, number of letter-graded courses taken and number of pass/fail courses taken. The last item was an optional, open-ended question, asking for a brief, general comment on the personal learning experience in the two kinds of classes. The purpose of this open-ended question was to let students comment freely on any of the issues raised by the questionnaire they considered important.
The sample of the survey consisted of graduate students (Master and Ph.D. students) in the SE and the LS at a competitive, middle-size university. One hundred questionnaires were administered in each school and a total of 54 anonymous questionnaires were returned - response rate of 26%. Half of the students were first year graduates, 15 were in their second year, 9 in their third and 3 in their forth (or more). Almost all respondents had sat in both letter-graded and pass/fail classes. One student reported not taking a letter-graded class; another reported not taking a pass/fail class. Table 2 presents a breakdown in gender and school membership among respondents.
To analyze the results from part I of the questionnaire, we began by doing a factor analysis of this section. Next, we performed t-tests to compare responses on different dimensions. We proceeded to examine correlations between student ability and student mastery scales. For part II of the questionnaire, we performed t-tests to compare responses on the effort and stress dimensions.Results
We entered the responses on the four scales, student ability orientation, student mastery orientation, school goal ability orientation and school goal mastery orientation into factor analyses. We conducted a total of two separate factor analyses--one for each of the two kinds of grading classes. We acknowledge that the sample size is not big enough for factor analysis, and that the number of items for the scales are few, especially for the school goal orientation groups (only 3 and 2).
Table 3 shows the rotated component matrix for the pass/fail items (Varimax rotation). Four factors emerged. Items 1, 6, 8, 10, all from the STM category, loaded on the first factor (loadings over 79%). Only item 16 from that category did not enter the factor. These items refer to students’ orientation towards learning, understanding and thinking. Items 2, 5, 11, 13, 15 with loadings over 54% loaded to the second component. They all come from the STA category, apart from item 5 (SCA category), and all include a notion of comparison to other students. Items on school orientations were too few to form distinct factors as the student orientation items did, but it is worth mentioning that SCA items grouped with STA items that did not enter the STA component, and SCM items grouped with STM items that did not enter the STM component. Items corresponding to letter-graded classes produced very similar results.
We compared the students’ scores on the STA scale for the two types of classes. Students scored significantly higher on the STA scale in letter-graded than in pass/fail classes (t = 7.09, p < 0.001). In other words, students reported being more oriented towards ability features, competition and performance relative to their peers in their letter-graded classes than in their pass/fail classes. The same result occurred in the comparison of STA scales within each of the two schools (t = 6.62 for LS, t = 4.3 for SE, p < 0.001 for both). “Letter grade has added an additional element of competitiveness in an already competitive environment” was the comment of a student in the SE.
We proceeded to examine our hypothesis that students in pass/fail classes would be more oriented towards mastery and understanding of the material than in letter-graded classes. The STM scores in pass/fail classes were higher in both schools, though not significantly different from scores in the letter-graded classes in each school. In the whole sample the difference was only close to significance (t = -1.71, p < 0.1). However, some of the open-ended responses addressed this issue: “… but for some reason I get more out of my pass/fail course perhaps I do not feel as stressed in p/f classes allowing me to focus more on what I am learning”, or similarly “I … focus more on learning and being exposed to new information or increasing my knowledge base of an exist[ing] area”.
We also wanted to examine differences in goal orientations between schools. Analysis of the results produced insignificant differences in letter-graded classes; even though LS students scored higher in measures of student ability and perceived school ability orientations, and SE students scored higher in student mastery and perceived school mastery orientations. In pass/fail classes, SE students scored significantly higher in the STA scale only (t = -2.24, p < 0.05). Other differences were not statistically significant.
Interesting patterns on the correlations between STA and STM scales occurred, as can be seen in Table 4. In letter-graded classes, there seems to be a tendency for a negative correlation between orientation towards mastery and towards ability goals. Although scores on the STM scale are relatively high, the more students are directed towards ability comparisons, the less they report emphasizing mastery of the subject matter. In contrast, in pass/fail classes the two scales were positively correlated in the LS. High positive correlations were also found in the global sample for letter-graded classes between STA and SCA scales (r = 0.513, p < 0.001) and between STM and SCM scales (r = 0.648, p < 0.001).Analyses on effort and stress
In the second part of the questionnaire, respondents answered comparative questions, e.g. “In classes of equivalent interest to me, I would put more effort into a letter-graded class than a pass/fail class”. Effort and stress in the two kinds of classrooms were two dimensions that were compared. The hypothesis was that students put more effort and experience higher stress levels in letter-graded classes. School and gender comparisons were also conducted.
As regards effort, 79% of the students agreed that they tried harder in letter-graded classes. They had a score higher than 3 on the 1-to-5 scale. School and gender differences on the reported effort level were not significant. A student noted that “I tend to work harder in letter-graded courses…” and another pointed to workload as the “main difference… pass/fail classes seem to have less [workload]”. On the item “I try my best”, students responded with a high score in both kinds of classes, but the difference was in favor of letter-graded classes (paired t-test, t = 5.72, p < 0.001). Only 37% replied positively to the item “I think that a pass/fail class is less demanding than a letter-graded one”.
No student reported that stress level was higher in pass/fail classes, 13% were uncertain (chose option 3) and 87% reported higher stress level in letter-graded classes. The pass/fail option according to a student “reduces the pressure of having to excel (i.e. get an ‘A’)”. Again school and gender differences did not turn out to be significant.
LS students agreed to having grading options in more classes (mean score = 3.58), while SE students’ mean response was in the “disagreeable” area (mean score = 2.71). This difference was significant (F = 5.32, p < 0.05). Whenever there is a grading option, SE students were more agreeable to choosing letter grading than LS students (mean responses 3.18 and 2.54 respectively). This difference however was only close to significance (F = 3.96, p < 0.1). Two SE students noted that a pass/fail assessment “might be interpreted negatively” or “look weak” on a transcript, for example when applying for grants.Discussion
The purpose of this study was to investigate how grading systems influence student attitudes towards learning. “Student attitudes” were operationalized in terms of student reports on goal orientations, effort, and stress.
How do grading systems influence goal orientations?
The tendency to make relative ability comparisons was significantly higher in letter-graded classes than in pass/fail classes. Students in letter graded classes were more likely to be concerned about competition, their status in relation to other students, and their position in the eyes of the professor than they were in pass/fail classes. For example, one student wrote, “In graded courses, I only concentrate on achieving, on getting a grade that will help my GPA. In contrast, with pass/fail I focus on learning and am less afraid to ask ‘dumb questions’ or otherwise take intellectual risks”. Another wrote, “I sometimes feel that we’d be better off with no grades so the competition would be less...We could choose to focus our work and time purely on what we want to learn...”.
Furthermore, in letter-graded classes there was an inverse correlation between relative ability goals and task mastery goals. This means that the more oriented a student was towards social comparison and competition in letter-graded classes, the less oriented the person was towards task mastery. These relationships between relative ability goals in the two conditions held true at both the LS and the SE.
Findings about mastery were quite different than those about relative ability. Students did not show a significant difference in mastery goals in letter-graded vs. pass/fail classes. This was true for students overall as well as for each school separately. Apparently, receiving a letter grade or a pass/fail grade does not make a significant difference for these graduate students’ desire to learn and master the material in their courses. It may be that graduate students at highly selective universities already have a strong mastery orientation, and that this orientation does not vary significantly depending on extrinsic factors such as grading systems. A student commented: “I don’t think the difference between my learning experience in the 2 types of classes is very great. Since I take my schoolwork seriously, just completing assignments implies a high standard of effort and learning”. An alternative explanation would be that these graduate students recognized an implicit value judgment in the mastery items. In other words, they felt that the “right” answer would be to strongly agree with statements such as “My main goal is gaining a deep understanding of the material” (item 1), regardless of the grading system.
How do grading systems influence student effort?
How do grading systems influence student stress?
Roeser et al. (1996) conclude that student perceptions of the school’s goal structure influence the goal orientations that students adopt. Students who perceive strong “relative ability” goal structures in their schools will likely be more competitive and oriented towards superficial rewards. In contrast, students who perceive “mastery” goal structures will likely emphasize personal improvement, task mastery, and higher level cognitive strategies.
We compared LS and SE because of our own perceptions of different goal orientations at the two schools. For example, LS has an enforced mean, which means that in every letter-graded class half the students must receive a grade below the mean and half must receive a grade above it. Furthermore, all of the students are ranked according to their GPA; these rankings figure heavily in decisions made for future employment. In contrast, GPA does not figure as prominently in the assessments of SE students; also, Ph.D. work tends to be highly solitary and focused on a final product (thesis or dissertation). Given these different contexts, we used Roeser et al.’s theoretical model (1996) to hypothesize that LS students would adopt goals more oriented towards competition and relative ability, whereas SE students would adopt goals more oriented towards task mastery.
The fact that these hypotheses were only partially confirmed should not be taken to mean that the claim is wrong. First, our findings about relative ability show that a feature of classroom context as specific as assessment method influences students’ goal orientation. Although the model does not suggest that grades alone influence a relative ability orientation, it is noteworthy that in our study this turned out to be the case. It would be interesting to further explore the role of grading systems in contributing to perceptions of school goal structures. One reason why we did not find significant differences between students in the two schools with respect to relative ability or mastery goals may be because of similarities among the graduate students in the particular highly selective university. Students attending this institution probably share a common experience of succeeding in school, and perhaps this involves a combination of positive experiences with both mastery and relative ability orientations.
It is important to acknowledge that there were several limitations to our research design which make it hard to draw implications for the theoretical model: first, Roeser et al.’s model (1996) was developed among middle school students, whereas our model was applied to graduate students. Despite our best efforts to “translate” the questionnaire to be more age appropriate, it may not have been conceptually wise to do so. Concepts of mastery and relative ability have a strong developmental meaning for young adolescents, which may not hold for adults in their twenties and thirties. Secondly, our analysis did not examine differences in students’ perceptions of their school’s goal structure, due to the small number of items we included in those categories; we only examined students’ own goal orientations. If we had found that there were not differences in student perceptions between schools, we should not have expected any differences in goals. Third, our samples were small, and were not randomly selected. It is especially difficult to generalize about the influence of school goal structures, because selection bias is also present; findings about student goal orientations at LS and SE may be saying more about the types of students that go to these schools than about the particular schools themselves.
On the practical side, when deciding about whether or not to use letter grades, professors may want to consider findings about effort, competition, and stress. On the positive side, letter grades seem to stimulate greater effort by students. On the negative side, they produce more relative ability comparison and stress. Importantly, in either letter-graded or pass/fail classes graduate students have the same wish to master the material and to learn for its own sake. While these findings will not prescriptively tell professors what system to use, they do provide information about the consequences of adopting each system. Clearly, different attitudes are associated with different grading systems.References
Bell, A. J. (1994). Let grades be emphasized. Education Communication Association (Washington, DC. April 28-May 1).
Ebel, R. L. (1980). Evaluation of students: implications for effective teaching. Educational Evaluation and Policy Analysis, 2 (1), 47-51.
Gage, N. L., & Berliner, D. C. (1998). Educational Psychology (6th ed.). Boston, MA: Houghton Mifflin Company.
Milton, O., Pollio, H. R., and Eison, J. A. (1986). Making Sense of College Grades. San Francisco: Jossey-Bass Publishers.
Quann, C. J. (1984). Grades and Grading: Historical Perspectives and the 1982 AACRAO Study. Washington DC: American Association of Collegiate Registrars and Admissions Officers.
Roeser, R. W., Midgley, C., & Urdan, T. C. (1996). Perceptions of the school psychological environment and early adolescents’ psychological and behavioral functioning in school: The mediating role of goals belonging. Journal of Educational Psychology, 88 (3), 408-422.
Sadler, D. R. (1983). Evaluation and the improvement of academic learning. Journal of Higher Education, 54 (1), 60-79.
Schevitz, T. (2000, February 24). UC Santa Cruz to start using letter grades. The San Francisco Chronicle, p. A17.
Stallings, W. M. & Leslie, E. K. (1970). Student attitudes toward grades and grading. Improving College and University Teaching, 18, 66-68.
Weller, D. L. (1983). The grading nemesis: An historical overview and a current look at pass/fail grading. Journal of Research and Development in Education, 17 (1), 39-45.
Winter R. (1993). Education of Grading? Arguments for a Non-subdivided Honours Degree. Studies in Higher Education, 18 (3), 363-377.
Michalis P. Michaelides is Associate Psychometrician at The College Board, New York, NY. His research interests are in the area of educational assessment and measurement and research methodology. He can be reached at firstname.lastname@example.org or by phone at 1-212-713-8119.
Ben Kirshner is Assistant Professor at the University of Colorado, Boulder, CO. His research examines learning and development in organized activities outside-of-school, including youth civic activism, tutoring programs, and various forms of project-based learning.
The views expressed by the authors are those of the authors and do not necessarily reflect those of The College Quarterly or of Seneca College.
Copyright © 2005 - The College Quarterly, Seneca College of Applied Arts and Technology