College Quarterly
Summer 2006 - Volume 9 Number 3
Home

Contents
Using Concept Maps as Assessment Tools: Defining Understanding

by Raymond W. Francis

Concept Maps provide a unique pictorial representation of an individual's understanding of a concept, problem, or idea. As tools in the undergraduate and graduate programs of an institution of higher learning, concept maps provide faculty with interesting and effective ways to assess learning by students in all academic areas. Concept maps can give students and faculty meaningful information about student performance, and indirectly the performance of faculty in the undergraduate and graduate classroom. These tools are objective, meaningful assessments through which the instructor can monitor student progress, self-evaluate instruction, and revise the delivery of instruction in the collegiate classroom. However, one question remains in making effective use of concept maps in student assessment. Specifically, what level of construction on a concept map represents a meaningful level of understanding by students?

What is a concept map?

Concept maps can take many forms. A concept map presents the relationships among a set of connected concepts and ideas. It is a pictorial way to display how an individual, or group, perceives a concept, problem, or topic. A concept map is constructed, in most instances, from general ideas to more specific ideas. In the following model the main idea (Level 0) located in the oval figure is the most general idea. Level 1 entries, located in the diamond shapes, are linked to the main idea. Links through Level 2, Level 3, and Level 4 are then constructed. Each level is more specific in content than the preceding level.

The preceding model demonstrates a situation where the student has more fully developed one aspect of an idea (links to the right), and has not developed or demonstrated understanding or learning in the other three links. In order to demonstrate learning, the perception exists that a concept map should include multiple levels of entries for all of the Level 1 entries.

Brief Research Overview

The use of graphic organizers as instructional strategies and for assessment of learning is not new. Graphic organizers have been used for many years, with published research on the topic including studies from the 1980s, such as Toms-Boronowski's (1983) study linking the improvement of student performance to the use of concept mapping. Margosian, Pascaralle, & Pflaum (1982) demonstrated that students using graphic organizers retained and used vocabulary terms more frequently than students not involved with graphic organizers, and Pittleman, Levin, & Johnson (1985) demonstrated that significant differences in learning and application of major concepts existed between students using graphic organizers and those who did not use a form of graphic organizer in their studies.

Research and applications from these early works has grown in many directions. Recent research in the use of graphic organizer includes Health (Tortora, 2002), Business (Novak, 2003), Philosophy (Steup & Sosa, 2005), Teacher Education (Haenisch, 2005; Schön, 1984), Geography (Strahler, 2005), School Administration (McEwan, 2003), Biology (Crowther & Cannon, 1998), Political Science (Johnson & Reynolds, 2005), Communication (Pehler, 2005), and many others.

Research Method Overview

Using the research question "what level of construction on a concept map represents a meaningful level of understanding by students?" as a focus, it was decided that the use of a random population approach to address the question would not provide an adequate answer. Instead, a representative sample of students preparing for their capstone field experience practicum were recruited based upon faculty recommendation. The approach was not to seek a random sample, rather to identify a sample of students performing above an average level. The assumption in using a representative sample of recommended participants was to use a sample of students who had achieved a high level of performance from which a baseline set of data could be determined.

A population of 51 teacher education students from a variety of academic areas, over three academic semesters, were asked to complete a concept map. Their topic was to depict "effective teaching" using a concept map. Participants represented content areas of mathematics, science, business, physical education, history, language arts, world language, art, speech communication, and health. Participants were instructed in the construction of concept maps during an orientation to the field experience, and completed the requested concept map during the opening seminar for the field experience.

Participants created their concept map representation of "effective teaching" during a fifteen-minute initial exercise and a five-minute follow-up opportunity. Each concept map was reviewed to determine the number of meaningful entries at Level 1, Level 2, Level 3, and Level 4 of the concept map. Due to the sample size, the non-parametric statistical procedure, Wilcoxon T-test for Dependent Samples (Falik & Brown, 1990), was employed to analyze the data.

Results

Examination of the data collected allows for a variety of observations. The greatest expansion of entries occurred between Level 1 and Level 2, with the greatest difference between levels being between Level 1 and Level 2. There is an approximate difference between Level 1 and Level 2 of a factor of four, an approximate difference between Level 1 and Level 3 of a factor of three, and an approximate difference between Level 1 and Level 4 of a factor of two. The relationships between factors can be viewed in Chart 1.

In addition to the general review of the data, statistical evaluation of the null hypothesis "there is no difference in the number of entries in the different levels of the concept map," was rejected in 66.67% of the calculations. Comparisons of the data revealed that there are four instances, of a possible six, where there are significant differences between the average number of entries for a given level of the concept map. These comparisons are presented in Table 2.

Table 2: Comparison of the number of entries for each concept map level.

Levels compared Result
Level 1 vs Level 2 Significant Difference
Level 1 vs Level 3 Significant Difference
Level 1 vs Level 4 No Significant Difference
Level 2 vs Level 3 No Significant Difference
Level 2 vs Level 4 Significant Difference
Level 3 vs Level 4 Significant Difference

Values reported at the .05 level

Discussion of the results

The reported significant differences (Table 2) are important to note. These statistical differences, demonstrate that between Levels 1 and 2, and Levels 1 and 3 there is a significant difference in the demonstration of understanding by participants related to the main idea. Students made connections between the main idea and Levels 1 and 2 sufficient to demonstrate both knowledge and understanding of the main idea. In other words, participants were demonstrating both a depth of knowledge by including entries in Level 2, 3, and 4, and as well as demonstrating a breadth of knowledge by having numbers of entries at a rate significantly different from Level 1.

One surprising result from the project was the ratio that emerged between the number of entries for Level 1, 2, 3, and 4. Using N as the number of links to the main idea, and L as the Level of the concept map, the ratio approaches the following equations for the number of links that demonstrate understanding of a topic: L1 = n, L2 = 4n, L3 = 3n, and L4 = 2n. Working from the premise that the participants were "above average" in their understanding and performance, these equations represent what we should have as general expectations for the demonstration of learning and understanding when using a concept map to assess student learning.

It should be noted that this project did not focus on the quality of the entries for each level. Only the number of entries for each level was monitored. It is possible that there exists a difference between the quality of the entries by the students for each level. This relationship should be explored in future research into the use of concept maps in the assessment of students. In addition, this study points to the idea that there is a general trend in data that indicates learning and understanding by students using concept maps. However, due to the use of non-parametric statistical measures, more data is required to provide for the use parametric approaches to the data.

Implications for the classroom

What does this study mean for the classroom? Specifically, it demonstrates that concept maps can be used as quantifiable assessment tools with some degree of objectivity. Concept maps can, and should, be used by students to represent their understanding about a concepts and topics. Faculty can then use the student's work as an evaluation tool with respect to the number of levels and number of entries to determine the depth and breadth of understanding by the student.

Using the collected data as a starting point and extrapolating to a variety of numbers of links to the main idea, the following table emerges from the calculations.

Table 3: Predicted values for numbers links to the main idea of a concept map

Number of links to the main idea Number of Level 1 entries Number of Level 2 entries Number of Level 3 entries Number of Level 4 entries Expected total number of entries
3 3 12 9 6 30
4 4 16 12 8 40
5 5 20 15 10 50
6 6 24 18 12 60

(Total = L1 + L2 + L3 + L4… This could also be rewritten as Total = 10(L1))

This table has great potential for effective, meaningful student assessment. If a student is given a topic and told that initially there should be 4 links to the main idea, it appears that to demonstrate an understanding of the topic, a student would need to develop a total of forty entries across four levels on the concept map. The application of this idea is easily seen for use in the classroom. However, more research is needed before broad generalizations about the process can be established.

Conclusion

In answering the question "what level of construction on a concept map represents a meaningful level of understanding by students?" it is evident that the concept map must demonstrate both depth and breadth of understanding of a topic. It appears that to demonstrate depth of understanding about a topic, an individual would construct a concept map with more entries at Levels 3 and 4 than the predicted values. In addition, if an individual is to demonstrate breadth of understanding of a topic, there will be more entries at Level 1 and 2. In order to demonstrate both depth and breadth of understanding an individual would follow the general trend of the total entries on the concept map being equal to the number of links to the main idea multiplied by ten.

References

Crowther, D.T. & Cannon, C.R. (1998). How much is enough? Preparing elementary science teachers through science practicum. In Proceedings of the Annual International Conference of the Association for the Education of Teachers in Science, Ruba, P. A., & Rye, J. A. (Eds.). Minneapolis: Association for theEducation of Teachers in Science.

Kirk, R.E. (1990) Statistics: an introduction. Holt, Rinehart & Winston Publishing.

Haenisch, S. (2005). Mathematics Concepts. AGS Publishing.

Johnson, J.B., & Reynolds, H.T. (2004). Political Science Research Methods. CQ Press (5th Ed.).

Margosein, C.M., Pascarella, E.T., and Pflaum, S.W.(1982). The effects of instruction using semantic mapping on vocabulary and comprehension. Paper presented at the annual Meeting of the American Educational Research Association, New Y0rk. (ED 217 390)

McEwan, E.K. (2003). Ten traits of highly effective principals. Corwin Press.

Novak, J.D. (2003).The Theory Underlying Concept Maps and How To Construct Them, Institute for Human and Machine Cognition, University of West Florida.http://cmap.coginst.uwf.edu/info

Pehler, S.R. (2005). Concept maps as a tool for learning standardized languages.International Journal of Nursing Terminologies and Classifications. 14(4), S39(3).

Pittleman, D.D., Levin, K.M., & Johnson, D.D. (1985)) An investigation of two instructional settings in the use of semantic mapping with poor readers. Program Report 85-4, Madison WI: Wisconsin Center for Educational Research, University of Wisconsin.

Schon, D.A. (1984). The reflective practitioner. Basic Books.

Steup, M. and Sosa, E. (eds. 2005). Contemporary Debates in Epistemology. Oxford: Blackwell.

Strahler, A. (2005). Physical Geography. John Wiley & Sons, 4th Ed.

Toms-Boronowski, S. (1983). An investigation of the effectiveness of selected vocabulary strategies with intermediate grade level students. Doctoral dissertation, university of Wisconsin at Madison. Dissertation Abstracts International, 1983, 44, 1405A, (University Microfilms No. 83-16, 238).

Tortora, S.R. (2002). Principles of Anatomy and Physiology. John Wiley & Sons Inc.


Dr. Raymond W. Francis teaches in Teacher Education & Professional Development at Central Michigan University. He can be reached at franc1rw@cmich.edu or raymond.w.francis@cmich.edu

Contents

• The views expressed by the authors are those of the authors and do not necessarily reflect those of The College Quarterly or of Seneca College.
Copyright ©
2006 - The College Quarterly, Seneca College of Applied Arts and Technology