Skip navigation
College Quarterly
Winter 2015 - Volume 18 Number 1
Defining and Measuring Teaching Excellence in Higher Education in the 21st Century
By Eileen De Courcy

Higher Education is in a time of immense change. Colleges and Universities are under greater pressure to demonstrate their value while experiencing increasing levels of economic constraints, changing accountability structures, and pressure to demonstrate excellence in teaching and learning/student outcomes. Technology, elearning and massification have fundamentally changed how faculty teach and interact with students. For administrators, defining and measuring teaching excellence in the new era will be progressively complex and elusive. The purpose of this paper is to examine higher education literature to explore frameworks/definitions and methods to define and measure teaching excellence and to investigate the usefulness of these performance measurement tools.

Introduction

Faculty, administrators and faculty developers in Canadian Universities and Colleges are increasingly pressured to demonstrate proficiency and excellence in teaching and learning. While excellence in teaching has always been a focus at Ontario’s colleges, universities have traditionally maintained a strong emphasis on research. External pressures from funders, students and other stakeholders are applying pressure on the entire post-secondary system to measure and legitimize the value of higher education (Levin, Kater & Wagoner, 2006). An analysis of scholarly literature exploring the massification and commodification of education situates post-secondary institutions within the new economy which is characterized by global competition, increased organizational flexibility, and driven by economic pressures (Levin et al, 2006). The Ontario Government’s discussion paper, Strengthening Ontario’s Centres of Creativity, Innovation and Knowledge (Ministry of Training Colleges and Universities, 2012) clearly defines the future vision of higher education as one being driven by economic constraints but supported by quality teaching and learning experiences:

Ontario’s college and universities will drive creativity, innovation, knowledge and community engagement through teaching and research. They will put students first by providing the best possible learning experience for all qualified learners in an affordable and financially sustainable way, ensuring high quality and globally competitive outcomes for students and Ontario’s creative economy.

Discussion: What are the competencies that you expect graduates of our institutions to have and how do you measure whether they have been acquired. How heavy a weight could learning outcomes have in a renewed funding formula? (p. 7).

A recent review of Ontario’s Colleges and Universities Strategic Mandate Agreements (2012), submitted to the Ministry of Training Colleges and Universities (2012) in response to their request to identify areas of innovations, productivity and differentiation in Ontario’s post-secondary institutions (2012), confirms, broadly, the shift from pure research focused institutions to teaching intensive institutions. Moreover, a shift can be seen generally with a renewed focus on teaching excellence, innovation in teaching and experiential learning as noted in mission statements of universities and institutions of higher education (Fitzmaurice, 2010). This shift is attributed to increased levels of accountabilities and an emphasis on measureable outcomes. Clark, notable scholar, stated that teaching in higher education cannot be judged because there are no adequate measures (as cited in Levin et al., 2006) yet institutions will be forced to demonstrate and authenticate their value. Colleges and universities will be looking for, with greater intensity, diverse ways to measure the effectiveness of their teaching.

Amidst the complex and changing circumstances, faculty’s role, working conditions, identity and professions are being challenged. Technology, elearning and massification have fundamentally changed how faculty teach and interact with students. For administrators, defining and measuring teaching excellence in the new era will be progressively complex and elusive. The purpose of this paper is to examine higher education literature to explore frameworks/definitions and methods to define and measure teaching excellence and to investigate the usefulness of these performance measurement tools. Their effectiveness to improve teaching is beyond the scope of this paper.

Defining Teaching Excellence in the 21st Century

External pressures have always influenced the working lives of faculty, however the pace, complexity and intensity of change has risen substantially over the past two decades (Day, 2012). A number of global trends disrupting higher education have consequentially impacted and, some would say, changed the role and working conditions of faculty. In response to economic pressures (Day, 2012) and a reduction in funding structures, faculty is expected to do more with less. Globalization and social migration have now positioned institutions competitively in an expansive international education market (Day, 2012). The decrease in funding and increase in competition for student enrolment has contributed to a shift in student demographics now seeing institutions welcoming a more diverse student population. The ubiquitous use of technology and an increase in faculty responsibility in institutional governance (Levin et al., 2006) has expanded and extended the types of work and working times for faculty both in and outside the classroom. There is little debate that teaching and learning in the 21st century requires a very different set of skills. Upon closer review, the literature is, seemingly, deficient, at least as far as I can tell, with very few studies attempting to redefine the role of faculty for 21st century teaching and learning. The lack of research emphasis in this area is problematic for institutions attempting to identify and measure exemplary teaching behaviours required for the current educational context. Still, an overview of the literature did provide enough fodder to speculate and describe some of the prerequisite skills. Whether or not they are measurable, with validity, will need to be determined. Nonetheless, defining/redefining and measuring teaching excellence skills in the 21st century may prove to be a herculean task.

Postareff, Lindblom-Ylänne, and Nevgi’s (2007) study exploring the effects of pedagogical training on teaching in higher education suggests, “teaching and learning are not two distinct phenomena” (p. 2). It has long been understood that learning outcomes are directly related to teaching approaches (Biggs, Kember & Leung, 2001; Postareff et al., 2007; Parry & Smart, 2007; Seidel & Shavelson, 2007; Shevlin, Banyard, Davies & Griffiths, 2000; Umbach & Wawrzynski, 2005) and that highly skilled and engaged teachers are positively associated with the adoption of more student-centered approaches to teaching (Trigwell, 2012).

So what skills, then, do excellent teachers in the 21st century need to possess? Pratt (2002) cautions against the adoption of any single dominant view of learning or teaching. A “one size fits all” notion of good teaching simply does not extend to encompass the many “philosophical orientations to knowledge, learning, and the role and responsibility of being a teacher (p.9). What frameworks, if any, are useful in determining skills and competencies for excellent teaching notwithstanding the changing dynamics and tensions between teaching and research, the use of technology and other disrupters? The review of research in this area calls for clearer definitions of high quality teaching and learning and for support structures, (professional development units) to be made available to promote better teaching practices (Scott, 2011). Zhu, Wang, Cai & Engels (2013) maintain that it is an unrealistic expectation to think that the current generation of students would benefit from traditional approaches to teaching and learning. Gibson (2010) proclaims “the time when academics in higher education could simply replicate the teaching methods that they experienced as students is quickly drawing to an end” (p. 3). The research clearly indicates that teachers are required to become more innovative and creative to support modern day learning (Gibson, 2010; Zhu et al, 2013).

Zhu et al (2013) further explore indicators of innovative teaching identifying the four core competencies as: learning competency, social competency, educational competency and technology competency. At the essence of these competencies is a willingness to embrace change, an openness to attempt and experience new approaches to teaching and learning, a willingness to take risks and the ability to be flexible (Gibson 2007). Specifically, Gibson (2007) refers to learning competency as the teacher’s willingness and ability to learn about innovative approaches to teaching and learning, to source new materials and to “solve teaching problems, study, self-reflection and research” (p. 2). Social competency describes a teacher’s ability to communicate to diverse groups of students, educational competency outlines the teacher’s passion for their disciple and technology competency underscores the teacher’s ability to integrate modern technology into their practice to enhance the learning outcomes (Gibson, 2007).

In other models, excellent teachers are characterized by their ability to co-construct learning with students. Described as a learning-centered approach to teaching, a series of strategies are designed to allow transformative learning to take place through active learning methodologies. Carnell’s (2007) analysis and resulting theory states, specifically, that effective teaching is characterized as: “learning is transparent, dialogue enables learning and a community of learners generates knowledge” (p. 6).

Abdous (2001) explores how the convergence of technology, learning management systems and social media applications are reconfiguring and transforming the higher education landscape and consequently the role of faculty. Abdous (2001) proposes a comprehensive framework which distinguishes faculty roles and competencies in the following distinct areas: Prior – planning and design; During – facilitation, interaction and feedback; and After – reflection. However, Skelton’s study (as cited in Fitzmaurice, 2010) concluded that excellent teaching abilities extend far beyond a set of skills that are easily qualified. Fitzmaurice (2010) states the belief that;

…individual attributes of the teacher, their ability to deal with complex human interactions and relationships, a concern for ‘weaker’ students, a commitment to student-centred methodologies and a commitment to ongoing professional development are very much part of what it means to be an excellent teacher (p. 10).

Fitzmaurice’s exploration (2010) suggests that teaching is not just an intellectual exercise. More importantly, teaching is “relational” and “involves creating and maintaining caring physical, cultural, intellectual, social and moral environments which induce learning” (p. 10). Fitzmaurice (2010) also contends that the evolving culture of accountability is influencing the current construct of teaching and learning that there is a “growing concern that teaching is being reduced to the acquisition of a set of competencies” (p. 1). Dinetke, Dolmans, Wolfhagen & Van Der Vleuten (2004) assert that good teaching cannot be described by listing a set of competencies and further suggest that existing frameworks are undermined when they do not pay attention to individual personal characteristics such as personality (p. 4). Dinetke et al (2004) continue the discourse on effective teaching recognizing that “changing visions on student learning and the teacher role require that teachers are continuously developing themselves professionally” (p. 1).

In an ever-changing context, identifying and classifying a finite set of skills for 21st century teaching is an unrealistic pursuit. The complexity and fluid nature of the teaching and learning experience is further intensified by personal perspective. The dichotomy between student and teacher perspectives further confounds the condition (Carnell, 2007) leaving one questioning if stable definitions and valid measures for 21st century teaching and learning can ever be achieved.

Measuring Teaching Effectiveness

The research on teaching effectiveness corroborates that there are a number of variables and dimensions to consider when describing the teaching and learning process. The research also indicates that there is some overall agreement, in the academy, that no one definition or set of existing dimensions can characterize the complexity of the teaching practice. Teaching and learning is contextual (Biggs, 2001; Dinetke et al, 2004; Fitzmaurice, 2010; Paulsen, 2002; Pratt, 2002; Saroyan, 2001), personal (Fitzmaurice, 2010) and situational and therefore very difficult, perhaps impossible, to measure with any sense of absolute certainty. However, colleges and universities have been trying to measure teaching effectiveness for many years. In a study tracking student evaluations of faculty, researchers noted an increase from 29% to 86% in a twenty-year period (Emery, Kramer, & Tian, 2003). Saroyan & Amundsen’s (2001) study on current methods of evaluating faculty found that 94% of the universities in Canada “use student rating questionnaires alone or in conjunction with some other method to evaluate teaching” (p. 4). Similarly, Wachtel’s (1998) research reports that there is an enormous amount of literature on student evaluations of faculty. Wachtel (1998) also situates the research by providing an historical overview of the genesis and evolution of faculty evaluations by era. Wachtel (1998) provides a comprehensive summary of the literature in support of and in opposition of student evaluations. Seven decades of research (Wachtel, 1998) provides copious amounts of data to support either view. Which leads to the conclusion that student evaluation methods will always be contentious. Adding to the controversy is the use of student evaluations in decisions making processes related to employment, retention, tenure and promotion of faculty (Catano, 2011). However, there has been some movement to recognize evaluation methods that extend beyond student questionnaires.

“Forces for change within and outside academe are modifying faculty work and the way that work is—or should be—evaluated”, (Theall & Feldman, 2014, p136) furthermore there have been several attempts to improve ratings and evaluation practice (p. 135). Theall and Feldman’s (2014) scheme position ratings as only one part of a “complex evaluation system” and reference Arreola’s (as cited in Theall & Feldman, 2014, p. 136) eight-step process designed to generate institutional dialogue prior to the evaluation process (p. 136).

Similarly, Paulsen (2002) provides an in-depth overview of contemporary approaches to evaluating teaching performance. His research explores and validates three key areas for consideration and posits that, “faculty should be actively involved in articulating and negotiating department specific responsibilities and criteria, and the methods and standards used to evaluate teaching”. Specifically, Paulsen suggested the following actions to assist in facilitating the evaluative process:

  • Clarifying expectation of and by the faculty
  • Identifying the nature and sources of data to be used for evaluation
  • Clarifying the purpose and uses of evaluation data

Clarifying expectations, as suggested by Paulsen (2002), is an imperative for institutions, departments and faculty. However, while in the process of describing expectations, it is critical that institutions and faculty understand the fluidity of the role and are able to map out both short and long terms changes in the role and in faculty work generally. Paulsen (2002) refers to the changing nature of the faculty role within institutions and how they are now required to pay greater importance to the art and skill of teaching. However, faculty are rarely rewarded for their good work in preparing and successfully executing on a good classroom experience. He suggests that the institution acknowledge and reward the behavior they expect.

Paulsen (2002) also provides a through overview of appropriate sources of data and their validity as a measure for faculty performance. He lists the following as the most common:

Student Rating – Quantitative student ratings of teaching are used more than any other method to evaluate student performance. Using between six and nine criteria or data points, reliability is generally strong. Regardless, student ratings are still perceived as problematic for faculty members.

Peer Review of Teaching – Teaching and Learning specialists recognize the importance of the peer review process in evaluating teaching performance. Similarly, the research peer review process is highly regarded and is accredited with increasing the quality of research. So too should the quality of teaching be improved under the peer review process. However, Paulsen (2002) notes that key issues and concerns include privacy, needs of the reviewer and the ability and validity of ratings needs to be considered. Once a foreign element is introduced, it automatically disrupts the natural state of the student-teacher relationship. Paulsen (2002) also questions what peer reviewers are best qualified to evaluate.

Self-evaluation or Report – Peer Review of the Teaching Portfolio – Self-evaluation are among the fastest growing sources of data used in evaluating teaching performance. Teams are often comprised of a dean, one peer selected by the instructor; another peer team approach to evaluation includes a dean, one peer selected by the instructor, another peer designated by the dean. This helps improve the validity of the rating.

Berk (2012) also outlines 12 strategies to measure teaching effectiveness. They are listed as:
Student rating, peer ratings, self-evaluation, videos, student interviews, alumni ratings, employer ratings, administrator ratings, teaching, scholarship, teaching awards, learning outcomes measures and teaching portfolio (p. 2). Berk (2012) suggests the following approach:

  • start with student ratings and one or more other sources that your faculty can embrace which reflect best practices in teaching;
  • weigh the pluses and minuses of the different sources (don’t bite off too much, but pick as many as possible);
  • decide which combination of sources should be used for both formative and summative decisions and those that should be used for one type of decision but not the other, such as peer ratings (p. 57).

The literature clearly demonstrates a variety of options available to both faculty and administration to measure and evaluate teaching effectiveness in meaningful and productive ways for all stakeholders.

Conclusion

As accountability measures become more stringent and government, along with other stakeholders, demand evidence of value and quality in higher education, colleges and universities will need to demonstrate their impact on student learning. In order to have a valid measure of teaching excellence, variables and dimensions need to be identified, agreed upon and then substantiated. This literature review simply skims the surface of this topic and indicates that accomplishing such a task would be no easy feat. Not considered within the scope of this paper, and what still needs to be determined, is the usefulness of faculty evaluations for actually improving teaching and achieving learning outcomes. To date, faculty evaluations have been perceived as a threat often confirmed by the dismissal of low performing teachers. Also, faculty have a tendency to discount the results claiming that external factors, beyond their control, are impacting the ratings (as cited in Theall & Fieldman, 2014). While “students’ evaluations of teaching effectiveness are the most thoroughly studied of all forms of personnel evaluation, and one of the best in terms of being supported by empirical research” (Theall & Fieldman, p. 369) there is still a gap in the literature pertaining to the changing role of faculty and faculty work in the 21st century.

This paper provides an overview of some of the current literature attempting to redefine teaching excellence in a time of immense change and the methods by which it could possibly be measured. Further research in a number of areas is required. There is little research on faculty’s perspectives on teaching excellence in the current educational climate, what they find most useful for development and what they perceive as good indicators of performance. Research regarding the alignment or misalignment of faculty student perspectives, and research regarding student outcomes correlated to faculty effectiveness in the new economy are all areas that require attention. A national study on teaching effectiveness in higher education in the 21st century would be of great interest.

References

Abdous, M. (2001). A process-oriented framework for acquiring online teaching competencies. Computer High Education, 23, 60-77 DOI 10.1007/s12528-010-90405

Berk, R. (2005). Survey of 12 strategies to measure teaching effectiveness.International Journal of Teaching and Learning in Higher Education, 17(1), 48-62.

Biggs, J., Kember, D., & Leung, D. (2001). The revised two-factor study process: R-SPQ-2F. British Journal of Education Psychology, 71, 199-149.

Cantano, V., & Harvey, S. (2011). Student perception of teaching effectiveness: development and validation of the Evaluation of Teaching Competencies Scale (ETCS). Assessment & Evaluation in Higher Education, 36 (6), 701-717.

Carnell, E. (2007). Conceptions of effective teaching in higher education: extending the boundaries. Teaching in Higher Education, 12 (1), 25-40

Day, C. (2012). New lives of teachers. Teacher Education Quarterly.

Dinetke, T., Dolmans, D., Wolfhagen, I., & Van Der Vleuten, C. (2004). The development and validation of a framework for teaching competencies in higher education. Higher Education, 48, 253-268

Emery, C., Kramer, T. & Tian, R. (2003). Return to academic standards: a critique of student evaluations of teaching effectiveness. Quality Assurance in Education, 11 (1), 37-46. DOI 10.1108/09684880310462074

Fitzmaurice, M. (2010). Considering teaching in higher education as a practice. Teaching in Higher Education, 15 (1), 45-55.

Gibson, R. (2007). Points of departure the art of creative teaching: implications for higher education. Teaching in Higher Education, 15 (5), 607-613.

Levin, J., Kater, S., & Wagoner, R,. (2006) Community college faculty: at work in the new economy. Palgrave Macmillan.

Ministry of Training Colleges and Universities (2014). Strengthening Ontario’s Centres of Creativity, Innovation and Knowledge. Retrieved from http://www.tcu.gov.on.ca/pepg/publications
/DiscussionStrengtheningOntarioPSE.pdf

Paulsen, M. (2002). Evaluating teaching performance. New Directions for Institutional Research, 114.

Postareff, L., Lindblom-Ylänne, S., & Nevgi, A., (2007). The effect of pedagogical training on teaching in higher education. Teaching and Teacher Education, 23, 557-571

Pratt, D. (2002). Good teaching: one size fits all? up-date on teaching theory, Jovita Ross-Gorden, San Francisco: Jossey-Bass.

Saroyan, A., & Amundsen, C. (2001). Evaluating university teaching: time to take stock. & Evaluation in Higher Education, 26 (4), 341-353

Scott, D. E., & Scott, S. (2011). Effective University Teaching and Learning. https://www.ucalgary.ca

Shevlin. B,. Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: love me, love my lectures? Assessment & Evaluation in Higher Education, 25 (4), 397-405. DOI: 10.1080/713611436.

Theall, M., & Fieldman, K., (2014). Commentary and update on Feldman’s (1997) “Identifying exemplary teachers and teaching: evidence from student ratings. The Scholarship of Teaching and Learning in Higher Education, 130-143. Springer.

Trigwell, K. (2012). Relations Between Teachers’ Emotions in Teaching and Their Approaches to Teaching in Higher Education. Instructional Science, 40, 607-621. DOI 10.1007/s11251-011-9192-3

Umbach, P., & Wawrzynski, M. (2005). Faculty do matter: The role of college faculty in student learning and engagement. Research in Higher Education, 46 (2).

Wachtel, H., (1998). Student evaluation of college teaching
effectiveness: a brief review. Assessment & Evaluation in Higher Education. 23 (2). 191-212. DOI: 10.1080/0260293980230207

Zhu, C., Wang, D., Cai, Y., & Engels, N. (2013)
What Core Competencies are Related to Teachers’ Innovative Teaching?
Asia-Pacific Journal of Teacher Education, 41(1), 9-27.


Eileen De Courcy, Associate Vice President, Teaching & Learning
Humber College Institute of Technology and Advanced Learning
eileen.decourcy@humber.ca