Skip navigation
College Quarterly
Fall 1995 - Volume 3 Number 1
The Inescapable Reality of Assessing College Performance
by Wm. Berry Calder and M. Elizabeth Calder

As Canada's provincial college systems approach the 21st century, the issue of how an individual college is performing becomes an increasing priority. The need to discuss, create and implement performance assessments becomes a provincial imperative, if not an inescapable reality for individual colleges. There are many compelling reasons for a college and college systems to start evaluating their performance.

First, education is no longer relegated to individual communities or areas. Faculty, educational leaders, and administrators now talk in more global terms as they define their strategic mandate. Thus, we have provincial college systems and colleges that are competing among themselves, with other educational systems, and with private sector trainers, for students, limited dollars, and professional recognition.

Second, as government agents of social and economic development, colleges are being asked by stakeholder groups to put concrete numbers on how well they are 'performing.' The public is making colleges and college systems more accountable for their spending. How are their limited dollars actually being spent? What are the outcomes of colleges? Who are the graduates of our provincial college systems and what are they giving back to the community and society? Performance assessments provide a historical record of how well a college has done and also a foundation for future changes.

Third, there is a strong need to have a common understanding regarding performance expectations of colleges. Canadian governments (local, provincial and federal), taxpayers, the media, governing boards, students, faculty and administrators are all asking questions about performance and each look at the issue of performance from very different perspectives. Cost/benefit ratios (i.e., comparative costs associated with student numbers in a program and the unit dollar costs of 'educating' them) could be seen from a wide variety of viewpoints depending on the special interests of the particular constituent group. Being in a competitive educational market, the need for each college and college system to work toward common performance goals is very important in order to answer questions from prospective students and external institutional stakeholders.

Finally, if we as a community of colleges do not develop a performance assessment process, it is inevitable that someone else will. Recently, the Canadian university system had its 'performance' measured and its members ranked by a national news magazine. This ranking caused tremendous concern among the universities and forced some of them to justify their academic program offerings.

Even though performance assessment is necessary, however, the assessment and management of college performance can be a real 'double-edged sword.' On the one side, it can play an important role in improving the effectiveness of an organization and developing better collaborative decision-making systems. When assessments of performance measures are used effectively, they can provide college decision-makers with valuable feedback on the approaches being used by staff and faculty in moving an institution toward its strategic goals. Likewise, the information can also show whether or not institutional improvements are producing the intended results. On the other side, though, it can stand in the way of real institutional improvements. If a performance assessment is poorly done or misused, the apparent relative value of 'performance outcomes' may have a negative impact on institutional morale, as well as stifling progressive actions, demoralizing team-building efforts and discounting quality initiatives. Even though a college's performance and assessed quantitative results form a historical record of how well that college has done, it may be an unreliable predictor of how immediate strategic decisions will impact tomorrow's results. It is tomorrow's results about which colleges and their constituents are concerned. The results of assessment will address what performance improvements must be made. The assumption, then, is that the college will move in progressive ways.

If college educators were asked by stakeholder groups to provide a framework for the assessment of performance, they might begin by referring to an institution's mission. Mission statements, however, can contain somewhat unmeasurable or vague language such as 'a college education that is innovative and learner-centred.' Moreover, determining performance-type outcomes such as retention rates and graduate employment rates often lacks support by the institution as a whole. The search for answers or, at the very least, a 'common ground' of understanding upon which to base institutional assessment becomes more one of looking for empirical outcome measures than one of seeking institutional stakeholder consensus on goal statements.

The identification of an institution's strategic mandate dictates what should ultimately be measured to assess performance. Certain key strategic areas such as student outcomes, educational processes and institutional services, must be identified. Colleges need to decide what is important and then set about targeting those areas with the results of performance assessment. While many performance measures are designed to meet the internal needs of the college (e.g., budgeting, human resource management), the emphasis of the assessment needs to reflect what is important to the institution as a whole (not just what is urgent) and what is valued by all those who serve in improving performance. In many ways, what the indicators say is much less important than what is being done with the information. Measurements that do not lead to meaningful action are of no real value and are wasteful of valuable institutional resources. Measurement is an essential performance tool for improving colleges. Choosing the right tool is important, but how skillfully the tool is used determines its overall effectiveness to institutional stakeholders.

In post-secondary education, especially in provincial college systems, there may be a number of general indicators that can be used to develop college performance. Among them are the number of training contracts and dollars awarded to an institution, the time it takes for a student to complete a program of study, the percentage of students changing their program or course, and students' overall satisfaction with their educational experience. But for performance indicators to be useful they must be clearly linked to a college's strategic goals, values and directions.

Whatever institutional measures are used to describe and to assess performance, they must have identifiable characteristics in order to be valued by staff and those critical observers of educational accountability. One of the first steps a college must take is to state its performance measures in objective and quantifiable units, if they are to win any degree of acceptance by staff, faculty and students. As well, different performance measures for different areas must be stated in the same units so that they may be 'transferred' and summed for meaningful interpretation and analysis. Retention rates and student cost/benefit ratios, for example, may need to be linked to performance analysis. Finally, performance measures must be flexible enough to permit individuals with different purposes and value constructs to use them for their own purposes.

The challenge, then, to assessing any meaningful college performance becomes one of identifying measures that meet the criteria of objectivity, quantifiability, acceptability, transferability and flexibility. As Berkowitz (1995) suggests, this will eliminate the confusion about what performance indicators are and what they mean. Institutional researchers know from experience that meeting all criteria all the time is practically impossible. Therefore, some degree of compromise is essential in determining performance indicators and providing a framework for measuring performance changes. “But even a useful indicator,” says Berkowitz, “may not on its own provide the information one expects. A retention rate may go up, but the fact that fewer students are dropping out does not in itself demonstrate any achievement on the part of the institution. It could be that the economy is in the dumps so students are staying in school rather than seeking non-existent jobs” (Berkowitz, 1995: 6).

Brown (1974) believes the realistic choice for measuring college performance is between no measures and imperfect measures that are based on subjective judgements. In this choice, criticisms from stakeholders that the performance measures used are not absolutely perfect are not relevant here. It follows then that measures that add educational value to the assessment are by far the most meaningful to institutional researchers, college educators and stakeholders. Changes in students' reading rates, mathematical competencies, and ability to think independently, etc., are all good examples of added value performance measures and should be used as meaningful assessments, whenever they are available. One problem in developing meaningful measures, however, is a characteristic known as addivity. There are certain measures such as increases in student retention rates and other measures like graduate employment rates that do not necessarily add together in any meaningful way for identifying performance outcomes.

The assessment of a college's performance is an ongoing process, and feedback from constituent groups is necessary to modify successive calculations of performance. A consistent method of looking at performance is essential to help institutional researchers, educators and stakeholders to understand the complex nature of college operations better and to have a positive impact on its educational outcomes. The assessment of a college's performance will provide a common starting point for dialogue in improving performance measures of colleges. It will provide a forum for different perspectives. It will change how colleges dialogue with each other and their stakeholders. As well, performance assessment will continuously strive for educational excellence which, in turn, will ensure that programs offered are relevant.

References

Berkowitz, P. (1995). Judging Performance. University Affairs, June-July, 6-7.

Brown, D. G., (1974). The outputs of higher education: Their identification, measurement, and evaluation. American Council of Education.


Wm. Berry Calder is President of the College of the Rockies in Cranbrook, B.C., M. Elizabeth Calder is Professor of Nursing at Georgian College in Barrie, Ontario