The question of when should a new training/education program be created is critical in the academic program planning for a college or institute. The purpose of this two-part article is to examine the role of needs assessment in the academic program planning process undertaken in post-secondary technical education. My earlier article on needs assessment (The College Quarterly, Vol. 2, No. 2, Winter 1994-1995, pp. 16-20) dealt with the nature and purposes of needs assessment as a planning tool. In Part Two, we will now focus on conducting and utilizing needs assessment more effectively.
Needs assessment can take a variety of forms based on the set of specific purposes and in the specific context. To be conducted effectively, a research plan needs to be developed first. The research plan defines the following three variables for a specific context. (Bohnen, 1988: p. 38)
- What information will be collected: that is, your focus;
- How will the information be collected: that is, your method;
- Who or what are your sources of information: that is, your sample.
Other similar frameworks can be found in Abbey-Livingston and Abbey (1982) and Witkin (1984), who outlines a set of questions that focus planning decisions and lead to an appropriate approach:
- Who wants the needs assessment (e.g. external or internal source)?
- Why is a needs assessment wanted (e.g. funding, decision making, etc.)?
- What should be the scope of the assessment (e.g. content and respondents)?
- On whose needs will you focus and on what level (e.g. on learners, the institution or on the community)?
- What kinds and amounts of data should be collected for your purposes (e.g. descriptive, performance or opinion)?
- What sources and methods might you use for data collection (e.g. qualified personnel)?
- What are your constraints on data collection (e.g. availability, quality, amount, costs)?
- What can you invest in people, money, and time?
- What needs assessment products meet your purposes, constraints, and resources? (Witkin, 1984: p. 36)
In planning needs assessment, it would also be useful to remember that our personal/professional ideology impacts on how we define information and how we use it in planning. To surface our ideology, especially our values, beliefs and interests, planners need to “build into their planning functions some form of self-evaluation process to increase their awareness of people who who relate to them.” (Hamilton, 1987: p. 19) This conscious self-reflection could take place at this initial planning stage of the needs assessment.
This question focuses the needs assessment and is linked directly with both purpose and context. It is essential to define the information requirements before conducting the research. By defining the information requirements in advance, the needs assessment can be better planned and conducted with the result that the findings will be better utilized by the decision makers. This can be facilitated through a list of specific requirements or questions that need to be answered. To deal with the issue of improper research design, Cross (1983) suggests asking the design question “what would we do differently if we knew how respondents would respond to this particular question” (p. 198) for each question that is being considered for inclusion in the assessment.
Some key questions that need to be answered by a specific labour market needs assessment include:
- occupational areas of greatest skill shortages;
- which of skill shortage areas will provide the best wages, job security or mobility;
- extent to which the demand will continue in the future;
- entry-level requirements in occupations with skill shortages including the education and skill requirements. (Bohnen, 1988)
At the Saskatchewan Institute of Applied Science and Technology, the following questions have also been raised:
- what is the educational background of people currently doing this work now for you?
- how many trained graduates with the required skill set would you hire (this year, next year, in five years, etc.)?
- would you pay this trained graduate more than an untrained person hired to do the same job and how much more?
The information required to answer these questions with a high degree of certainty needs to meet the research quality standards of validity and reliability. The information must have value for the needs assessment process and final product to be useful in the planning process. Some of the key attributes of valuable information include: timeliness, precision, accuracy, quantifiability, verifiability, accessibility, objectivity, comprehensiveness, relevance, and clarity. (Burch et al., 1983: p. 6)
The value of information should increase with additional processing. Dixon (1993) differentiates four stages of development from data to information to knowledge and ultimately to wisdom. For example, “knowledge,” he says, “does not exist until information has been made meaningful by discussing, questioning, debating and using it.” (p. 9) If the goal is wise decisions, then how the data and information are to be used becomes a critical factor in the planning of the needs assessment.
Needs assessment methods will vary according to purpose and context. As with most other research activities, there are two main sources of information to be gathered for the needs assessment: primary and secondary sources. The initial step is to gather information from secondary sources since there is usually considerable information already available, especially about the characteristics and conditions in the general labour market. Secondary data gathering in needs assessment involves literature reviews and analysis of existing records, reports, statistics and other related documentation.
Secondary labour market data can provide sufficient information to conduct various kinds of analysis. One potential form of analysis is an economic base study, a technique from educational planning in the K-12 sector. (Guy, 1978) In this technique, a special economic activity of an occupational group can be identified through a location quotient (LQ). The LQ is a ratio of ratios-a comparison of the ratio of specific local economic activity to specific national activity to the ratio of the total local economic activity to the total national economic activity. If the LQ is over 1, then the degree of local economic activity is greater than the degree of that specific economic activity across the country. A positive result like this could then become evidence used to support a need for training in that area of economic activity.
Data obtained directly from primary sources (e.g. students, employers, agencies. etc.) may involve a variety of research techniques. Methods most commonly used for gathering primary data include surveys (e.g. self-administered or mail-in questionnaires), individual interviews (e.g. face-to-face or telephone), and structured groups such as focus groups or community forums. Other applicable techniques include: delphi and nominal group; as well as observation and the use of a key informant. Choosing the most appropriate method involves a determination relating to research purposes, the information required, the sample characteristics, and the resources available.
The recommended option is to use multiple methods wherever possible. McKillip, for instance, advocated a convergent analysis, a “stepwise, multilevel, multitechnique strategy” in which the complementary multitechniques combine to provide various different perspectives into a holistic view of the situation. (1987: p. 95)
The next step is to identify the sources of information or to determine your sample. Bohnen (1988: p. 52) identifies three key decisions to make regarding the sample for the needs assessment:
- the sample size (how many people./documents);
- the specific samples (which people/documents);
- the sampling method.
Gathering information from secondary sources involves literature reviews and analysis of existing or found data. Data can be obtained from sources such as federal, provincial and municipal government documents and reports, licensing or accreditation agencies, business and industry reports, and other educational institutions. Among the numerous resources available for secondary data gathering are the federal government's new National Occupational Classification (NOC) system, the Canadian Jobs Strategy Designation List, the Canadian Occupational Projections System (COPS), and local bulletins and reports about local labour market conditions. Statistics Canada also provides a wide range of publications that deal with economic and labour force development. Provincial governments also publish regular reports on provincial labour market conditions and trends. Municipal governments may also have information on local conditions.
Other secondary sources include training and educational centres (both universities and colleges), public libraries, commercial information services, experts or consultants, business and trade publications, social planning councils, and even want ads in newspapers. Other educational institutions may even have conducted needs assessments already and may be willing to share information. Another growing source for information is computer based, both on-line databases or CD-ROM storage systems.
For sampling primary sources in the labour market needs assessment, the people that should be included in the sample are: employers; local advisory committees; business or industrial associations; unions; federal, provincial and municipal government officials; other educational and training institutions; other community resources such as chambers of commerce, labour force development boards, etc.). Potential students could also provide an indication of the potential demand by people seeking the training being considered. An example of the primary data related to key questions of the assessment and that can be obtained from a local survey includes:
- Number of local agencies, businesses/industries contacted;
- Number of respondents;
- Number of persons now employed in the occupation;
- Number of persons hired in these positions in the previous year;
- Number of projected positions (one year, five-years);
- Willingness of employers to hire graduates;
- Rate of turnover and estimated increased need;
- Types of positions and yearly salary ranges available to graduates (Raulf and Ayres, 1987: p. 14).
Once the data are gathered, the needs assessor must organize, analyse and interpret them to formulate some conclusions. Cross (1983) outlines four lessons dealing with the interpretation of data. First, the lesson of relativity deals with relative value of real numbers compared with percentages: both are needed in interpretation. Often low percentages may still mean high numbers of real people responding to a question. Second, the lesson of interpretation error occurs when the data are simply misread, or a consequence is misinterpreted or assumed incorrectly. This can occur especially if contextual information is lacking. The third lesson is the lumping error where segments of the population are inappropriately lumped together, leading to an inaccurate depiction of their true needs or interests. Fourth is the lesson from small picture error. Cross sees the necessity for a more artistic interpretation process as the “needs assessor needs the sensitive observations and experience of the adult educator as much as the adult educator needs the technical expertise of survey researchers. But what all of us need,” she continues, “is more imaginative, creative approaches to the art of needs assessment.” (p. 205)
Once the data have been analyzed and interpreted, conclusions can be drawn in answer to the initial questions outlined in the research plan. Then, a summary report has to be written to recount the findings, including the research methodology, data analysis, and the resulting conclusions or recommendations. Reporting mechanisms can take the form of a detailed research report, an executive summary, a detailed report combined with an executive report, or an oral presentation. The report could also be contained in a specific proposal, addressing the needs identified with a specific program design. Whatever the reporting mechanism, the results of the needs assessment are examined and used in a formal academic planning process to decide whether or not to respond to the identified needs outlined in the report.
One issue with the utilization of results revolves around decision making criteria and the process of priority setting. There is little guidance offered in the literature to assist with priority setting or determining factors to make a decision about whether a need is large/strong enough to warrant any training response by a college. The difficulty is most apparent if there are two or more apparent needs for education and training identified. How does a college choose one over the other, given limited resources to offer a program response? The answer to this remains a fugitive, for now.
Perhaps a clue to this decision making dilemma lies in the political nature of educational planning. Decisions about resource allocation are political decisions, so educational planning is linked directly to the politics of the institution. Weiler (1980) advocated a stronger link between planning and reform and called for planners to become more involved in the political processes that impact on both planning and implementation. Cervero and Wilson (1994) see planning primarily as a social activity, not simply a technical or scientific activity (i.e., “value free”), and subsequently view politics and ethics as central to planning. Planners, in this social context, have to integrate the roles as technical consultant, social activist and political organizer, and work in ways that are “ethically sensitive, politically astute, and technically sound” to plan responsibly. (p. 170) Needs assessors as planners need to heed this advice.
To be effective, needs assessments should be systematic and rigorous in design and methodology, and include measures and means to determine priorities for program decision making. In planning the needs assessment, assessors/planners should consider what information is required to fulfill the specific purpose, how information will be gathered and from what sample. Using multiple methods allows the assessor to obtain the most complete picture of the situation, including important contextual characteristics. Interpreting data and information should be done both carefully and artistically, combining technical expertise with an adult education perspective. Utilizing the results of the needs assessment in planning and decision making involves the political processes of the institution. Needs assessors/planners therefore must be aware of both the technical side of needs assessment and also the ethical and political aspects.
Abbey-Livingston, D. and Abbey, D. S. (1982). Enjoying research: A “how-to” manual on needs assessment. Toronto: Ontario Ministry of Tourism and Recreation.
Bohnen, E. D. (1988). Effective proposal development: a “how-to” manual for skills training programs. Toronto: Ontario Ministry of Skills Development/The George Brown College.
Burch, J. G. Jr., Strater, F. R. and Grudniski, G. (1983). Information Systems: Theory and Practice, 3rd edition. New York: Wiley.
Cervero, R. M. and Wilson, A. L. (1994). Planning responsibly for adult education: A guide to negotiating power and interests. San Francisco: Jossey-Bass.
Cross, K. P. (1983). The state of the art in needs assessments. Community/Junior College Quarterly, 7, 195-206.
Dixon, N. M. (1993). Organizing learning. Report 111-93. Ottawa: Conference Board of Canada.
Guy, A. (1978). Anticipating declining enrolments: A tale from two cities. Windsor Mini Study No. 5. Windsor: CODE.
Hamilton, D. (1987). Linking theory and planning action. Educational Planning 5 (4), 12-20.
McKillip, J. (1987). Need analysis: Tools for the human services and education. Newbury Park: Sage.
Raulf, J. F. and Ayres, M. C. (1987). The challenge of curriculum development: From idea to reality. New Directions for Community Colleges, No. 58, 9-23.
Weiler, H. N. (1980). The future of educational planning: Some sceptical notes. In R. Farquhar and I. Housego (eds.), Canadian and comparative education. Vancouver: University of British Columbia Extension.
Witkin, B. R. (1984). Assessing needs in educational and social programs. San Francisco: Jossey-Bass.
Grant Young is a program consultant with the Saskatchewan Institute of Applied Science and Technology in Saskatoon.