Introduction and learning outcomes
On completion of this weeks activities, students should be able to:
- describe similarities and difference between evaluation and research
- assess barriers to evaluation
- examine methods for conducting an evaluation
Monitoring and Evaluation
An important component of being a reflective educator is program evaluation. There are many similarities between program evaluation and research methods, however evaluation is not necessarily research nor vice versa.
There are three additional readings on curriculum evaluation approaches in the reference list available through eReserve (Hall 2014; Kesting 2015; Lindemann & Lipsett 2016) . Please review these if you are interested in building on your knowledge of the field of evaluation.
Boland (2015 p278-9) cites Wandersman et al’s (2012) nine principles associated with empowerment that educators should incorporate into their program evaluations. This raises some great principles to consider when planning and conducting program evaluations. Boland (2015 p285) then presents a list of things that need to be considered when developing and doing a program evaluation:
- identify and engage stakeholders
- clarify goals of the evaluation
- assess resources needed for evaluation
- design the evaluation
- determine appropriate methods of measurement and procedures
- develop a work-plan, budget, and timeline for evaluation
- collect the data using agreed upon methods and procedures
- process and analyse data
- interpret and disseminate the results
- take action
Given the purpose of undertaking a program evaluation is to judge the worth and value of a program, and its ability to meet the intended aims, it is vital that results be actioned, meaning it must lead to a modified curriculum as required.
Drawing on similar points of view, the following reading is another look at curriculum evaluation, but with a focus on technology enhanced learning.
All curricula, including those for the health professions, benefit from timely periodic revision. The revision process usually includes curriculum monitoring evidence, but also embraces emerging societal trends, health care innovations and educational practices is called curriculum renewal. The next reading articulates twelve tips on how to assure dynamic, ongoing curriculum renewal. The overall goal of the renewal should be to assure timely, evidence-based curriculum responsiveness to changes in practice, health care, student needs and educational approaches based on quality research (McLeod & Steinert 2015).
As mentioned in the presentation, evaluation methods draw on research methods for data collection, interpretation and analysis processes. The following is links to the BetterEvaluation site which will provide more detail on specific processes as well as philosophical approaches and theoretical underpinnings.
BetterEvaluation: An international collaboration to improve evaluation practice and theory by sharing and generating information about options (methods or processes) and approaches
The nine procedural steps promoted by Better Evaluation include:
An alternate online resource is the
This is also available as a downloaded PDF booklet here
Evaluation and research often overlap. In the following reading McKenna & Williams (2017) set out to study the near-peer learner and teacher experiences of participating in near-peer learning and to explore students’ engagement beyond the skill being learnt. What they found was related to the hidden curriculum of the educational activities. Such information, emanating from educational research should still be used to inform curriculum renewal.
Putting it all together!
Much of the programs at Flinders University have layers of curriculum evaluation:
- Tutors can undertake their own class learning/topic understanding through the ‘muddiest point’ etc
- Tutors can engage student feedback on teaching, peer evaluation or 360 degree review
- Topic coordinators can do topic evaluations – to check for consistency across classes, ensure ILOs are being met, teachers are delivering syllabus as intended.
- Topic coordinators can do learning analytics of student engagement
- Topic coordinators are required to review assessment outcomes for students and report to aberrations to a normal distribution curve
- University require SETs (Student evaluation of topic/teaching) to be done
- TC/School/College review of SETs
- Annual Course advisory committees
- Five year course accreditation reviews
- Annual university student satisfaction survey (not specific to individual topics thought)
- Annual national graduate surveys
Any of the aforementioned approaches to the evaluation may be employed.
Following this weeks readings, respond to one of the following activities.
Choose one of the following, and respond in this weeks discussion forum:
- describe similarities and difference between evaluation and research and how they may inform each other
- describe how you have actively contributed to an evaluation or curriculum renewal as an educator
- describe potential barriers to evaluation in a course/program you currently teach in or have great awareness of
- describe appropriate methods for conducting an evaluation in your new curriculum you are developing