Evaluating the impact of professional learning

The importance of evaluation

Effective professional learning should support teachers and leaders to reflect on, question and deliberately improve their learning and teaching practice. Additionally, teachers and leaders need to assess the effectiveness of the professional learning programs they undertake to improve the quality of future professional learning and ensure it results in an increased impact on learners’ outcomes. Teachers and leaders of professional learning should ultimately be evaluating the impact of the change.

Useful evaluations can provide reliable information for making responsible decisions about ongoing professional learning and the effects of the implemented changes to practice. There are various ways to evaluate the effectiveness of professional learning.

Evaluation for continuous improvement

Changing teacher and leader practice is not a linear process. It is a continuous cycle that can include:

• active engagement with teachers in a collaborative setting and raising awareness about the goals for improvement (based on an analysis of current learners’ outcomes)
• determining what is currently working well and why
• understanding the professional learning goals of individual teachers
• increasing teacher knowledge which includes the exploration of how and why different practices are more effective than those being used currently
• application of the knowledge within planning and classroom practice through the first attempts at change, then analysing successes and problem-solving issues that arise
• sharing of expertise, reflection and ongoing improvement of that practice.

How to evaluate the effectiveness of professional learning

There are four factors to consider when evaluating the impact of professional learning. Evaluation should:

• be considered from the start
• measure outcomes at multiple levels
• focus on changes in teacher and leader practice leading to improved learner outcomes
• track change over the short, medium and long-term

Multiple strategies should be used when evaluating the impact of professional learning to assess its effectiveness for educators and learners. By measuring the impact of change from various points of view the results will be practical and applicable.

Figure 1: Effective evaluation of professional learning, from the Essential Guide to Professional Learning: Evaluation

Read AITSL’s guide for information on collecting and analysing learners’ data as part of your evaluations.

Evaluation checklist

Is the evaluation of professional learning driven by clear and measurable goals for improvement?

• What are the desired changes in:
  • learner outcomes?
  • teacher and/or leader skills, knowledge and practices?
• How are teachers actively engaged in the design and implementation of the evaluation of their professional learning?
• What will be the indicators of success in the:
  • short-term?
  • medium-term?
  • long-term?

When will the evaluation of professional learning take place?

• When will initial impressions about the professional learning and changes in thinking be gathered?
• When will the impact of the professional learning be visible?
  • How long do teachers need to work with the idea before it can be put into practice effectively?
  • How long will the changes need to be in place before there will be detectable improvements in learner outcomes?
  • When will be an appropriate time to re-evaluate? 

How will the results of the evaluation be used to inform ongoing planning for professional learning?

• Was the evaluation of the professional learning built into the plan from the start?
• Are there opportunities to adjust the implementation of the professional learning plan in response to findings?
• How will the final summative evaluations be used to inform future planning?

Evaluating for sustainability and embedding of practice

Research shows that teachers need to see the impact of a new instructional approach or innovation working in their classroom before they are willing to commit to its long term implementation (Guskey, 2002). This visible evidence – impact on learner outcomes – is the key lever for change. Once this has occurred, the new approach can be embedded into your practice and continue to reap rewards in terms of learner outcomes.

Figure 2: Stages of the embedding of practice (Guskey, 2002)

Connecting with other teachers who are implementing the same strategies is especially useful in whole site approaches, if multiple teachers have attended the same professional learning, or if a member of staff has returned and shared their findings.

It is also a good idea to incorporate the implementation into professional goals for the year, or over several years depending on the approach. This serves as a reminder to reconsider this practice, check if it is still working or if anything needs to be adapted.

Measuring efficacy over time

Once an approach has been tried and embedded into practice, create checkpoints to review. These could be:

• regular professional goal review and feedback conversations
• learner assessment or reporting
• learner perception surveys
• professional learning community or team meetings for shared professional learning goals.

Site-wide evaluation and change

Collecting data at a site-wide scale can be daunting and complex, but it is achievable when a strong professional learning culture is fostered. In such a culture, leaders and teachers are invested and can see the potential for positive change. With all staff working towards shared outcomes, such as whole-site goals and plans, evaluating impact enables progress to be tracked and good practice to be replicated.

Thomas Guskey - Plan right, and evaluation takes care of itself

Most school leaders see evaluating professional learning as a burdensome task that involves collecting data when activities are completed to determine if anything made a difference. These formal, add-on, summative responsibilities require time and energy that busy school leaders can ill-afford. But in truth, if professional learning is planned well, nearly all essential evaluation tasks are addressed before activities begin. When you plan well, evaluation takes care of itself.

How do we plan well? One of the best ways to plan effective professional learning is simply to reverse the order of steps outlined in Evaluating Professional Development (Guskey, 2000), starting at the end and working backward. We begin by describing the student learning outcomes we want to influence and what evidence we trust to verify that impact. These outcomes typically relate to academic achievement goals, but also may involve particular student behaviours, attitudes, or dispositions.

Second we must select the strategies, techniques, or practices most likely to produce those results. This entails collaboratively investigating the research supporting those strategies, the quality of that research, and the critical elements involved in implementation.

In the third step we identify the specific aspects of organizational support required to ensure high quality implementation. For example, do our schools offer a safe and supportive environment that encourages innovation? Are we ready to provide teachers with the resources, time, materials, and, most importantly, backing and assistance from school leaders needed to put these strategies into practice?

Fourth we need to specify the knowledge and skills required for implementation. In other words, what do teachers need to know and be able to do to implement these strategies with fidelity in the classes they teach? And the final step is determining the activities and experiences that best help teachers acquire that essential knowledge and skills.

By following this backward planning process, school leaders will address nearly all essential evaluation questions, determine upfront what evaluation evidence they need, and provide a framework for ensuring professional learning activities are optimally effective in improving teachers’ practice and student learning outcomes.

References

Australian Institute for Teaching and School Leadership 2012, Australian Charter for the Professional Learning of Teachers and School Leaders, AITSL, Melbourne. https://www.aitsl.edu.au/docs/default-source/national-policy-framework/australian-charter-for-the-professional-learning-of-teachers-and-school-leaders.pdf?sfvrsn=6f7eff3c_6

Guskey, T., Professional Development and Teacher Change, Teachers and Teaching: theory and practice, Vol. 8, No. 3/4, 2002, http://keystoliteracy.com/wp-content/pdfs/orc-implement-science/PD%20and%20Teacher%20Change.pdf

Guskey, T. R. (2000). Evaluating Professional Development. Thousand Oaks, CA: Corwin.