Periodic Reporting for period 4 - SELECTION (Improving Educational Outcomes by Transforming the Selection of Future Teachers)
Reporting period: 2020-03-01 to 2021-02-28
This project is focused on developing theory- and practice-informed methods for selecting prospective teachers. In particular, we are designing and testing theory- and practice-informed teacher trainee selection tools based on an innovative situational judgment test (SJT) approach that uses indirect measurement of non-cognitive attributes using a scenario-based methodology. The research will: (a) dramatically improve understanding of the essential non-cognitive attributes of prospective teachers, (b) transform the teacher candidate selection process, and (c) provide system-wide improvements to educational, social, and economic outcomes by increasing the quality of teachers selected for training programs. The overall objective is to generate new research knowledge with novel and consequential theoretical and practical outcomes, with the potential to renew teacher selection procedures and improve educational outcomes by improving the quality of candidates selected for teacher training.
At the conclusion of this project, we have developed a viable validated methodology (situational judgment tests or SJTs) for future teacher selection activities in a wide range of settings and we have implemented these selection tools in multiple settings in the UK and internationally. We have shown that the selection tools that we have developed are predictive of teaching outcomes, and we have tested new ways to select prospective teachers, using multiple mini-interview approaches. In addition, we contributed to a new understanding of video-based SJTs and examined some of the issues regarding bias in teacher selection (i.e. gender and ethnicity biases) and how varying formats of selection tools can minimise these biases.
We have also developed a new approach to developing the non-academic competencies of novice teacher through a method (scenario-based learning) that is based on the classroom simulations created as part of the SJT development process. Our work has attracted the attention of our UK national government through the Department for Education, and we have had extended conversations with other international bodies (including ministries of education in Lithuania, Peru, Morocco). Finally, we have developed an attraction/recruitment intervention--'realistic job previews'--that raises the interest in pursuing a teaching career in STEM undergraduates who have not previously shown interest in teaching.
At the conclusion of this project, we have developed a viable validated methodology (situational judgment tests or SJTs) for future teacher selection activities in a wide range of settings and we have implemented these selection tools in multiple settings in the UK and internationally. We have shown that the selection tools that we have developed are predictive of teaching outcomes, and we have tested new ways to select prospective teachers, using multiple mini-interview approaches. In addition, we contributed to a new understanding of video-based SJTs and examined some of the issues regarding bias in teacher selection (i.e. gender and ethnicity biases) and how varying formats of selection tools can minimise these biases.
We have also developed a new approach to developing the non-academic competencies of novice teacher through a method (scenario-based learning) that is based on the classroom simulations created as part of the SJT development process. Our work has attracted the attention of our UK national government through the Department for Education, and we have had extended conversations with other international bodies (including ministries of education in Lithuania, Peru, Morocco). Finally, we have developed an attraction/recruitment intervention--'realistic job previews'--that raises the interest in pursuing a teaching career in STEM undergraduates who have not previously shown interest in teaching.
Our work is focused on four sub-projects. In Study1, we are focused on integrating deductive and inductive non-cognitive domains in our development of situational judgment tests (SJTs). We have now conducted the necessary preliminary work (through focus group and individual interviews) to test three new deductive non-cognitive domains--Integrity, Emotion regulation, and Growth Mindset--to complement the inductively-derived domains of Empathy & Communication; Organisation & Planning, and Resilience and Adaptability. Study 2, which examines the longitudinal predictive validity of SJTs, is currently underway with a pilot test linking teacher effectiveness and selection tools conducted in spring 2018, and further work in this area continuing in 2018-2019. Study 3, which examines expert and novice cognitive processing and decision-making when completing SJTs, is now complete with findings showing key differences and similarities between expert and novice teachers. Study 4, developing on-line video-based SJTs is currently progressing well, with a pilot test comparing video- and text-based SJTs scheduled for 2018 and continuing development of video-based SJTs through the next 12 months.
At the conclusion of the project we can see that we have developed an integrated framework of non-cognitive domains (i.e. non-academic competencies) that underpin effective teaching (Study 1). We have shown the longitudinal predictive validity of the SJTs we developed in multiple locations (Study 2), and we have explored the cognitive processing of expert and novice teachers as they approach SJT-like tasks (Study 3). We have developed video-based SJTs as proposed in Study 4, and have tested these in various settings. In addition, we have developed additional teacher selection methods using an multiple mini-interview approach and we have extended our SJT methodology through the creation of 'developmental SJTs, aka as 'scenario-based learning' tools that we have shown have a significant impact on the self-efficacy and classroom readiness of trainee teachers.
At the conclusion of the project we can see that we have developed an integrated framework of non-cognitive domains (i.e. non-academic competencies) that underpin effective teaching (Study 1). We have shown the longitudinal predictive validity of the SJTs we developed in multiple locations (Study 2), and we have explored the cognitive processing of expert and novice teachers as they approach SJT-like tasks (Study 3). We have developed video-based SJTs as proposed in Study 4, and have tested these in various settings. In addition, we have developed additional teacher selection methods using an multiple mini-interview approach and we have extended our SJT methodology through the creation of 'developmental SJTs, aka as 'scenario-based learning' tools that we have shown have a significant impact on the self-efficacy and classroom readiness of trainee teachers.
Now at the end of the project, we can show that the SJTs are better predictors of teaching performance than other measures currently used. We can show that video-based SJTs have certain benefits over-and-above text-based SJTs for selecting new teachers, but that these differences are not large. We can show that 'developmental SJTs' are a useful intervention that have proven to be a useful training and development tool for teacher training programmes. Our ERC-funded research has also resulted in commercialisation opportunities that we are currently exploring through an ERC Proof of Concept grant.
In addition to the 'expected results' from our project, we wish to highlight 3 main achievements that go beyond the state of the art and the expected results.
1. Cross-cultural development of inductive/deductive teacher non-cognitive attributes framework
We have been able to work in multiple contexts internationally to develop teacher attribute frameworks which form the foundation for developing selection tools.
Indicative publication: Klassen, R. M., Durksen, T. L., Al Hashmi, W., Kim, L. E., Longden, K., Metsäpelto, R.-L. Poikkeus, A. M., & Györi, J. (2018). National context and teacher characteristics: Exploring the non-cognitive attributes of prospective teachers in four countries. Teaching and Teacher Education, 72, 64-74.
2. Developmental SJTs (Sceanrio-based learning (SBL) interventions
We have extended the SJT methodology used in this project to provide developmental tools for teacher development. We have used our SBL methodology in multiple settings in the UK and now are developing a project in Kenya and Ghana with African partners.
Indicative publication: Bardach, L., Klassen, R. M., Durksen, T. L., Rushby, J. V., Bostwick, K. C., & Sheridan, L. (2020). The power of feedback and reflection: A brief online scenario-based learning activity designed to increase student teachers’ self-efficacy and classroom readiness. https://psyarxiv.com/whsny/. In press with Computers and Education.
3. Realistic job previews
We have again exploited the simulated classroom scenarios which form the basis of our SJT research to develop an attraction/recruitment tool that has proven to be effective as a means of raising the interest in teaching of STEM undergraduates.
Indicative publication: Klassen, R. M., Granger, H., & Bardach, L. (2021). Attracting prospective STEM teachers using realistic job previews: A mixed methods study. https://psyarxiv.com/r6efk/ Manuscript in review.
In addition to the 'expected results' from our project, we wish to highlight 3 main achievements that go beyond the state of the art and the expected results.
1. Cross-cultural development of inductive/deductive teacher non-cognitive attributes framework
We have been able to work in multiple contexts internationally to develop teacher attribute frameworks which form the foundation for developing selection tools.
Indicative publication: Klassen, R. M., Durksen, T. L., Al Hashmi, W., Kim, L. E., Longden, K., Metsäpelto, R.-L. Poikkeus, A. M., & Györi, J. (2018). National context and teacher characteristics: Exploring the non-cognitive attributes of prospective teachers in four countries. Teaching and Teacher Education, 72, 64-74.
2. Developmental SJTs (Sceanrio-based learning (SBL) interventions
We have extended the SJT methodology used in this project to provide developmental tools for teacher development. We have used our SBL methodology in multiple settings in the UK and now are developing a project in Kenya and Ghana with African partners.
Indicative publication: Bardach, L., Klassen, R. M., Durksen, T. L., Rushby, J. V., Bostwick, K. C., & Sheridan, L. (2020). The power of feedback and reflection: A brief online scenario-based learning activity designed to increase student teachers’ self-efficacy and classroom readiness. https://psyarxiv.com/whsny/. In press with Computers and Education.
3. Realistic job previews
We have again exploited the simulated classroom scenarios which form the basis of our SJT research to develop an attraction/recruitment tool that has proven to be effective as a means of raising the interest in teaching of STEM undergraduates.
Indicative publication: Klassen, R. M., Granger, H., & Bardach, L. (2021). Attracting prospective STEM teachers using realistic job previews: A mixed methods study. https://psyarxiv.com/r6efk/ Manuscript in review.