Background
The Department for Education (DfE) has provided annual grant funding for the design, delivery, quality assurance and evaluation of the Education and Training Foundation (ETF) Leadership and Governance (L&G) development programmes since 2014. In March 2023, DfE awarded ETF a two-year contract to provide L&G programmes to leaders and governors in further education (FE) and sixth form colleges. These programmes aim to support DfE’s delivery of the Skills for Jobs White Paper ambition for “excellent college governors and leaders [who are] pivotal to delivering high-quality provision and enabling their workforce and learners to succeed”.
ETF commissioned Fortia Insight (previously part of RSM UK Consulting LLP) to conduct an independent theory-based impact evaluation of the three L&G development programme areas delivered from 2023 to 2025. The programme scope has evolved over time to comprise some continuing but refined programmes, as well as some new or substantially changed programmes.
Fortia Insight’s evaluation has just concluded; however, ETF has incorporated recommendations from a previous evaluation (2022–2023) into its existing programmes.
- Improved lines of communication with providers: Process evaluation recommendations from the previous evaluation have been used by ETF to share participant feedback with training providers in real time. This enables agile adjustments and adaptations to better meet participant needs.
- More streamlined evaluation process: Recommendations from the previous evaluation resulted in ETF opting for a longitudinal approach and a two-year evaluation contract. This enabled ETF to capture both short- and medium-term outcomes from programmes.
- Continuing engagement with participants: ETF conducted alumni events as a result of Fortia Insight’s recommendations, bringing together current and past programme participants. Continued engagement enabled solidifying concepts from the programmes and additional benefits of networking.
Challenges
- Synthesising outcomes across multiple different programme areas: Fortia Insight applied a theory-based evaluation approach, conducting a theory of change exercise to establish causal pathways and common outcomes for leaders and governors in the FE sector.
- Capturing longitudinal insights: Fortia Insight aimed to capture evidence of short- and medium-term outcomes using baseline and endline surveys, as well as in-depth interviews with programme participants six months after training.
- Re-engaging programme participants after an intervention: A drawback of the longitudinal approach was potential disengagement of programme participants after they took part in training. Fortia Insight worked with the client to develop tailored and planned communications to keep stakeholders engaged.
- Synthesising mixed data sources: Fortia Insight brought together qualitative evidence from 47 programme participants and quantitative data from approximately 300 survey responses, including FE leaders and governors.
- Adapting the research plan: Fortia Insight re-scoped the project following a restructure to the client’s programmes. Agile working and regular communications with the client ensured this was a smooth process and did not impact the quality of insights gathered.
Impacts
Key evaluation findings include evidence demonstrating the following.
- Direct contribution of ETF programmes to improving skills, knowledge and capacity: Programme participants were able to evidence how ETF programmes influenced behaviour change and decision-making, including how they managed their teams and organisations.
- Improved strategic and change management skills: Senior leaders in FE institutions cited increased strategic leadership skills making them better equipped to manage restructures, mergers and curriculum changes.
- Increased collaboration within and between FE sector organisations: Programme participants were able to cite examples of outcomes achieved as a direct result of networking through ETF programmes through formal and informal collaboration with other colleges and institutions.
- Increased resilience and motivation to stay in the sector: Over 80% of programme participants felt highly motivated to stay in the FE sector, showing an almost 20% increase from baseline.
Improved ability to handle a complex regulatory environment: Programme participants cited ETF programmes as a contributing factor to helping them navigate regulation and policy change in the FE sector.