Inspection of Operational Assurance in the Scottish Fire and Rescue Service

13. The Health and Safety at Work Etc. Act 1974 (HSAW) provides the legislative framework for occupational Health and Safety (H&S). The legislation sets out the statutory duties on both employer and employee in relation to H&S at work. This includes a duty as an employer to ensure, so far as reasonably practicable, the health, safety and welfare at work of all its employees. Additionally, employees have a duty under the Act to take reasonable care for their H&S and to cooperate with the employer to comply with their duties. The ability to capture, record and track information is essential in adhering to the Service’s duties under the Act.

14. The SFRS has further statutory duties as an employer under the Management of Health and Safety at Work Regulations 1999 (MHSWR), to manage the workplace appropriately in relation to H&S and to put arrangements in place to control H&S risks. The regulations place a duty on the employer to review workplace activity, such as preventative and protective measures, as well as review risk assessments. In the case of the SFRS, this includes operational activity on the incident ground.

15. The Fire Standards Board (FSB) is an independent body which oversees the identification, organisation, development and maintenance of professional standards for Fire and Rescue Services (FRS) in England, with the National Fire Chiefs Council (NFCC) as an integral partner. The Board details that a desired outcome for OL is that an FRS will have ‘developed a learning culture, acting on learning from operational and non-operational activity as well as external sources, to improve their operational response. The Service will have embedded the management of learning into their policies, procedures, tailored guidance and training. The Service will have developed a culture which seeks to share their learning with others to improve operational response within their own service; with other fire and rescue services; and with the wider sector if appropriate.’⁽⁷⁾

16. The NFCC also provide guidance regarding OL from a national basis, which forms the foundations of NOL. The NFCC NOL good practice guide⁽⁸⁾ states that the principle of learning from incidents ‘goes beyond simply identifying what went well or what might have gone wrong. While this information is useful in determining how things should be done, learning has truly been achieved only when some form of change is implemented that ensures actions will be different in the future.’ It further states that: ‘learning should also consider the organisational vulnerabilities that are identified during monitoring, audit and review processes. Effective learning from incidents also gives the opportunity to reflect on and understand the information and take action to reduce risk. It involves the organisation embedding changes so that, even if there are staffing changes, measures to prevent reoccurrence stay in place.’

Strategy, Policy, Process and Procedures

17. The SFRS H&S policy⁽⁹⁾ details that the Service is ‘committed to the continual improvement and compliance with its legal duties under the HSAW, and other supporting regulations, to ensure the safety of…staff and others who may be affected by…activities in the communities we serve.’ To achieve this, the SFRS has implemented a ‘H&S management system supported with topic-specific management arrangements, improvement plans and assurance processes to ensure legal compliance. Performance is monitored and reviewed by senior management through established governance processes ensuring continual improvement of our safety culture.’ It also details that they ‘are committed to sensible and proportionate H&S management that recognises the need to balance operational risk against firefighter and public safety. This policy is inclusive of and supports the content outlined within the SA Strategy 2022-2026.’

18. The SFRS Safety and Assurance (SA) Strategy 2022 2026⁽¹⁰⁾ details that the number one priority is to ‘work together for a safer Scotland and safety is at the core of everything we do’. It then goes on to detail that the safety objective is that ‘we will care for our people through progressive health, safety and wellbeing arrangements’. The strategy identifies five themes which are underpinned by priority actions.

19. The five themes are Compliance, Culture, Control, CI, and Communication and Engagement with related actions over a five-year period. Regarding CI, the Service details that it will monitor the effectiveness of H&S arrangements, to maintain continual improvement and performance and aim to enhance through ORL and implementation of assurance processes. Priority actions cutting across the five themes are OA-specific and some include:

a. create a programme for the development and implementation of topic-specific Health and Safety Management arrangements and OA procedures which are prioritised based on risk;

b. develop an OA campaign to embed and enhance the outcomes of robust operational assurance on the incident ground;

c. develop feedback arrangements to inform staff involved in changes following lesson learned;

d. develop business partner engagement feedback processes;

e. review operational performance through OA processes and make recommendations for improvement where necessary;

f. development of a lessons learned programme for organisational learning; and

g. develop and implement a programme of topic-specific SA audits.

20. The SFRS OA Policy details that the Service shall ensure that there ‘are suitable OA processes in place to provide effective feedback and review of performance at operational incidents and training events, to influence future practice, enhance performance and improve firefighter safety. This policy supports the concept of a learning organisation and supports the SFRS commitment to the continuous improvement of operational response and Health & Safety performance standards within the context of operational activity.’⁽¹¹⁾

21. The SFRS has aligned its OA management with the guidance model contained within the Health and Safety Executive (HSE) publication HSG65 - managing for health and safety, known as the ‘Plan, Do, Check, Act’ cycle. Figure 1 shows the key links between OA processes and the HSG65 cycle:

HSG65 Cycle

PLAN:

• Policy and Planning

DO:

• Risk profiling
• Organising
• Implementing your plan

CHECK:

• Measuring performance
• Investigating accidents, incidents and near misses

ACT:

• Reviewing performance
• Learning lessons

SFRS OA Processes

PLAN:

• Supports Policy and Procedure review process
• Audit & Review Programmes

DO:

• Data Analysis
• Reporting

CHECK:

• Station Audit
• Operational Review
• OA21 Investigations

ACT:

• Debriefing
• Communication Platforms

Figure 1 - HSG 65 Cycle mapped against SFRS OA process

22.The SFRS details that the primary function of OA is to review and assure all aspects of operational activity and capture learning, to improve individual and organisational performance in support of strategic objectives. To achieve this, an OA model and process diagram have been created to provide a clear and integrated approach to CI, underpinned by a targeted and robust methodology. Figure 2 shows the OA Cyclical Model and the key inputs and outputs.

23. The OA Model details three key areas of input;

• Station / Thematic Audit – a measure of pre-incident station preparedness, looking at how effectively policies and procedures have been implemented and how standards are being applied at Community Fire Stations (CFS);
• Operational Review – active auditing and monitoring during an operational incident, enabling the collection of information on the efficiency and effectiveness of policies and procedures; and
• Debriefing – reactive auditing and review of post-incident debriefs within a structured process, enabling the collection of information on the efficiency and effectiveness of policies and procedures.

24. The OA Model details three key areas of output;

• Learn – the gathering, analysis and reporting of pre-incident preparedness, operational performance and review of activities are key components of the OA process and support organisational learning;
• Improve – by learning from activities, the organisation will improve in terms of procedures, equipment and training which will ultimately enhance firefighter safety; and
• Develop – by reviewing activities, the organisation will continuously learn and develop with support from the OA processes.

25. The OA process diagram (Appendix 1) has been developed by the Service to capture the key workflow processes undertaken by OA and which can be summarised into five distinct areas:

• OA Inputs – the OA Department (OAD) primarily work with internal partners but also react to information provided from other stakeholders, such as other FRS, the NFCC, National Operational Guidance (NOG) and NOL;
• OAD – the OAD is responsible for the gathering, analysis, reporting and dissemination of operational activity information with a focus on organisational learning to promote firefighter safety;
• OA Outputs – the inputs are then processed by the OAD, resulting in a variety of internal outputs (Action Plans, Regional Safety and Assurance Improvement Groups (SAIG) Reports, Briefing Reports, etc.) and external outputs (NOL submissions and Multi-Agency reports);
• OA Governance – all OA activities and outputs are subject to a governance process; and
• Organisational Outcomes – the outcomes from the OA process support the continual review of SFRS policy and procedure, training standards, and assets and equipment, with a focus to continually improve firefighter safety.

26. The Service has developed a series of General Information Notes (GIN) that manage the processes with the cycle/model, so that input and output are consistent and align with pre-, during- and post-incident issues. These GINs are titled During Incident OA GIN, Station Audits and Thematic Audits GIN, and Operational and Event Debriefing GIN, and are discussed later in the report.

27. It is noted that Service documentation indicates a strong link between relevant statute and standards with the inclusion of a learning ethos in strategy, policy, process, models and procedures. This whole structure would appear to provide a positive foundation for the management of OA and subsequent ORL.

Governance

28. It can be observed from the OA process diagram (Appendix 1) that there is a linear process for managing OA input and outputs to achieve improved outcomes. Most commonly, information is collected by the OAD, which is then reported through their Functional Management Team. Primarily, outputs result in tasks that require to be actioned by ‘Local’ level managers with ‘Regional’ level matters actioned by the SAIG, i.e. a matter that is contained to just an SDA. ‘National’ or Service-wide learning are approved by the Safety & Assurance Sub-Group (SASG), actions progressed through the Operational Learning Group (OLG) and, thereafter, progress monitored by the Training Safety & Assurance Board (TSAB).

29. The purpose of the SAIGs is to support the delivery of the objectives at the SDA regional level, by providing strategic control within a local area context. The SDA SAIG Chair or ‘Lead’ (sometimes referred to as the SA Coordinator (SAC)) provides strategic guidance and direction to area-based managers, to ensure all H&S and OA processes are fully supported, and emerging issues are actioned accordingly. They are responsible for ensuring TSAB strategies are implemented at a local level and that Service standards are maintained. The SAIG deliverables are both a blend of Safety and OA business. Attendees normally required at the meeting are the Chair, an OA representative, GCs from Service Delivery (SD), Training, Prevention Protection and Preparedness (PPP) and Ops. In addition, the Service has designated Safety and Assurance Liaison Officers (SALO), and deputies, for each LSO area.

30. We found robust evidence that the SAIG meetings are routinely held and form an integral part of the OA governance and communication process which extends into the SD areas. OAD managers routinely attend these meetings to discuss reports with the local teams. The meetings have an order of business and a commensurate action plan, although this can be heavily weighted to safety issues rather than being OA-specific. SA management are aware of this and are working to improve understanding of assurance as part of the SA function which, supports the identification of learning and improvements in support of the safety of people. Managers provided feedback to us regarding the lack of attendance of some Directorates at the meetings, which occasionally restricted issue resolution, but this did not seem to stop the overall business.

31. Disappointingly, when speaking to SD middle and supervisory managers, we found limited awareness of SAIG meetings and their content, unless staff had been directly involved in attendance at the meetings previously. These SD staff were unable to articulate or provide evidence that OA-specific issues had been fed down or fed back up via this route and were unable to link this as a legitimate route for their OA issue resolution. Many supervisory staff were not able to provide evidence of interacting with a SALO or deputy SALO and could not recall being briefed about ongoing OA issues covered within the SAIG forum.

Area for Consideration 1

The SAIG, SAC and SALO are an integral part of the OA management and governance process. There is scope to improve the understanding of these roles for middle and supervisory managers. The Service should consider this potential improvement for any future training, development or review in relation to OA.

32. The SASG is a group established with the authority of, and under the remit of, the TSAB. The SASG is directed by and has its work agreed by the TSAB. The primary purpose of the SASG is to ensure priority is given to the introduction of control measures to manage H&S risks at operational incidents. The scope of the SASG includes, but is not limited to, considering any safety and assurance matter that assists the TSAB in the discharge of its responsibilities. This should include consideration of SA matters and determining the route to address identified risk.

33. The TSAB provides a forum where the strategic review of operational performance is undertaken. The TSAB is designed to provide visible leadership and ensure any relevant recommendations developed from OA activities are accepted by the appropriate Directorates and implemented timeously. Work streams and/or any Service-wide response should be agreed and allocated by the TSAB. Any subsequent action plans will be monitored through the OLG and OA processes with progress reports available to the TSAB. For scrutiny purposes, the TSAB reports to the Board of the SFRS, Service Delivery Committee (SDC) and People Committee (PC), via the Strategic Leadership Team (SLT) (Appendix 2).

34. The Service has recently supplemented the OA governance structure with the addition of an OLG. The purpose of the OLG is to support the SFRS in meeting its statutory obligations in relation to H&S, both in the operational and non-operational environment. Its scope includes, but is not limited to, considering any relevant safety and assurance matter that arises both externally and internally to the SFRS, and progress to completion of actions required to assist the TSAB and SASG in the discharge of their responsibilities. In effect it is a cross-directorate group that has a sufficient level of responsibility and leadership to be able to work through outputs in a timeous manner, thus speeding up the ORL process. Staff perceived that the ORL process was taking too long to improve outcomes and this group was created following the completion of a proactive Compare and Contrast (C&C) benchmark process.

Good Practice 1

Staff involved in the processing and management of OA provided positive feedback on the OLG and its development as a ‘clearing house’ for actions. It has been a positive addition to the governance process.

35. The Service has an OL Governance GIN⁽¹²⁾ that contains internal processes and arrangements for OA in the first instance. It focuses in the main on external sources of learning such as NOL, JOL, NFCC, HSE, Scottish Multi-Agency Resilience Training and Exercising Unit (SMARTEU) and other FRSs, as well as the process for feeding learning back to these organisations. The document details the responsibility for the learning and how it is integrated into the existing OA management and governance processes. The Service provided evidence to suggest that interagency (JOL) and intra-agency (NOL) learning is being used proactively and fed into the governance process to create OA outputs for TSAB consideration. This included examples of NFCC National Operational Learning User Group (NOLUG) case study information notes, NFCC NOL reports to SASG and SMARTEU multi-agency debriefs (JOL) involving Service personnel.

36. The Service provided evidence that a GC had been appointed the NFCC NOLUG chairperson, giving the SFRS a prominent national position within the UK FRS OL community. It also detailed that it had entered into a formal data sharing agreement that allowed sensitive information to be transferred between the two organisations. Additionally, and following a C&C benchmark process, the Service identified a need to have a dedicated Single Point of Contact (SPoC) for NOL contact and communication. It was considered that the SPoC should have suitable seniority and experience to deliver the learning activities identified in the NFCC guidance and manage the subsequent learning outcomes that arise from them, as well as outcomes received from and submitted to the wider sector. Consequently, the OAD GC was appointed on behalf of the SFRS.

Good Practice 2

The Service has developed a positive connection within the UK OL community and is viewed as a productive partner. Having a GC as NOLUG chair and the Service SPoC is an extremely encouraging indicator of the success of this relationship and should be given ongoing support.

37. The Service has a mature OA governance structure that is used for both internal and external information input and output management. It involves clear lines of reporting as well as key stakeholder and business partners who can contribute and engage with the learning process at an appropriate level. Action Plans and responsibilities are developed and monitored to ensure the effective management of tasks and timelines. Criticisms from staff of the process are the length of time it takes to deal with serious issues, compounded by, what is perceived by some, as the overly bureaucratic nature of the process and cross Directorate working. As detailed, this was recognised by the Service and the creation of the OLG, as well as improved OAD administration, has gone someway to improve this. Another criticism cited by some of those we interviewed, included a lack of reporting transparency, probably precipitated by the potential sensitive and litigious nature of the subject, leading to a level of frustration and suspicion. Lastly, difficulty in measuring success, or linking output to improved outcomes, was an ongoing issue throughout the inspection (and is discussed later in the report).

Systems

38. The Operational Assurance Recording and Reporting System (OARRS) is a bespoke software package hosted within the Service’s Information and Communication Technology (ICT) network, which is available via the SFRS intranet platform. The OARRS system was developed inhouse and has had a number of modifications over the intervening period. Ongoing redevelopment requires engagement with the original developers which has both financial, prioritisation and capacity implications for the Service and as such, has been limited. The system has several preset forms and parameters that prompt staff and allow a degree of free text flexibility to provide feedback and storage of information regarding incidents and audits. It is also linked to the email system which allows for a degree of automation and communication. The OAD can monitor the information input and are then able to extract data for assessment and analytics. Managers cannot access the system on the incident ground and as such input is always conducted post-incident.

39. Primarily, managers can input information regarding the pre-incident station audit process (OA02 Form), the during-incident review process (OA06 Form) and the post-incident review process (OA13 Form). This information is then stored within the OARRS. Operational staff can also self-generate these forms and there is a high degree of reliance placed on staff doing this, especially for smaller-scale incidents. In addition, OAD staff routinely undertake a review of incident activity and are then able to generate the requirement for an OA06 and OA13 submission, based on pre-agreed incident triggers, factors, types and scale.

40. Staff reported that the OARRS system was accepted as easy enough to use but was apt on occasion to be unavailable and unreliable. They reported that there was a high degree of manual intervention required in the use of the system as well as restrictions with access to submitted information from out with the OAD. In addition, staff also detailed limitations on functionality for analytics, interrogation and data retrieval to allow effective and efficient monitoring, analysis and reporting. OARRS is an integral part of the OA system within the Service allowing the collection of vast amounts of information and data input. The restrictions to the system are inhibiting the maximisation of that data and information use, to the detriment of developing efficient and effective output. It is understood that management acknowledge this position and notwithstanding the redevelopment issues previously mentioned surrounding capacity, finance and priorities, are in the process of procuring a replacement system.

Area for Consideration 2

The Service should consider the prioritisation of the OARRS replacement to improve OA data analytics and output development.

Structures and Administration

41. The Chief Officer is responsible for the discharge of the legal obligations that apply to OA and the content of the Policy. Whilst the Director of TSA is the strategic lead for OA and provides strategic direction and policy. The Head of SA has the delegated responsibility from the Director of TSA for strategic management of OA, whilst the OA Manager has a day-to-day responsibility for the effective delivery of the OA Policy. As detailed, the OAD GC is the NOL SPoC.

42. The OAD sits as part of the SA Function within the TSA Directorate. The department has six dedicated personnel, a GC who is the designated OA manager, two SCs and three WCs. The GC reports to the Deputy Head of SA who reports to the Head of SA. There is also an Operations Control (OC) SC who has the partial reference for OA and whilst not sitting directly within the OAD, does routinely engage and communicate with them. OC have indicated that they are actively assessing whether this manager becomes a full-time OA role.

43. The OA Policy⁽¹³⁾ details that the OA manager and the department are responsible for:

a. supporting the Director of TSA in the development and review of the OA Policy;

b. managing and delivering key business planning processes that support OA;

c. developing, monitoring and reviewing the OA processes and procedures;

d. developing operational performance audits and review programmes to support the delivery of agreed operational strategies and business plans;

e. ensuring an appropriate balance of proactive and reactive performance audits are conducted that support continuous improvement;

f. working collaboratively with other Directorates and using available resources in an efficient manner;

g. collating, reviewing, reporting and disseminating OA outcomes across all relevant areas of the SFRS;

h. assist in the development of operational performance audits and review programmes to support the delivery of agreed operational strategies and business plans;

i. manage and, where appropriate, deliver OA arrangements, e.g. pre- / during- / post-incident audits, monitoring, review and debrief;

j. support managers undertaking planned audit and monitoring activities at SDA and local level;

k. co-ordinate and collate data gained from pre- / during- / post-incident audits, monitoring, reviews and debriefs undertaken at SDA, Local Senior Officer (LSO), Station and Watch level;

l. liaise with national / SDA H&S teams in relation to timely progression of safety-critical information;

m. prepare reports on national, SDA and local performance;

n. collate and analyse national, SDA and local performance information;

o. liaise with LSOs, GC and SC on local performance issues;

p. monitor progress of areas of improvement actions; and

q. contribute to the effective delivery of the OA management system.

44. In addition, the Service also details⁽¹⁴⁾ the OA Department is responsible for:

a. ensuring the outcomes from SDA station audits are reviewed to identify SDA or national trends with the support of OA;

b. ensuring thematic audits are applied / supported, where deemed necessary, to ensure that lessons identified become lessons learned;

c. ensuring any notable practice(s) arising from operational activity is shared appropriately at a local and national level;

d. the support of a cross-directorate strategy for dealing, as necessary, with findings from internal and external investigations;

e. ensuring trends in OA activity are identified and disseminated as appropriate;

f. ensuring the OA21 investigation process is implemented at SDA level on instruction from the TSAB;

g. ensuring localised issues identified through OA processes are managed by the SAIGs through appropriate channels;

h. ensuring any issues identified through the OA process that cannot be resolved at a local level and/or have national implications are advanced to the SASG and/or the TSAB for consideration; and

i. ensuring NOL is shared across the SFRS.

45. It is observed that this is a significant amount of responsibility and workload for a small team, which we understand can be very labour-intensive and require a high degree of manual intervention, given the issue with OARRS previously mentioned. Throughout the inspection it was observed that staff were frustrated and/or disappointed by the lack of support, feedback, communication and engagement that could be provided by the OAD due to capacity issues. We noted that tasks and aspirational outputs assigned to the team seemed to outweigh the actual capacity of the team. Frequent comments indicated the perception that there was ‘a lot going into the system but not a lot coming out’. Many staff recognised this as one reason that OA may not be as effective as it could be and empathised with the OAD position.

46. We observed that the level of responsibilities assigned to the OAD was considerable and that focus on administrative tasks was done at the sacrifice of ongoing awareness development, communication and engagement. Given the voluminous nature of input and output generated, there seemed to be a need to review the blend of capacity, responsibilities and technological tools to assist with management. It is noted that the C&C benchmark process did not address this issue and as such, there is no reference point as to comparable team size, capacity and responsibilities for OAD in other FRSs.

Area for Consideration 3

We are confident that the OAD is performing but within its limitations as detailed. The Service should consider a review of the team size and responsibilities as well as use of automation and analysis tools to help improve ORL outputs.

47. A related issue raised on several occasions was the structural position of OA sitting within the SA Function and TSA Directorate. Staff referred to the preferred historic position whereby OA was a department within Ops Function within the SD Directorate. This could have been perceived as a recent change management issue, many staff articulated compelling arguments for and against being within both SA or Ops. The overriding sentiment was that OA outputs have on many occasions an impact on Ops and the extended links between the workforces made problem resolution less efficient and less effective.

48. In addition, there was also the perception that OA was inevitably subservient to Safety within a busy professional H&S department. As such, staff thought that prominent issues within OA maybe had less significance for management and therefore less impetus for resolution. Many staff kindly used the analogy of OA ‘living next to the noisy neighbour’ of safety, as a means to express their concerns. That concern being that if OA was so important to the organisation that it should not be perceived as subservient and should be championed, visible and have a higher profile.

49. Whilst the Service accept and understand these challenges, they believe that OA is correctly positioned within the organisational structure and that it provides a healthy degree of independence from the workloads of other Functions and Directorates. Additionally, TSA have recognised the need for strategic operational focus on OA within the Function and have recently reassigned a dedicated AC to lead the team.

Area for Consideration 4

Structural positioning of a department within the organisation is a management function. The Service should continually review whether the current structural position allows for OA to be given the appropriate focus, visibility and profile, whilst ensuring managers can resolve issues as efficiently and effectively as possible.

50. As detailed, the OAD has a number of responsibilities and consequential administrative tasks. One of the main tasks is operational incident debriefing, which can be scaled up depending on the size and type. To ensure a degree of quality assurance a series of task cards were developed which assists staff regarding consistency in actions, timescales, engagement, reporting and review. These task cards were born from the C&C benchmark analysis and are a positive addition to the OAD management as they provide targets and process for keeping inputs and outputs in a reasonable and consistent timeline.

Good Practice 2

The OA debrief tasks cards are a positive addition to the internal administrative procedures and the OAD should be commended for their innovation.