This case study explores Buckinghamshire's journey with Copilot, highlighting the positive outcomes, challenges encountered, and lessons learned.
Overview
Buckinghamshire Council stands out as a pioneer in adopting Microsoft Copilot. They were among the first councils to embrace this AI technology, granting them valuable experience and a chance to shape the technology's development. This case study has been collected via in-depth discussion with Tony Ellis (IT Director) and Peter Parfitt (Head of Digital and AI Programme Lead) who have led and overseen the council’s early adoption of Copilot. This case study explores Buckinghamshire's journey with Copilot, highlighting the positive outcomes, challenges encountered, and lessons learned.
Early adoption and strategic rollout
As a Copilot early adopter, Buckinghamshire had access to the AI tool in October 2023 before the technology had been generally released. The council strategically rolled out 300 Copilot licenses across various departments. Initially, the council set five key strategic objectives for the pilot of the technology:
- Enhanced staff productivity
- Improved work quality
- Achieved cost savings
- Ensured risk aversion
- Promoted staff well-being
Early adoption offered Buckinghamshire distinct advantages. Microsoft provided dedicated support, including training sessions that proved highly popular – attracting over 140 attendees despite being held on a Friday afternoon. However, being among the first councils to trial Copilot also presented challenges. As the council was at the forefront of implementation, some aspects of the technology as well as the associated advice and training were still under development and continue to be developed.
To fuel interest in their copilot trial, Buckinghamshire hosted an innovative "Dragon's Den" style event. Teams from nine different departments (including HR, IT and finance) pitched how Copilot could benefit and improve their work. This not only identified several promising use cases but also fostered excitement and buy-in across the organisation.
AI governance: A proactive approach
In a pre-emptive move, Buckinghamshire established an AI governance board in October 2023, before deploying Microsoft Copilot. This ensured senior management and members were engaged with AI initiatives from the outset. Chaired by a corporate director, the board oversees all AI endeavours, fostering transparency with residents and partners. It emphasises that AI acts as a supporting tool, not a final decision-maker.
The board leverages technical expertise across the organisation, including information governance colleagues. The board convenes as regularly as every fortnight, highlighting the speed of adoption within the council. Regular meetings showcase progress with Microsoft Copilot, highlighting successful use cases. Discussions delve into ongoing challenges associated with AI adoption, alongside a dedicated risk register to guarantee proper risk management and mitigation. Essential processes and procedures, including usage policies, Data Protection Impact Assessments (DPIAs), and Equality Impact Assessments (EQIAs), have also been implemented by the board to ensure safe AI deployment.
The AI governance board actively manages Copilot-related risks through a dedicated risk register process, incorporating Microsoft's six principles for responsible AI. Regular reviews and mitigation strategies ensure potential issues are monitored and addressed. Enhanced confidentiality agreements with specific staff were implemented to bolster data security. Clear communication emphasises that Copilot assists with tasks; final decisions and accountability remain human-centric. An attribution statement is currently being developed to clarify AI co-authorship within reports.
Training and support: Building capabilities
Buckinghamshire provided comprehensive support to staff. AI leads offered one-on-one training sessions on copilot’s application to existing systems, a champions scheme was established, and regular demonstrations showcased Copilot's use cases within different departments across the council. As formal Microsoft training wasn't available initially, Buckinghamshire fostered peer-to-peer learning, crucial for the initial rollout. The council are now refining training based on experience to prepare for wider adoption.
Measurable benefits: Time savings and beyond
Buckinghamshire prioritised measuring the benefits of Copilot. Following the initial rollout, they focused on quantifying the return on investment (ROI) to align with their financial plans. From the initial pilot of the technology, the council have seen several tangible benefits of utilising Copilot in key service areas.
- Customer service: Copilot is being utilised to transcribe and summarise calls, saving agents an average of 2 minutes per call wrap, equating to a 33 per cent reduction. This is significant considering there are more than 100 customer service agents who receive a high volume of calls every day.
- Executive assistants: Assistants using Copilot have reported a 20 per cent reduction in task completion time (60-90 minutes saved daily) for activities such as creating reports and meeting minutes. This data was capture from closely monitoring a group of 6 executive assistants.
- Project management: Copilot streamlines project initiation by capturing meeting transcripts and prompting population of project documentation templates within the IT department.
- Human resources: HR uses Copilot for sentiment analysis and job description/interview question writing. Copilot helps compare interview transcripts to job specs and pre-written questions, reducing advert creation time from hours to minutes.
- Children's social care: Copilot is being piloted to condense information from multiple documents into standardised education, health, and care plans.
An unexpected benefit of deploying Copilot in Buckinghamshire has been its positive impact on colleagues with neurodiversity. For example, the advanced transcription capabilities have significantly transformed the daily work of a deaf colleague. Unlike subtitles, transcription software offers a more comprehensive picture of conversations, allowing deaf participants to identify speakers, capture action items, and track unresolved issues. This has been particularly transformative, enabling the colleague to fully participate in meetings, review discussions later, and independently verify emails for accuracy, reducing reliance on others.
Challenges and considerations
Despite identifying numerous benefits, Buckinghamshire also encountered limitations and challenges.
- Adult social care: Initially, Copilot exhibited issues with "hallucinating" or fabricating information about patients. The council collaborated with Microsoft to develop specific prompts that addressed this problem, ensuring data fidelity.
- Inconsistency: The same question might yield different responses. Staff are adapting by understanding Copilot as a tool for generating helpful suggestions, not guaranteed consistency or accuracy.
- Data and Functionality Limitations: Copilot currently struggles with quantitative data and isn't fully integrated with Microsoft Excel. However, ongoing development suggests these limitations may be addressed in future updates.
Looking ahead: Building on the foundation
Buckinghamshire sees immense potential in Copilot beyond its current applications and are excited to continue their adoption journey as the technology develops. They are actively exploring new use cases as they continue their pilot.
To secure continued investment, the council is developing a more sophisticated method for calculating ROI to strengthen their business case before license renewal in November 2024. While quantifying the value of improved staff well-being can be challenging, the council recognises the importance of a holistic view of Copilot's impact.
In conclusion, Buckinghamshire’s pioneering approach to Microsoft Copilot adoption serves as a valuable case study. Their proactive measures in governance, training, and exploration provide valuable insight for councils looking to adopt Microsoft Copilot.
The Deputy Cabinet Member for Resources, Councillor Jonathan Waters said:
It is early days, but we are already seeing real results in areas like our customer contact centre, which deals with around 8,500 calls a week. Using AI is improving quality and complaints procedures, as well as streamlining response drafting and call summarising. We believe even bigger wins will follow when we begin to employ Copilot to enhance frontline services, such as social care and housing, which take up the bulk of our budget. If we can be more efficient, that's fantastic and if we can trim just 1 per cent off the cost of social care, that's where the real pressures are.
As AI continues to expand, it is vital that it is deployed ethically, and we have a dedicated AI Governance Board overseeing its use across council services. This is especially important in the public sector. Using AI promises a future where we can proactively identify areas for improvement before they become a drain on resources. I’m confident we will continue to see the benefits."