Call for views on the cyber security of AI – policy brief

The Department for Science, Innovation and Technology has published a call for views on the cyber security of artificial intelligence. 

The call for views is for input on government proposals to take forward a two-part intervention on the cyber security of AI. The planned intervention is to create a voluntary, principles-based Code of Practice that will then be used as the basis for the development of a global technical standard. The proposed voluntary code sets baseline security requirements for all AI technologies and distinguishes actions that need to be taken by different stakeholders across the AI supply chain. 

The LGA is committed to supporting local government in strengthening cyber security measures and adopting AI safely and responsibly. 

The LGA’s relevant consultation responses to date highlight concerns about cyber security risks associated with software generally and AI deployment:

• Data security: AI systems rely on multiple components, and the security of any one element can compromise the entire system. Users, including local governments, require transparency and assurance regarding suppliers' security practices.
• Software supply chain risks: Rapid innovation increases vulnerabilities and exposes software supply chains to exploitation. Lack of transparency in the software market hinders councils' ability to assess security risks.
• Regulatory priorities: The current regulatory environment appears to prioritise supplier interests over the security of public services and resident data.

The LGA proposes solutions to build trust in AI:

• Mandatory assurance frameworks: Establish mandatory assurance standards for AI suppliers, with external verification processes instead of self-assessment.
• Enhanced role of Public Buying Organisations (PBOs): Empower Public Buying Organisations to conduct centralised security assessments for local governments, saving resources and fostering consistency.
• Standardised security measures: Implement standardised cyber and information security practices within AI development and deployment, allowing for easier comparisons and reduced risks.

By addressing these concerns and implementing the proposed solutions in the proposed code of practice and global standard, DSIT can foster a secure and trustworthy environment for AI development and deployment within local government.

You can read the full responses below:

The use of AI in government

Software resilience and security

Large language models

DSIT has commissioned targeted research to establish an initial evidence base to inform the development of their policy interventions, notably the Code of Practice. The key findings from the research include:

• The vulnerabilities and threats across various AI technologies are broadly similar to each other.
• The exploitation of vulnerabilities in an AI system can have a substantial impact on end-users, such as the loss of sensitive data linked to consumers and employees as well as providing malicious actors with a way of breaching an organisation’s infrastructure.
• Organisations generally lack awareness and understanding of what security should be built into models and systems and whether practices/processes should be in place when adopting AI to protect their organisations.
• Key organisations, such as governments and those that develop standards advocate for security requirements for AI models and systems.
• Vulnerabilities found in AI systems can enable the models and systems to be weaponised which can result in cyber attacks and significant harm on users.
• The majority of research conducted in the field of the security of AI is being conducted by academic institutions. Out of the 415 sources on the cyber security of AI fully analysed by Queens University Belfast, only 28 per cent were created by industry organisations.

In the case of the voluntary code contained within this call for views, DSIT expects that relevant organisations should, at a minimum, also adhere to the provisions in both the Software and Cyber Governance Codes of Practice. While the Cyber Governance Code of Practice sets the baseline expectations for all organisations using digital technologies, the Software Code will also be relevant since software is an integral part of how AI models and systems function. Organisations deemed in scope of this code would also be expected to assess whether their circumstances warrant consideration of adherence to additional cyber codes published by the UK Government which may cover specific products or services relevant to them. 

The Code of Practice for the Cyber Security of Artificial Intelligence includes 12 Principles. These will apply to developers, system operators and data controllers, and they are all applicable to local government:

Secure Design: 

• Principle 1: Raise staff awareness of threats and risks.
• Principle 2: Design your system for security as well as functionality and performance.
• Principle 3: Model the threats to your system.
• Principle 4: Ensure decisions on user interactions are informed by AI-specific risks.

Secure Development:

• Principle 5: Identify, track and protect your assets.
• Principle 6: Secure your infrastructure.
• Principle 7: Secure your supply chain.
• Principle 8: Document your data, models and prompts.
• Principle 9: Conduct appropriate testing and evaluation.

Secure Deployment

• Principle 10: Communication and processes associated with end-users.

Secure Maintenance

• Principle 11: Maintain regular security updates for AI models and systems.
• Principle 12: Monitor your system’s behaviour.

This work on AI is closely linked to the Government’s recent publication on software resilience due to the inherent overlap between these technology areas. The Government previously held a call for views on software resilience and security for businesses and organisations in 2023. The feedback received on the software resilience call for views highlighted the need for a voluntary code of practice that set clear expectations concerning the cyber security responsibilities of software vendors.

Within the LGA’s AI state of the sector survey, AI security concerns and a lack of standards were reported as key barriers to AI adoption.

This DSIT consultation presents a valuable opportunity for local government to shape the future of AI security. Councils are increasingly exploring AI across services from waste management to adult social care. This requires a proportionate consideration of risk.

A code of practice would support the capability of councils to avail of the opportunities presented by AI technologies, strengthening assurance between councils as customers and suppliers. It would also provide guidance for councils developing in-house AI tools.

The voluntary nature of the code of practice raises concerns regarding adherence from suppliers. Given there are multiple voluntary codes (including the software code of practice), there isn’t clear incentivisation for suppliers to thoroughly adhere to any or what priority they should be taken in.

Local government needs must be effectively represented in the development of the voluntary code of practice and global standard. This will ensure effective protection of resident data, more secure AI-powered services, and inform confident procurement of AI technologies to minimise misuse and mitigate data breaches.

Read the full call for views.