MITRE Corporation · USA
A comprehensive approach to regulating AI to ensure security and mitigate risks. It categorizes AI into three domains: as a subsystem, as human augmentation, and with autonomous agency. Each category presents unique risks that require tailored regulatory measures. Recommendations include improving AI assurance, developing robust audit mechanisms, and focusing on the interplay between AI components in complex systems. The paper emphasizes the need for continuous regulatory analysis and the creation of frameworks that align AI development with public safety, while avoiding regulatory overreach that could stifle innovation.
mitre.org ↗
National Institute of Standards and Technology (NIST) · USA
This document presents a detailed taxonomy and standardized terminology for adversarial machine learning (AML), focusing on attacks and mitigation strategies for AI systems. It classifies attacks into categories based on learning methods, attacker goals, and stages of the machine learning lifecycle. The report emphasizes the importance of understanding adversarial risks across predictive and generative AI systems and outlines corresponding mitigation techniques. By providing a common language, the framework aids in securing AI systems against threats like evasion, data poisoning, and privacy violations. This entry points to the 2025 edition, NIST AI 100-2e2025, published in March 2025, which extends the 2023 taxonomy to generative AI, including attacks on large language models, retrieval-augmented generation and agents.
nvlpubs.nist.gov ↗
OWASP · International
An open standard for runtime control of AI agents: it defines the hooks an agent platform exposes so that separate tooling can inspect an agent's intended actions and permit, deny or modify them before they execute, with policy enforcement and an audit trail. Where the agentic threat guides describe what to defend against, this specifies where the enforcement point sits. The specification, at version 0.1.0, is written as JSON Schemas with normative prose and is developed in the open on GitHub, code under Apache 2.0 and documentation under CC BY-SA 4.0. It joined the OWASP GenAI Security Project on 1 September 2026.
genai.owasp.org ↗
OWASP · International
A proposed scheme for agents to discover and authenticate each other, addressing the question of how one agent knows which other agent it is talking to. Early work, published as version 1.0.
genai.owasp.org ↗
OWASP · International
Threat reference for agentic systems, working through what changes once a model can plan, remember and call tools: goal manipulation, misuse of delegated authority, and trust between agents. Paired mitigations for each.
genai.owasp.org ↗
MITRE Corporation · USA
The document presents a comprehensive framework for AI assurance, focusing on managing risks across the lifecycle of AI-enabled systems. The framework integrates risk management practices and emphasizes a repeatable engineering process to ensure AI systems operate effectively while managing safety, security, privacy, and other key trust factors. Through a detailed assurance plan, the process addresses system characterization, risk discovery, and evaluation, making it adaptable to various sectors. The approach leverages real-world pilot studies to highlight its applicability, and promotes sector-specific solutions for domain-specific outcomes.
mitre.org ↗
Cloud Security Alliance (CSA) · International
A control framework for AI systems in the same shape as the Cloud Controls Matrix: a list of controls with implementation guidance and mappings to other standards, meant for assessing an AI deployment rather than reasoning about it.
cloudsecurityalliance.org ↗
Department for Science, Innovation and Technology (DSIT) · UK
A voluntary code published on 31 January 2025 setting out baseline cyber security principles for the organisations that develop, deploy and operate AI systems. It runs across the lifecycle, from secure design and development to deployment, maintenance and end of life, and ships with an implementation guide. The department submitted it to ETSI, where it forms the basis of the technical specification TS 104 223 and the accompanying implementation guide TR 104 128.
gov.uk ↗
Cybersecurity and Infrastructure Security Agency (CISA), National Security Agency (NSA) and Federal Bureau of Investigation (FBI) · USA
A joint cybersecurity information sheet released on 22 May 2025 by CISA, the National Security Agency, the FBI and partner agencies in Australia, New Zealand and the United Kingdom. It sets out the risks that arise from data security and integrity problems across the AI lifecycle, from development and testing through deployment and operation, and the practices that address them: data protection measures, risk management, monitoring, threat detection and network defence. This CISA alert page links to the full PDF.
cisa.gov ↗
Coalition for Secure AI (CoSAI) · International
Incident response adapted to AI systems, from the Coalition for Secure AI. It sorts AI incidents into data, model, deployment, infrastructure and user-interaction incidents, then walks the familiar preparation, detection, containment and recovery, and post-incident phases with the AI-specific work added to each, including the split of responsibilities between a model provider and its consumer. Playbooks and a RACI for the roles involved are included. Version 1.0, dated 27 October 2025, from CoSAI's workstream on preparing defenders.
coalitionforsecureai.org ↗
Cloud Security Alliance (CSA) · International
Framework for managing risk at the model level, covering how a model is selected, validated, monitored and retired. Aimed at organisations answerable for a model's behaviour in production rather than at the people training it.
cloudsecurityalliance.org ↗
Microsoft · USA
This document provides a framework for organizations to assess the security posture of their AI systems, aligned with existing information security efforts and business objectives. It covers various areas related to AI systems, including machine learning security policies, data collection, data processing, model training, model deployment, system monitoring, incident management, and business continuity planning. The document aims to provide a comprehensive perspective on AI system security, outline threats to critical AI assets, and enable organizations to conduct AI security risk assessments.
learn.microsoft.com ↗
National Technical Committee 260 on Cybersecurity of Standardization Administration of China (SAC/TC260; 全国网络安全标准化技术委员会) and National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT/CC; 国家计算机网络应急技术处理协调中心) · China
China's national framework for AI safety governance, covering risk management across the AI lifecycle. It sorts risks into three classes: risks inherent in the technology, such as weaknesses in models, algorithms and data, risks arising when the technology is applied, including its abuse for cyberattacks and the output of harmful content, and derivative risks that wide adoption creates for society. Each class gets technical countermeasures, followed by fourteen broader governance measures and security guidelines for those who develop, deploy, operate and use AI applications. Version 2.0 was issued jointly by TC260 and CNCERT/CC in September 2025 and released on 15 September 2025, replacing the September 2024 version 1.0. The PDF carries the Chinese text followed by an English translation.
cac.gov.cn ↗
Federal Office for Information Security · Germany
This guide introduces key security concerns associated with AI systems, particularly focusing on attacks and defenses relevant to machine learning models. It outlines categories such as evasion attacks, information extraction, and backdoor attacks, emphasizing the importance of secure AI system design. With AI being used in critical sectors like healthcare and autonomous driving, the guide stresses the importance of defensive measures against malicious manipulation, especially when using pre-trained models or public datasets. The document aims to provide initial guidance for developers on securing AI systems without claiming to be exhaustive.
bsi.bund.de ↗
OWASP · International
A verification standard for AI applications, structured as testable requirements at several levels of rigour, in the same shape as the ASVS for web applications. Intended for stating what was verified rather than claiming a system is secure. CC-BY-SA-4.0.
github.com ↗
Coalition for Secure AI (CoSAI) · International
Sets out who is responsible for which security control when an AI system is assembled from several suppliers, in the manner of the cloud shared responsibility model. It names eight roles, from agentic platform, model and model-serving providers through platform provider, application developer and data provider to users and governance, and maps responsibilities across five layers and three operating models (AI software, platform and infrastructure as a service). Useful for vendor contracts and for finding the controls nobody has picked up. Version 1.0, dated 26 May 2026, from the Coalition for Secure AI.
coalitionforsecureai.org ↗
National Institute of Standards and Technology (NIST) · USA
NIST's voluntary framework for managing risk across the design, development, use and evaluation of AI systems, published on 26 January 2023. It is organised around four functions: govern, map, measure and manage. This is the base document that the generative AI profile (NIST AI 600-1) and other sector profiles build on, and NIST publishes a companion playbook, crosswalks to other standards, and a roadmap alongside it.
nvlpubs.nist.gov ↗
National Institute of Standards and Technology (NIST) · USA
This document is a cross-sectoral profile of the AI Risk Management Framework (AI RMF 1.0), specifically tailored for generative artificial intelligence (GAI). It builds upon the guidelines from the 2023 Executive Order on Safe, Secure, and Trustworthy AI, helping organizations manage the unique risks of GAI across its lifecycle. The framework offers insights on governance, testing, and risk management for GAI, focusing on emerging risks like confabulation, harmful bias, and privacy issues while providing strategies for organizations to mitigate these risks. A companion https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook also has been published by NIST along with an https://www.nist.gov/itl/ai-risk-management-framework/roadmap-nist-artificial-intelligence-risk-management-framework-ai, https://www.nist.gov/itl/ai-risk-management-framework/crosswalks-nist-artificial-intelligence-risk-management-framework, and various https://www.nist.gov/itl/ai-risk-management-framework/perspectives-about-nist-artificial-intelligence-risk-management. In addition, NIST is making available a https://www.nist.gov/video/introduction-nist-ai-risk-management-framework-ai-rmf-10-explainer-video about the AI RMF.
nvlpubs.nist.gov ↗
Cybersecurity and Infrastructure Security Agency (CISA) and Australian Signals Directorate's Australian Cyber Security Centre (ASD's ACSC) · USA
Joint guidance published on 1 May 2026 by CISA with the Australian Signals Directorate's Australian Cyber Security Centre and other international and US partners. It sets out the security challenges and risks that come with agentic AI systems and the steps for designing, deploying and operating them safely, with the stated aim of helping organizations align AI risk management with the cybersecurity frameworks they already run and strengthen oversight as agentic adoption grows.
cisa.gov ↗
Cyber Security Agency (CSA) · Singapore
The "Guidelines on Securing AI Systems," developed by the Cyber Security Agency of Singapore in October 2024, provides a comprehensive framework for securing AI systems across their lifecycle. These guidelines emphasize understanding AI-specific threats, conducting risk assessments, and applying security principles throughout AI development, deployment, and maintenance. Although non-mandatory, they aim to assist system owners in mitigating risks and implementing secure AI practices, alongside traditional cybersecurity measures. The document also serves as a foundation for further collaboration with the AI and cybersecurity community. The CSA also released a “Companion Guide on Securing AI Systems” to support AI system owners in securing their AI adoption across the systems’ lifecycle available at: https://www.csa.gov.sg/Tips-Resource/publications/2024/guidelines-on-securing-ai
csa.gov.sg ↗
National Institute of Standards and Technology (NIST) · USA
Applies the NIST Cybersecurity Framework 2.0 to AI, organised around three focus areas: securing AI systems, using AI for cyber defence, and building resilience against AI-enabled attacks. It complements the AI Risk Management Framework rather than replacing it. Published as an initial preliminary draft on 16 December 2025; the comment period closed on 30 January 2026 and NIST plans an initial public draft during 2026.
csrc.nist.gov ↗
European Union Agency for Cybersecurity (ENISA) · EU
A position note published on 7 July 2026 on what frontier AI models change for cybersecurity in the EU. In the agency's own words it gives national competent authorities in member states, EU policymakers, defenders and service providers "an initial set of recommendations to support them in their respective roles towards developing the necessary operational capabilities to face machine-speed threats". It is a short strategic document, so expect direction and priorities here and controls in ENISA's framework publications.
enisa.europa.eu ↗
Coalition for Secure AI (CoSAI) · International
A map of the AI supply chain and what can go wrong along it, from the Coalition for Secure AI. It separates model generation (training data, training, infrastructure) from model integration and consumption (data, model, application, infrastructure), lists the threats at each stage, such as data poisoning and weaknesses in model training, and pairs them with existing mitigations and the documentation a consumer should expect from a supplier. It closes with separate takeaways for executives, security practitioners and ML engineers. Version 1.0, dated 12 June 2025.
coalitionforsecureai.org ↗
OWASP · International
Published by the OWASP Gen AI Security Project on 22 January 2025, this guide describes what red teaming a generative AI system involves in practice. It covers model evaluation, implementation testing, infrastructure assessment and runtime behaviour analysis, and treats red teaming as a programme rather than a one-off exercise. The PDF downloads without registration.
genai.owasp.org ↗
OWASP · International
A method for connecting observed threats to the defences that address them, so a programme can be described in terms of coverage rather than a list of tools. Released as version 1.0.
genai.owasp.org ↗
National Cyber Security Centre (NCSC) · UK
Guidelines covering the whole lifecycle, from secure design through development, deployment and operation. Published by the UK NCSC with CISA and co-sealed by agencies from a further sixteen countries, which makes it one of the few pieces of AI security guidance with broad international sign-off.
ncsc.gov.uk ↗
UK AI Safety Institute · UK
Inspect,a framework for large language model evaluations created by the UK AI Safety Institute, provides many built-in components, including facilities for prompt engineering, tool usage, multi-turn dialog, and model graded evaluations. Extensions to Inspect (e.g. to support new elicitation and scoring techniques) can be provided by other Python packages.
inspect.ai-safety-institute.org.uk ↗
International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) · International
The vocabulary standard the other ISO AI documents build on, fixing what terms such as machine learning, model and AI system mean. Useful mainly when a contract or policy needs words that are defined somewhere other than a vendor's marketing. Published 2022.
iso.org ↗
International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) · International
Guidance on applying risk management to AI, mapping the general ISO 31000 approach onto AI-specific sources of risk. Guidance rather than requirements, so it is read alongside 42001 rather than certified against. Published 2023.
iso.org ↗
International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) · International
The certifiable management-system standard for AI, in the same family as ISO/IEC 27001 for information security. It specifies how an organisation establishes, runs and improves an AI management system rather than how any model is built, which makes it the document auditors and procurement ask for. Published 2023.
iso.org ↗
ISO · International
This document (still in draft status) offers guidance to organizations on managing security threats and failures in AI systems. It provides insights into the consequences of such threats throughout the AI lifecycle and offers strategies for detection and mitigation. The guidance is applicable to organizations of all sizes and sectors, including public and private companies, government agencies, and non-profits, that are involved in the development or use of AI systems.
iso.org ↗
OWASP · International
Practices for the data side of generative AI: what goes into training and retrieval, how it is classified and retained, and where it can leak back out through a model. The half of AI security that is a data-governance problem.
genai.owasp.org ↗
OWASP · International
A checklist for organisations adopting LLM applications, covering governance, legal and regulatory questions as well as technical controls. Written for the people who have to sign off on a deployment.
genai.owasp.org ↗
Cloud Security Alliance (CSA) · International
A taxonomy of threats specific to large language models, giving named categories to sort findings into. Sits alongside the NIST adversarial-ML taxonomy already listed here, from an industry rather than a standards-body perspective.
cloudsecurityalliance.org ↗
National Institute of Standards and Technology (NIST) · USA
Guidelines for improving the safety and security of dual-use foundation models, focusing on mitigating the risks of deliberate misuse. It addresses the evolving threats posed by foundation models, such as their potential to aid in the development of harmful technologies, cyberattacks, and other dangerous outputs. The guidelines emphasize managing misuse risks across the AI lifecycle, offering best practices for developers and organizations involved in foundation model development. The recommendations build upon the NIST AI Risk Management Framework, focusing on both technical and social factors influencing misuse. This entry points to the second public draft, released on 15 January 2025, which NIST describes as an expanded version of the July 2024 initial draft, revised after public comment. No final version had been published as of October 2026.
nvlpubs.nist.gov ↗
MITRE Corporation · USA
MITRE's ATLAS is a knowledge base of adversarial tactics and techniques targeting AI systems, derived from real-world attacks and demonstrations by red teams. With the increasing adoption of AI, ATLAS highlights the unique risks, expanding attack surfaces, and vulnerabilities that traditional cyber defenses may not cover. Modeled after MITRE's ATT&CK framework, it provides valuable insights for security analysts, AI developers, and others involved in AI security, enabling effective threat assessments and red teaming. ATLAS also facilitates global collaboration through secure incident reporting and open-source tools to enhance AI threat mitigation strategies.
atlas.mitre.org ↗
Coalition for Secure AI (CoSAI) · International
A threat model and control set for systems built on the Model Context Protocol, from the Coalition for Secure AI's workstream on secure design patterns for agentic systems. It separates threats specific to MCP from general threats that MCP changes, then sets out controls covering agent identity, delegation and access control, input sanitisation, sandboxing, cryptographic verification of resources, transport security, human approval and logging. It also defines security assurance profiles, stating which controls each assurance level requires and mapping them to common deployment patterns. Version 2.0, dated 12 August 2026 and still marked as a draft.
coalitionforsecureai.org ↗
European Union Agency for Cybersecurity (ENISA) · EU
A scalable, three-layer framework for cybersecurity practices in AI systems, addressing both general ICT security and AI-specific risks. Layer I covers foundational cybersecurity practices for ICT infrastructures hosting AI systems. Layer II introduces AI-specific security controls, addressing dynamic risks throughout AI’s lifecycle. Layer III provides sector-specific practices for high-risk AI applications, such as in health and energy. The framework integrates existing standards, legislation, and cybersecurity measures, aiming to build trust and enhance the security and resilience of AI systems across various sectors.
enisa.europa.eu ↗
MITRE Corporation · USA
OCCULT is a framework designed to quantify the risks of LLMs when used in offensive cyber operations. Instead of red-teaming the model to see how it might fail, OCCULT is about assessing how an LLM itself could be weaponized by attackers or used as a “cyber agent.”
arxiv.org ↗
OWASP · International
An open reference covering AI security threats, the controls that address them and the practices around both, written by a group of more than 170 contributors and published as a browsable site and a single PDF. It runs to over 300 pages and feeds into the standards work behind the EU AI Act and ISO, with mappings to other frameworks. Free, no registration, and the source repository was last updated on 31 August 2026.
owaspai.org ↗
OWASP · International
A testing methodology for AI and LLM systems, with repeatable test cases spanning the application, model, infrastructure and data layers. It covers adversarial manipulation, information leakage, data poisoning, bias, hallucination, unsafe autonomy and model degradation. Version 1 was published on 26 November 2025 and is available as a PDF and as browsable documentation. Led by Matteo Meucci and Marco Morana.
owasp.org ↗
OWASP · International
The OWASP Gen AI Security Project's list of the ten most critical security risks in applications built on large language models, with mitigations for each and mappings to NIST, MITRE ATLAS and the CSA AI Controls Matrix. The 2026 edition was published on 4 August 2026 and is the current one: prompt injection and sensitive information disclosure hold the first two places, excessive agency moves up to third, and hidden context exposure replaces system prompt leakage. The PDF downloads without registration. The older owasp.org project page is now kept as a historical archive and states that "active development has moved to the OWASP GenAI Security Project repository".
genai.owasp.org ↗
OWASP · International
The Top 10 for machine learning systems generally, covering input manipulation, data poisoning, model inversion, membership inference and supply-chain attacks. Distinct from the LLM Top 10 also listed here: this one is about ML systems rather than language-model applications.
owasp.org ↗
OWASP · International
A companion list to the LLM Top 10, covering the risks that appear once a model plans, holds memory, calls tools and acts with delegated authority. The ten categories run from agent goal hijack and tool misuse through identity and privilege abuse, memory and context poisoning, insecure inter-agent communication and cascading agent failures, each mapped to reported incidents and to defences across the development lifecycle. The 2026 edition was published on 9 December 2025 and downloads without registration.
genai.owasp.org ↗
Coalition for Secure AI (CoSAI) · International
A model-agnostic set of security rules and skills for AI coding agents, from the Coalition for Secure AI, an OASIS open project. It puts secure-by-default practice in a form an agent consumes directly, so the guidance travels with the tooling instead of sitting in a document beside it.
github.com ↗
Japan AI Safety Institute · Japan
The “Guide to Red Teaming Methodology on AI Safety” is intended to help developers and providers of AI systems to evaluate the basic considerations and implementation points of red teaming methodologies for AI systems from the viewpoint of attackers (those who intend to abuse or destroy AI systems). This guide was prepared based on domestic and international studies and precedents, taking international alignment into account. It summarizes the issues considered important when conducting red teaming.
aisi.go.jp ↗
Department of State · USA
This guide provides a framework for integrating human rights into the design, development, and governance of AI systems. It emphasizes the potential benefits of AI, while acknowledging risks such as bias and misuse for surveillance. The guide builds on the NIST AI Risk Management Framework, outlining how organizations can use it to respect international human rights throughout the AI lifecycle. It also recommends practices for risk assessment and mitigation, helping stakeholders across sectors incorporate human rights into their AI processes. The document now sits on the State Department's 2021-2025 archive site; the address on the main state.gov site returns 404.
2021-2025.state.gov ↗
Department of Homeland Security · USA
The framework outlines the responsibilities for ensuring the safe and secure development of AI in critical U.S. infrastructure. It provides recommendations for five key roles: cloud infrastructure providers, AI developers, critical infrastructure operators, civil society, and the public sector. Addressing risks like cyber threats and operational failures, it advocates for voluntary responsibilities, transparency, and collaboration to enhance AI safety and security while safeguarding civil rights and liberties.
dhs.gov ↗
Google · USA
Google's public framework for AI security, written as a practitioner's guide. The site carries an interactive map of AI development seen through a security lens, a risk framework covering 15 risks inherent to AI development together with the controls that address them, and a self-assessment questionnaire that returns a list of relevant controls. SAIF 2.0 extends the material to agent security.
saif.google ↗
National Institute of Standards and Technology (NIST) · USA
This document augments the https://csrc.nist.gov/projects/ssdf by providing specific practices and recommendations for developing secure generative AI and dual-use foundation models. It focuses on mitigating risks associated with AI model vulnerabilities throughout the software development lifecycle. The guidelines aim to assist AI model producers, AI system developers, and acquirers in applying secure practices, ensuring that models are protected from unauthorized access, tampering, and other potential risks. These additions address the evolving needs outlined in Executive Order 14110 regarding safe, secure AI development.
nvlpubs.nist.gov ↗
RAND Corporation · USA
This report focuses on protecting the weights of frontier artificial intelligence models - those at the cutting edge of current capabilities - from theft and misuse. It highlights the critical role of model weights, which encode core AI capabilities derived from extensive data and computational resources. The report identifies 38 distinct attack vectors and estimates their feasibility for various adversaries, ranging from cybercriminals to state actors. Recommendations include prioritizing access control, defense-in-depth strategies, and insider threat prevention. The report aims to aid AI organizations in updating security models and guide policymakers on AI security practices.
rand.org ↗
European Union Agency for Cybersecurity (ENISA) · EU
This report by ENISA provides a comprehensive analysis of security challenges related to machine learning algorithms. It highlights emerging threats such as adversarial attacks, data poisoning, and model extraction. The report emphasizes the need for tailored security controls beyond traditional information system measures to address vulnerabilities specific to machine learning. It also offers a taxonomy of algorithms, maps threats to vulnerabilities, and suggests security controls from standards like ISO 27001 and NIST. Recommendations include raising cybersecurity awareness and developing benchmarks for security controls in machine learning.
op.europa.eu ↗
Coalition for Secure AI (CoSAI) · International
A short paper on model signing as the basis for verifying where a model came from and that it has not been altered. It explains what makes signing a model different from signing ordinary software, sets out what model producers and model consumers each need from it, and proposes maturity levels for adoption. It closes with NVIDIA's work on the OpenSSF Model Signing standard, which supports both conventional PKI and keyless signing with Sigstore. A follow-on to the same workstream's paper on AI supply chain risks. Approved by the CoSAI governing board on 29 September 2025.
coalitionforsecureai.org ↗
