I am not affiliated with, endorsed by, or directly connected to any of these courses, and their inclusion is not an endorsement of their quality or effectiveness. Do your own research and due diligence before enrolling.
A job role path on Hack The Box Academy built in collaboration with Google and mapped to Google's Secure AI Framework. Twelve modules and 230 sections covering prompt injection, jailbreaks, adversarial machine learning, model privacy attacks, supply chain risk and deployment threats, worked through hands-on in the platform's labs. The curriculum outline is public; working through the modules needs an HTB Academy account. Rated hard.
A limited video series from Microsoft Learn on red teaming generative AI, released in July 2025 and built from case studies and the tooling Microsoft's own AI red team uses. The videos are keyed to the AI Red Teaming Playground Labs challenges listed under Code and PoCs, so the series can be watched on its own or worked through against those labs.
"AI Security Essentials" is an online course designed to deepen understanding of AI system security, covering threat modeling, anomaly detection, ethical considerations, and best practices in AI security. It offers practical exercises, case studies, and real-life examples to equip security professionals, business professionals, and students with the skills to protect AI models, datasets, and infrastructure, promoting trust in AI technology.
A two-day course for cybersecurity professionals covering how AI can be exploited, the threat vectors and risk analysis involved, and how AI-driven defences are designed, implemented and evaluated. The Security Academy that ran it was acquired by DNV, so the course is now delivered by DNV as tutored online or classroom training. The exam is supervised by SECO-Institute, which awards the S-AISF certification. Basic knowledge of IT security is recommended.
In "Artificial Intelligence for Cybersecurity," Sam Sehgal teaches how to apply AI to tackle complex cybersecurity challenges effectively, with proper preparation and safeguards. He clarifies AI's definition, its application in cybersecurity, and outlines the distinctions between discriminative and generative AI. The course emphasizes the critical cybersecurity principles of confidentiality, integrity, and availability, discusses existing cybersecurity gaps, and explores practical solutions using AI and machine learning to enhance security measures.
This course is designed to address the unique risks introduced by AI and Machine Learning, focusing on governance, cyber-security frameworks, and risk management without requiring prior technical knowledge. It targets risk management professionals, cyber-security experts, and AI professionals, offering insights into AI risks, creating governance frameworks, addressing cyber-security risks in AI systems, implementing security controls throughout the Machine Learning lifecycle, and leveraging ChatGPT for enhanced security processes.
Tonex programme for leaders overseeing AI security work, covering governance frameworks, policy enforcement and building security practice around AI initiatives. Instructor-led online and scheduled on request, listed in the CISA NICCS catalogue.
Self-paced course from HYSN Technologies (Practical DevSecOps) on the risks in the AI supply chain. Covers adversarial machine learning, data poisoning, model inversion and evasion attacks, securing data pipelines and model integrity, and maps risks against MITRE ATLAS. Includes hands-on labs, differential privacy and federated learning. Listed in the CISA NICCS catalogue.
Tonex programme for those leading an AI security strategy, covering risk governance, compliance, AI cyber threats and organisational security planning at leadership level. Instructor-led online and scheduled on request, listed in the CISA NICCS catalogue.
A deliberately vulnerable ReAct agent built with LangChain, for experimenting with Thought/Action/Observation injection, which targets an agent's reasoning steps rather than its final answer. Adapted from a challenge WithSecure ran at the BSides London 2023 capture the flag, and accompanied by the original write-up and video. Last updated June 2025.
A deliberately vulnerable MCP server carrying ten challenges of increasing difficulty, each showing a different way an MCP implementation can be attacked. Aimed at people shipping or reviewing MCP servers who want to meet the failure modes on a target where breaking things is the point.
An OWASP Gen AI Security Project capture the flag built on a simulated financial services application, covering vulnerabilities specific to agentic systems. The setting is the useful part: these are the failure modes that appear once an agent holds tools and acts on data it did not author.
This course dives into Generative AI and LLMs in cybersecurity, offering an understanding of their architecture, technology, and security implications. With hands-on experience using open-source models for cyber defense, participants learn to identify and mitigate AI-related security risks. It requires knowledge of cybersecurity, AI/ML principles, and programming in Python. The course covers LLM architecture, technology stacks, security considerations, and practical applications, empowering learners to enhance their organization's cyber resilience.
This learning path, comprised of 4 courses spanning 2 hours, delves into how Generative AI can revolutionize cybersecurity. It covers the integration of Generative AI in security operations, addressing data privacy, security risks, and how to harness Generative AI's potential effectively. The courses explore the balance between enhancing security measures and the inherent risks of Generative AI, offering insights into leveraging AI for both offensive and defensive cybersecurity operations.
The Generative AI in Cybersecurity Certification by GSDC is designed for cybersecurity professionals to master the use of Generative AI in detecting and mitigating cyber threats. It covers generative models, ethical considerations, and AI application skills, aiming to enhance digital defenses and provide a competitive edge in cybersecurity. The program requires programming knowledge, involves a 60-minute exam, and offers a deep dive into Generative AI fundamentals, cybersecurity basics, GANs, VAEs, and ethical considerations in AI.
A self-hosted training platform covering the OWASP Top 10 for LLM applications, with a deliberately vulnerable scenario for each item, run locally through Docker. Every exercise is meant to be exploited and then defended, which makes it a practical counterpart to the OWASP list rather than a reading of it.
The objective of this course is to provide students with the fundamental elements of machine learning security in the context of different application domains. The main concepts and methods of adversarial machine learning are presented, from threat modeling to attacks and defenses, as well as basic methods to properly evaluate adversarial robustness of a machine learning model against different attacks.
SANS course introducing generative AI and LLM security for practitioners with no prior AI experience. Covers tokenisation and attack surface, prompt injection, jailbreaking, RAG manipulation and MCP security, and applies the OWASP Top 10 for LLMs, MITRE ATLAS and the NIST AI RMF through hands-on labs.
SANS self-paced course on building and securing retrieval-augmented generation systems, covering contextual and agentic RAG architectures and the security considerations specific to each.
The course "Security Basics for Artificial Intelligence Software and Services" provides an overview at securing AI technologies. It covers ethical issues, common threats, and practical security strategies like secure coding and vulnerability assessments. Aimed at developers and security professionals, it equips participants with the skills to protect AI systems against emerging threats, ensuring their reliability and integrity.
This course, led by Diana Kelley, explores the trust we place in AI and ML technologies despite their vulnerabilities to attacks and failures. Kelley uses real-world examples to demonstrate how ML and AI can falter, offering guidance on creating resilient systems. She covers both intentional attacks and unintentional failures, highlighting security and privacy risks. The course provides strategies for enhancing ML robustness, including dataset hygiene, adversarial training, and secure API access.
Free interactive labs on attacking applications that have a language model wired into them. It works through prompt injection, mapping the API and plugin surface a model has been given, insecure output handling, indirect injection through content the model reads, training data poisoning and leaking sensitive training data, then closes on defences. Each topic ends in a hosted lab solved in the browser with nothing to install, and a later section covers weaknesses in AI-powered scanners themselves.
