Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Models for Agentic AI

• Categories of agentic threats, including misuse, privilege escalation, data leakage, and supply-chain risks
• Adversary profiles and the specific capabilities of attackers targeting autonomous agents
• Identifying assets, trust boundaries, and critical control points within agent architectures

Governance, Policy, and Risk Management

• Governance frameworks for agentic systems, covering roles, responsibilities, and approval gates
• Policy design focusing on acceptable use, escalation rules, data handling, and auditability
• Compliance requirements and strategies for collecting evidence for audits

Non-Human Identity & Authentication for Agents

• Creating identities for agents using service accounts, JWTs, and short-lived credentials
• Implementing least-privilege access patterns and just-in-time credentialing
• Managing the identity lifecycle, including rotation, delegation, and revocation strategies

Access Controls, Secrets, and Data Protection

• Applying fine-grained access control models and capability-based patterns for agents
• Managing secrets, encryption in transit and at rest, and enforcing data minimization
• Safeguarding sensitive knowledge sources and PII from unauthorized agent access

Observability, Auditing, and Incident Response

• Developing telemetry for agent behavior, including intent tracing, command logs, and provenance
• Integrating with SIEM, setting alerting thresholds, and preparing for forensic analysis
• Creating runbooks and playbooks for handling agent-related incidents and containment

Red-Teaming Agentic Systems

• Planning red-team exercises, defining scope, rules of engagement, and safe failover mechanisms
• Adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse
• Executing controlled attacks to measure exposure and potential impact

Hardening and Mitigations

• Engineering controls including response throttles, capability gating, and sandboxing
• Policy and orchestration controls, such as approval flows, human-in-the-loop checks, and governance hooks
• Model and prompt-level defenses, covering input validation, canonicalization, and output filters

Operationalizing Safe Agent Deployments

• Deployment strategies including staging, canary releases, and progressive rollouts for agents
• Managing change control, testing pipelines, and pre-deployment safety checks
• Cross-functional governance involving security, legal, product, and operations teams

Capstone: Red-Team / Blue-Team Exercise

• Conducting a simulated red-team attack against a sandboxed agent environment
• Defending, detecting, and remediating as the blue team using established controls and telemetry
• Presenting findings, a remediation plan, and necessary policy updates

Summary and Next Steps