Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Using EXO_OFFLINE to prevent runtime internet access
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors
- Verifying model weight integrity using SHA-256 checksums and signed model cards
- Running EXO in air-gapped networks without dependencies on HuggingFace
Dashboard and API Access Control
- Installing and configuring reverse proxies (nginx, Caddy) with TLS termination
- Implementing role-based access control for the EXO dashboard and REST API
- Storing secrets for API authentication using macOS keychain or Linux pass
- Restricting administrative endpoints to specific source IP ranges
Cluster Isolation and Network Security
- Segmenting EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports
- Preventing unauthorized device discovery and rogue node injection
- Encrypting libp2p traffic between nodes when RDMA is unavailable
Model Governance and Provenance
- Building an internal model registry with approved model lists and metadata
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside source checkpoints
- Enforcing loading of only specific HuggingFace repos or internal artifacts
- Documenting model lineage, license terms, and acceptable use policies
Audit Logging and Compliance
- Configuring EXO log forwarding to immutable audit trails (SIEM, WORM storage)
- Correlating API call logs with user identity and timestamp
- Capturing events for model instance creation, deletion, and inference requests
- Generating periodic compliance reports for internal and external auditors
Threat Modeling and Incident Response
- Identifying threats such as data exfiltration via model outputs, prompt injection, and side-channel leaks
- Implementing prompt monitoring and content filtering pipelines
- Creating incident response runbooks for cluster compromise scenarios
- Isolating affected nodes, preserving forensic logs, and rebuilding clean environments
Physical Security and Hardware Boundaries
- Securing Thunderbolt ports against unauthorized RDMA cable connections
- Utilizing secure enclaves and Apple Silicon hardware attestation where applicable
- Controlling physical access to clustered Macs and shared storage
- Documenting hardware lifecycle and decommissioning procedures
Regulatory Considerations
- Mapping EXO deployments to GDPR, HIPAA, and SOC 2 requirements
- Maintaining data residency by keeping inference on-premise
- Documenting vendor supply-chain risks (MLX, EXO, model weights)
- Preparing for AI governance frameworks such as EU AI Act Article 53
Requirements
- Experience with EXO or other local LLM runtimes
- Understanding of Unix filesystem permissions and networking ACLs
- Familiarity with TLS/SSL certificate management and encryption fundamentals
Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators managing sensitive data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
