Building Secure and Responsible LLM Applications Training Course

Overview of LLM Architecture and Attack Surface

• Understanding how LLMs are built, deployed, and accessed via APIs.
• Key components within LLM application stacks (e.g., prompts, agents, memory, APIs).
• Identifying where and how security issues manifest in real-world scenarios.

Prompt Injection and Jailbreak Attacks

• Defining prompt injection and understanding its dangers.
• Exploring direct and indirect prompt injection scenarios.
• Examining jailbreaking techniques used to bypass safety filters.
• Strategies for detection and mitigation.

Data Leakage and Privacy Risks

• Risks of accidental data exposure through model responses.
• PII leaks and the misuse of model memory.
• Designing privacy-conscious prompts and implementing retrieval-augmented generation (RAG).

LLM Output Filtering and Guarding

• Utilizing Guardrails AI for content filtering and validation.
• Defining output schemas and constraints.
• Monitoring and logging unsafe outputs.

Human-in-the-Loop and Workflow Approaches

• Determining where and when to introduce human oversight.
• Implementing approval queues, scoring thresholds, and fallback handling.
• Calibrating trust and leveraging explainability.

Secure LLM App Design Patterns

• Applying least privilege and sandboxing for API calls and agents.
• Managing rate limiting, throttling, and abuse detection.
• Ensuring robust chaining with LangChain and maintaining prompt isolation.

Compliance, Logging, and Governance

• Ensuring the auditability of LLM outputs.
• Maintaining traceability and managing prompt/version control.
• Aligning with internal security policies and regulatory requirements.

Summary and Next Steps