Securing LLM Applications and AI Agents

The engineer's job and the principle behind the course, how an LLM application is built, three kinds of failure, testing safely, the lab and its evidence, and the review, prove, fix and verify loop run once. Start here.

Threat-modeling the service-desk agent before it goes live: its design read from the code, trust boundaries, abuse paths from writer to harm, coverage against OWASP, ATLAS and ATT&CK, the threat-model board, and a ranked, written threat model.

How a customer's ticket steers the service-desk agent, the forms injection takes, attacks that blend in, the layers that lower the rate and how to measure them, and the call check that holds, built and verified in the lab.

How a shared knowledge assistant hands one tenant's documents to another, the confused deputy behind it, where the permission check belongs, the check and the cache key built in the lab, and the result verified as evidence.

Who can put text into the knowledge assistant's index: the write paths, poisoning them, the editor, tenant and second-approval gates, provenance and removing every copy, and the boundary verified as evidence.

What happens to the model's output after it is written: rendering replies safely, generated URLs, queries and paths, schema and semantic validation, and output handling verified as evidence.

Whose authority the agent uses when it calls an API: application versus delegated access, the confused deputy, one scope and one record per call, tokens as credentials, and the identity boundary verified as evidence.

Securing an agent's tools over MCP: a token per server, no passthrough, audience checks at the server you run, tool definitions approved by hash, and the integration verified as evidence.

Securing what an agent does: approvals bound to the exact action and shown from its arguments, destinations matched against a registry, a budget per request, an idempotency key, and the approval matrix.

Securing what an agent remembers and who it listens to: provenance on every memory write, recall scoped by customer and expiry by kind, handoffs accepted only from known agents for their own field, and agent-to-agent trust by role.

Securing what an agent loads and the updates proposed for it: files verified against their publisher's record, formats that run nothing on load, a model swap gated on every earlier case, backdoors and what holds them, and connector and configuration changes reviewed by reach.

Turning security checks into an evaluation that can decide a release: six measures with their conditions, enough runs for the claim, judging by effects, guardrails measured beside controls, a suite grown with variants, and a gate for every change.

Investigating and responding to incidents in LLM applications and agents: what to record, scoping across services by trace, containing without destroying evidence, deciding what is an incident, and verifying containment by its cause.

One sheet per area of LLM application and agent security: the question, the control or check that settles it, how to use it and what it does not prove.

Eight runbooks for the controls an LLM application or agent needs, each built in the lab and tested from both sides.

Installing the SEC502 lab, the ai-app workspace the console queries, and checking both work before the first exercise.

Three cases worked end to end across the modules, including the wrong turns, each ending in a scorecard, a report card and the point that transfers.

Eight playbooks, one per thing that lands on an LLM application team: what to check first, how to contain it, and what the record must say.

The range to practice on freely, and every tool scenario, predict block and console query in the course, by module.

The standards, lists, protocols and tools the course relies on, each with the version it was read at.

Securing LLM Applications and AI Agents end-of-course exam: 40 questions in four parts, from threat modeling to incident response, ending on a case study.