In the latest Code Story bonus episode, host Noah Labhart speaks with Ave Gatton, Director of Generative AI at Protegrity, about a security reality many teams overlook: AI safety doesn’t end with training — it begins at inference. The episode digs into how prompt injection, data leakage, and manipulation show up during live usage, and what it takes to build practical guardrails that operate in real time without slowing AI adoption to a crawl.
What’s in the piece
- Inference-time threats, defined: The discussion breaks down what “inference-time risk” means and why it’s becoming a primary focus as GenAI moves into production.
- How inference-time differs from training-time: The episode contrasts model training concerns with the risks that emerge when models interact with live prompts, users, and connected systems.
- Why traditional security models fall short: It explores why perimeter controls and static defenses struggle when AI behavior is dynamic and context-driven.
- Compliance and guardrails: The conversation highlights how compliance expectations influence runtime controls and what “good” looks like for scalable deployment.
- Practical steps teams can take now: Ave shares actionable guidance for securing inference today — including how to balance performance and protection when adding guardrails.
Why it matters
As organizations operationalize GenAI, the biggest failures often happen during real-world use — not in model development. Inference is where prompts, tools, and data collide, and where “helpful” model behavior can turn into exposure if guardrails aren’t designed for runtime reality. If teams want safe, scalable AI adoption, they need controls that work where the risk actually shows up: in production, in real time, at the point of data consumption.
Key shifts highlighted
- From training-time safety → inference-time safety: The episode reframes security as a runtime discipline, not a one-time model milestone.
- From static controls → real-time guardrails: Protection needs to adapt to live prompts, evolving manipulation patterns, and changing context.
- From “secure the model” → “secure the interaction surface”: The risk lives in prompts, connected tools, and sensitive data access — not just weights and training data.
- From “policy stated” → “policy enforced”: Effective guardrails translate intent into controls that hold up under real-world usage and scrutiny.
Protegrity POV (from the piece)
Ave Gatton emphasizes that inference is where organizations either keep control — or lose it. If teams want to scale GenAI safely, they need guardrails that operate at runtime, protect sensitive data during consumption, and reduce the blast radius of prompt-driven manipulation and leakage. The message is practical: build AI that can move fast, but don’t let “fast” become “unguarded.”
How Protegrity helps
- Protect data at the point of use: Apply fine-grain controls that help reduce exposure when AI systems access and return sensitive information.
- Support governed AI adoption: Help teams enforce guardrails that align AI usage with security and compliance expectations as deployments scale.
- Reduce inference-time risk: Add practical, real-time protections designed to address leakage, manipulation, and misuse during live interactions.
Key takeaways
- Inference is the new frontline: Runtime threats like injection and leakage require guardrails that work during live usage, not just during training.
- Safe scale requires real controls: Teams need practical, performance-aware protections that keep sensitive data governed as AI moves into production.
Note: This page summarizes a third-party podcast episode for convenience. For the complete context, please refer to the original source below.