2025-09-13 –, Track 2
Building AI agents requires more than prompt engineering - it demands thoughtful architectural decisions. This talk explores the system design choices that make or break production AI. This talk navigates the key decisions that we faced at HappyFox: choosing between workflows and multi-agent systems, engineering context for reliability, and building your agent toolbox right. We'll tackle the hard problems - maintaining systems across model changes, implementing security without breaking functionality, and handling privacy in production - covering the full stack of production AI architecture.
At HappyFox, we've spent the last two years building AI apps that serve thousands of users daily. Along the way, we've discovered that the real challenges aren't just in crafting clever prompts or choosing the latest model - they're in the architectural decisions and implementation details that determine whether your app survives contact with reality. This talk pulls back the curtain on what it actually takes to ship AI apps to production.
Outline
System Design Considerations
- Product Design / UX considerations
- Source attribution for trust
- Making retry actions seamless
- Feedback loops for continuous improvement
- Workflows vs Multi-Agents
- When modularization beats monolithic prompts
- Building pausable/resumable workflows
- Human-in-the-loop integration patterns
- Context engineering
- Truncation vs Compression vs Isolation strategies
- RAG pipeline design for dynamic context
- Managing conversation history effectively
- Your toolbox for an agent - Prompting techniques, Structured outputs & Tool design
- Extracting structured outputs from your agents
- Self-healing tool calls with error handling
- Building AI-friendly tool interfaces
Implementation / Practical Challenges
- Maintainability
- Version control for prompts with A/B testing capabilities
- Building eval suites that mirror production scenarios
- Treating LLMs as interchangeable components
- Fallback strategies and resilience patterns
- Security
- Action-selector and Map-Reduce patterns for injection defense
- Baking RBAC into tools, not prompts, and avoiding data exfiltration
- Syntactic, content safety, and factuality guardrails
- Privacy / Redaction
- PII/PHI handling: Redaction vs Masking vs Anonymization
- Maintaining response quality while preserving privacy
- Real-world examples and tradeoffs
Looking ahead - Future outlook: upcoming developments and implementation opportunities
None
Prerequisites –- Familiarity with python
- Basic understanding of LLM’s, RAG and AI Agents
Intermediate
