PyCon India 2025

Saksham Aggarwal

Saksham Aggarwal is a founder and engineer building AI agents that automate engineering grunt work, starting with SDK integrations so that product teams can move faster.

He was the #1 engineer / founding member at PYOR, a Castle Island–backed financial data startup, where he built an enterprise data terminal for on-chain analytics.

Saksham has also driven growth at Flint (now LogX) and scaled Conquest, India’s largest student-run startup accelerator.

He’s passionate about: Agentic retrieval systems, Programmable prompts, Evals (Text/Image/Video), Interaction design, and Synthetic data pipelines.


Preferred Pronoun

He/Him

Speaker Tagline

Co-founder, Iterate AI | CS@BITS Pilani

Gravatar - Professional Photo

https://gravatar.com/sxmawl

LinkedIn Profile

https://linkedin.com/in/sakshamagg27

Twitter (X) Profile

https://x.com/sxmawl

Instagram Profile

https://www.instagram.com/saksham.awl

Professional Link

https://x.com/sxmawl


Session

09-14
10:50
30min
Evals First, Code Later: A Practical Guide to Evaluations, Rerankers & Caches
Saksham Aggarwal

Many retrieval-augmented generation (RAG) and code-search pipelines rely on ad-hoc checks and break when deployed at scale.

This talk presents an evaluation-first development workflow applied to a production code-search engine built with Python, PostgreSQL (pgvector), and OpenAI rerankers. Introducing automated evaluation suites before optimisation cut average query latency from 20 min to 30 s, delivered a 40 × speed-up, and raised relevance by ≈ 30 %. We will cover:

  • Constructing task-specific evaluation datasets and metrics
  • Hybrid (lexical + ANN) retrieval
  • Cross-encoder reranking for precision boosts
  • Semantic caching strategies that keep indexes fresh and queries fast

The session includes benchmark results, a live demonstration, and an MIT-licensed reference implementation that attendees can clone and extend.

AI, ML, Data Science
Track 3