Description:

Struggling to build RAG chatbots that are fast, accurate, and can handle complex documents containing images and tables? Traditional RAG pipelines often hit bottlenecks in data ingestion and retrieval efficiency, leading to slow updates, irrelevant responses, and high token costs.

This talk unveils a robust framework for building high-efficiency chatbots using Retrieval-Augmented Generation (RAG) with a hybrid re-ranking strategy, demonstrated through a real-world use case. We'll showcase innovations across the pipeline:

• Accelerating Data Ingestion: See how our asyncio-powered scraper processes websites concurrently, dramatically reducing crawl time by over 9× (5h 18m → 40m).
• Handling Complex Documents: Learn our Multi-Modal Chunking technique for seamlessly ingesting documents containing intricate text, images, and tables, preparing them effectively for RAG.
• Optimizing Query Workflow: Understand how classifying and rephrasing user queries improves the initial retrieval stage for better relevance.
• Implementing Hybrid Re-Ranking: Dive into our core Hybrid Re-Ranking mechanism that combines semantic relevance with structural metadata (like site depth and authority) to surface the most pertinent passages from potentially noisy initial search results.
• Building the Full Pipeline: Get a clear, step-by-step blueprint for integrating these techniques to minimize token usage, accelerate response times, and consistently deliver precise, context-aware answers.

Attendees will leave with practical insights and a framework to significantly enhance the performance and capability of their own RAG implementations.

Session Outline:

• Introduction & Motivation (1 min)

  • Challenges with traditional chatbots: high token usage, latency, generic responses
  • Objectives: boosting efficiency and accuracy with RAG + re-ranking

• Standard RAG Pipeline Overview (3 min)

  • Chunk → embed → vector store
  • Document-type chunking (HTML vs. PDF) and multi-modal ingestion of images/tables

• Web Scraping Abstraction (5 min)

  • Async scheduler-driven graph traversal over a website’s HTML & PDF content. (9× faster compared to synchronous scraping: 5 h 18 m → 40 m)
  • Parsing dynamic webpages and complex PDFs, containing images and tables, and extracting content.
  • Key metadata captured (crawl depth, doc type, nav hierarchy)

• Enhanced LLM Interaction Workflow (5 min)

  • Classifying queries (gibberish, general chat, site-specific)
  • Rewriting into semantically rich variants for broader recall

• Hybrid Re-Ranking Strategy (5 min)

  • Scoring snippets by semantic relevance, page-level weight, domain authority
  • Applying weighted formula (α, β, γ) to surface the most pertinent chunks

• Comparison: Traditional vs. Our RAG-Driven Approach (3 min)

  • Traditional: raw LLM calls → token bloat, context loss
  • Ours: RAG + hybrid re-ranking → targeted retrieval, cost-effective precision

• Demonstration & Insights (3 min)

  • End-to-end flow: user query → initial search → re-ranking → final LLM answer
  • Tips on balancing accuracy, latency, and token budget

• Conclusion & Q&A (5 min)

  • Applying this pipeline in your own projects
  • Open floor for questions and future directions

Prerequisites:

Attendees should have a basic understanding of:

• Python 3.8+ and familiarity with asyncio for concurrent programming.
• Large Language Models (LLMs) and text embeddings.
• Experience with vector databases (e.g., Pinecone, FAISS) or general retrieval systems.
• General familiarity with REST APIs and prompt engineering best practices.