2025-09-12 –, Room 6
Fine-tuning large language models (LLMs) used to be an expensive and resource-intensive process, traditionally accessible only to large organizations with powerful GPUs. With recent advances, this landscape has dramatically changed. Using cutting-edge techniques like Low-Rank Adaptation (LoRA), Quantized Low-Rank Adaptation (QLoRA), and Fully Sharded Data Parallelism (FSDP), it's now possible to fine-tune massive models such as the 70B-parameter LLaMA at home on consumer-grade GPUs.
In this hands-on tutorial, you will gain an intuitive understanding of the fine-tuning process, how it differs from pre-training, and its practical applications. By the end of this session, you will have hands-on experience fine-tuning a large model efficiently and cost-effectively, empowering them to create and deploy customized LLMs using accessible hardware setups.
In this hands-on tutorial, participants will learn the following:
1. Installation and Setup with Axolotl via Conda: Participants will start by installing and configuring the Axolotl framework using miniconda. Detailed guidance will be provided to ensure everyone has a fully functional local environment suitable for large model fine-tuning.
2. Understanding Fine-Tuning: Participants will learn about the concept of fine-tuning - how it differs from pre-training, why it is important, and what specific applications benefit from fine-tuned models.
3. Technical Deep Dive into LoRA, QLoRA, and FSDP: The session will introduce and explore the technical principles behind LoRA, QLoRA, and FSDP.
4. Introduction to Hugging Face Ecosystem and Transformers Library: Participants will gain hands-on familiarity with Hugging Face’s widely-used Transformers library. This part of the session will include an overview of the ecosystem, highlighting important tools and resources such as model repositories, datasets, and integration techniques.
5. Axolotl Configuration Overview: Detailed walkthroughs on how to configure Axolotl for fine-tuning tasks, covering key settings, parameters, and best practices to optimize training runs. Participants will understand how to customize configuration files to suit specific tasks and datasets.
6. Executing Fine-Tuning Runs in Axolotl: Attendees will learn the step-by-step process to launch fine-tuning runs using Axolotl. Practical examples will be demonstrated, enabling participants to replicate successful fine-tuning workflows independently.
7. Implementing Observability and Monitoring: Participants will learn about setting up observability tools for finetuning, learning to effectively monitor and track key performance metrics during model training.
8. Evaluating the Fine-Tuned Models: We will end the tutorial with approaches to evaluate fine-tuned models. Participants will learn how to set up benchmarks and objective tests to assess improvements accurately.
Resources required
- Access to two GPUs with ≥ 24 GB VRAM each (e.g., RTX 3090/4090, A6000). This is optional and you can follow along in the tutorial with a smaller toy model running on Colab
- Software stack - python 3.11 with either venv/conda, git
- We will edit yaml files in the workshop. So either have a code editor (like VSCode etc.) or terminal if you are comfortable with the likes of vim
Links to Github repos and papers
- https://github.com/axolotl-ai-cloud/axolotl
- https://github.com/huggingface/transformers
- https://github.com/bitsandbytes-foundation/bitsandbytes
- https://github.com/huggingface/peft
- https://arxiv.org/abs/2304.11277
- https://arxiv.org/abs/2305.14314
- https://arxiv.org/abs/2106.09685
Knowledge about large language models, generative models.
Additional Resources –Resources required
- Access to two GPUs with ≥ 24 GB VRAM each (e.g., RTX 3090/4090, A6000). This is optional and you can follow along in the tutorial with a smaller toy model running on Colab
- Software stack - python 3.11 with either venv/conda, git
- We will edit yaml files in the workshop. So either have a code editor (like VSCode etc.) or terminal if you are comfortable with the likes of vim
Links to Github repos and papers
- https://github.com/axolotl-ai-cloud/axolotl
- https://github.com/huggingface/transformers
- https://github.com/bitsandbytes-foundation/bitsandbytes
- https://github.com/huggingface/peft
- https://arxiv.org/abs/2304.11277
- https://arxiv.org/abs/2305.14314
- https://arxiv.org/abs/2106.09685
Intermediate
Aniket Kulkarni is the founder of Curlscape, where he helps businesses bring practical AI applications to life fast. He has led the design and deployment of large-scale systems across industries, from finance and healthcare to education and logistics. His work spans LLM-based information extraction, agentic workflows, voice assistants, and continuous evaluation frameworks. An engineer at heart, Aniket blends deep technical expertise with a product mindset to build AI that’s both reliable and usable.