2025-09-14 –, Track 3
We’ve all had that moment — sitting at a keyboard, fingers frozen, wondering where to even begin. The dream of playing music lives in many of us, but the reality? Long workdays, unpredictable schedules, uninspiring lessons, and slow progress often extinguish the spark before it can catch fire.
What if technology could change that?
This project combines Python, Deep Learning, and Music Theory with a custom hardware setup to create an intelligent and engaging virtual music tutor. Currently designed for keyboards and electric pianos, the system visualizes songs, guides your practice sessions, and adapts to your skill level — making music learning more intuitive, flexible, and enjoyable.
Overview -
Picture this: you hear a song you love and think, “I wish I could play that.” With this tool, you can.
You start by uploading or recording the song. The system quickly analyzes the audio and reveals the notes and patterns behind the music. Your keyboard lights up with each note, guiding your hands as you play along. No sheet music, no complicated theory — just you, your instrument, and the music you already love.
As you continue learning, the system adjusts to your pace. It gradually introduces new concepts — like chords, scales, and harmony — woven naturally into the songs you're learning. You can slow things down, loop tricky sections, or challenge yourself with more complex arrangements as you grow.
Whether you're picking up the keyboard for the first time or returning after years away, this project transforms learning into a hands-on, personalized experience. By combining smart technology with the joy of real music, we aim to help people practice, play, explore, and fall in love with learning again.
Outline of the Talk -
1. Introduction – Challenges in learning music and how this system addresses them
2. Extraction – Analyzing and separating song layers using deep learning
3. Theorization – Applying music theory to generate playable, structured lessons
4. UI Presentation – A MIDI-based interface for interacting with lessons of varying complexity, including theory and play modes
5. Hardware Visualization – LED-strip visualizer to gamify practice sessions with real-time feedback
6. End Result – Demo video showcasing the complete learning experience
Tech Stack -
Python, React, Deep Learning, Raspberry Pi (or similar)
Speaker Info -
Anant Gupta – Applied AI/ML Lead, passionate about exploring innovative ideas and hands-on hardware experimentation.
Lakshya Gupta – Software Engineer with a passion for music, piano, and the creative applications of AI.
Passion to learn music
Target Audience –Beginner
Software Engineer at JP Morgan Chase
Undergraduate in Computer Science and Engineering from IIT (BHU), Varanasi, 2021
Background in ML/AI, Deep Learning, and IDP, trained in both Hindustani Classical and Western Classical Music (Instrumental)
I am a technologist who enjoys finding and solving problems. I believe that Hardware, Software, Backend, Frontend are all abstractions and one should not limit your scope to just a small vertical
I have around 14 years of experience dabbling with various types of techstack