Real-time Human-AI Improvisation over Jazz Standards
Master's thesis at the Chair for AI Methodology, RWTH Aachen
I am currently pursuing my master’s thesis under Prof. Holger Hoos, exploring real-time human-AI musical interaction on a Yamaha Disklavier MIDI keyboard. The goal is to fine-tune a chord accompaniment agent with reinforcement learning (RL) to generate musically sensible chords in response to a melody played by a human performer. My project proposes methodological extensions to the lineage of work:
- Adaptive Accompaniment with ReaLchords (Wu et al., 2024)
- ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers (Scarlatos et al., 2025)
- Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction (Wu et al., 2025)
Data. The original training mix covered four datasets (Hooktheory, POP909, Nottingham, Wikifonia). To cover a broader range of musical style, I added five more datasets: The Chord Melody Dataset, JAZZMUS (popular tunes and jazz standards), EMOPIA+ (emotion-labeled pop piano), the Weimar Jazz Database and FiloBass (jazz solos over with walking bass).
Model paradigm. Pre-training a Decoder on the data is not enough to produce sensible accompaniments, because the model’s response goes out of distribution (OOD) after the user starts playing something unexpected that was not covered in the training data. Mistakes compound with time during the jam session and the model cannot recover from these OOD states. To mitigate this, the authors train an Encoder-Decoder on the same data and distill knolwedge from it during RL fine-tuning. A contrastive- and discriminative model are used as rewards in the RL objective to score how data-like the rollout is that was generated by the policy (the Decoder).
Multi-scale reward. I trained reward models that score chord-melody coherence over fixed and sliding windows at several time scales, to see whether reward signal at a finer granularity helps RL training pick up local phrasing rather than only global style.
Voicing extraction. I built a pipeline that mines real simultaneous note-onsets from vast MIDI corpora (Aria-MIDI and Jazzvar) and maps them back to chord symbols, to eventually give the model a lookup table of prettier voicings instead of a simple block voicings.
Evaluation. I extended the original benchmark suite with a metric that scores whether the melody notes played over a chord are in the right mode.
Next up: Ablations: Does the added data diversity actually improve accompaniment quality, do the multi-scale rewards outperform single-scale ones once plugged into RL, and how sensitive is the whole pipeline to the per-dataset sampling weights.
