Midi Harness: An Experimental Stack

Motivating Midi-Harness: Piano Youtube

"And remember: I'm a video. You can pause me!"

Brenda Earl Stokes (https://www.youtube.com/@PianoandVoicewithBrenda)

Brenda (one of my many faves) knows she's a video—a level of self-awareness I aspire to.

As a student, though, it's really hard to remember you can pause the video! It feels so much more natural to just watch and try to follow along, even when that means I'm huffily thinking to myself: I can't believe they're moving on to the next section already!!

Video isn't a perfect medium. Best we can do is just be aware that the media is media, and remember to pause and rewind if you're actually trying to learn.

Midi-Harness augments Piano Youtube

Midi-Harness is not claude-as-piano-teacher. It's just a stack for getting Midi data into Claude somewhat natively so I can ask clarifying questions to the LLM (the videos tutorials often mention things about music I'm barely aware of).

The hope is I can take a small nugget from one of the lessons that I don't understand, "play" it for my harness, and through the harness give the infinitely patient Claude the native-ish context it needs to answer my very basic questions.

Midi-Harness: Recipe

Off the shelf:

wisprflow ($15 / month, really good UI. Any voice-to-text will work)

Claude Code (any LLM will work)

Custom software:

Go listener (takes in Midi data, writes it to jsonl. LLM generated)

Claude Code customizations:

1 tiny CLAUDE.md (46 words total)

1 hook (fires on send-message, grabs + cleans Midi data then erases jsonl (so we're only sending most recent key events))

1 Skill + html template which Claude can use to show finger positions

Demo

code: github.com/jakesimonds/midi-harness

Lessons

a harness needs "lego blocks"

Obsidian is a nice "lego block" because it takes raw files, you can make plug-ins, and it does a few small things well.

Claude code is a nice lego block because of all the extensibility.

Wisprflow is a nice lego block because it does one thing well, and that thing plays nicely with Claude code.

When you build small little lego blocks, you allow future flexibility and the leverage of LLMs.

smaller can be better

Before midi-harness, I tried to build toward claude-as-piano-teacher. I hit copyright issues with song lyrics, a wall of complexity that worked against purposes, and also...it just wasn't a great flow to go back and forth with the LLM. Felt off.

The narrow use-case of, "okay, what the heck is going on with this specific chord and am I even playing what I think I'm playing?" was much more tractable, and still a big win.