meetscribe

I start meetscribe when a meeting begins and press Enter when it ends. It then transcribes the meeting, labels who's speaking, and writes notes with a summary, decisions, and action items, all on my own computer. The recordings folder is excluded from version control, so meetings are never committed by accident.

I wanted my meeting notes to stay local, mostly out of consideration for the people I was meeting with. Most transcription tools upload your audio to a server, and I'm not sure how those companies use the data from our meetings, so I wanted to find a way to transcribe them myself. Along the way, I got to experiment with open-source tools, and I'm saving money too.

I couldn't find an open-source tool that fit, so I built one that runs locally, with Whisper for transcription and a local language model for notes. I kept it to the command line for now. The design work is in the experience around it: what gets captured, how speakers are labeled, how little the user has to do, and the guarantee that nothing leaves their computer.

I was surprised by how complex working with audio is. The first build recorded system audio, meaning whatever the meeting played through the speakers. That worked for meetings where I mostly listened, but in interactive meetings the transcript only had the other people's side, so the notes summarized half a conversation.

I redesigned the capture pipeline to record the microphone and system audio together, which raised a timing problem. System audio sends nothing while the call is silent, but the microphone streams constantly, so joining the two naively drifts out of sync. The fix was to timestamp every audio buffer against one shared clock and place it at its true position on a master timeline.

Recording the two sources separately also made speaker labels possible: the mic track is "You" and the system track is "Others." The mic also picks up the other speakers through your speakers, so the pipeline removes that echo to avoid duplicate lines. Headphones avoid the echo entirely.

I directed Claude Code through the build and focused my own effort on evaluating the output. Whisper's model size trades speed for accuracy, so I settled on a small default that runs well on a CPU. For notes, larger local models were better at telling decisions apart from discussion.

Long meetings don't fit in a local model's context window, and the model would quietly cut off whatever didn't fit. So meetscribe splits the transcript into sections, summarizes each one on its own, and then combines those summaries into one set of notes. To compare models fairly, I added a script that regenerates notes for every past recording without re-transcribing, so each change in model or prompt could be judged against the same meetings.

meetscribe has taken a lot of iterations. I didn't find most of the problems until I started using it in my own meetings, and I'm still finding them. A month in, I'm adding an automatic stop, because I keep forgetting to stop the recording when a meeting ends. Learning how to line up audio and split notes around a model's limits was a great experience, even when it was harder than I expected.

For now, it's just for me, although the code is public on GitHub. At some point, I want to learn enough to teach others how to do the same for themselves. I'd like to tie it into my community AI toolkit, and maybe add a visual interface, since many people aren't familiar with the command line.