Set up your own speech to notes app
We’ll use Whisper, Llama 3, Cloudflare R2 storage & Supabase database to create our own Voice to Notes app. You’ll have it running in 5 minutes!
Voice to notes app using Whisper and LLaMA 3
To build your own Voice to notes app you’ll need to have Supabase, Replicate, Groq and Storage set up. If you haven’t, please start by doing that.
Set up user authentication & PostgreSQL database using Supabase
Set up audio, pdf and image storage using Cloudflare R2
Set up Replicate & understand how it’s used throughout the app
Set up Groq & understand how it’s used throughout the app
Once that is ready, you can go ahead and follow the steps below.
The Voice to Notes feature requires three tables that should already exist if you followed the Quick Setup guide and ran the Supabase migrations:
These tables are:
recordings
: Stores audio file information and metadatatranscripts
: Manages transcriptions and their chunkssummaries
: Stores summaries and action items generated from transcriptsIf you haven’t run the migrations yet, please go back to the Quick Setup guide first. Alternatively, you can manually create the tables by running the SQL from this file in your Supabase SQL editor:
supabase/migrations/20240000000006_voice_to_notes.sql
The tables include Row Level Security (RLS) policies to ensure users can only access their own data.
Great job! The database setup is complete, and your app is ready to go. Visit http://localhost:3000/audio
to see it in action.
app/api/(apps)/audio/*
Key API routes:
app/api/(apps)/audio/upload/route.ts
: Uploads audio recordings to Cloudflare R2, returns a public URL, and updates the recordings
table. It also reduces the credits
of the user if you’ve defined paywall === true
in app/audio/toolConfig.ts
app/api/(apps)/audio/transcribe/route.ts
: Transcribes the audio recording using incredibly-fast-whisper
(Replicate) and stores the data in Supabase.app/api/(apps)/audio/summarize/route.ts
: Generates a short summary and a list of action points.app/api/(apps)/audio/delete/route.ts
: Deletes data from both Supabase and Cloudflare R2./app/(apps)/audio/*
Contains all front-end logic, including paywall checks and dynamic pages per recording.
/components/audio/*
Contains all front-end components unique to the audio app.
If you have any questions, feel free to reach out to me on Twitter or Discord!