We’ll use Whisper, Llama 3, Cloudflare R2 storage & Supabase database to create our own Voice to Notes app. You’ll have it running in 5 minutes!

Voice to notes app using Whisper and LLaMA 3

​
Pre-requisites

To build your own Voice to notes app you’ll need to have Supabase, Replicate, Groq and Storage set up. If you haven’t, please start by doing that.

Supabase

Set up user authentication & PostgreSQL database using Supabase

Storage

Set up audio, pdf and image storage using Cloudflare R2

Replicate

Set up Replicate & understand how it’s used throughout the app

Groq

Set up Groq & understand how it’s used throughout the app

Once that is ready, you can go ahead and follow the steps below.

​
Database Setup

The Voice to Notes feature requires three tables that should already exist if you followed the Quick Setup guide and ran the Supabase migrations:

npx supabase db push

These tables are:

  1. recordings: Stores audio file information and metadata
  2. transcripts: Manages transcriptions and their chunks
  3. summaries: Stores summaries and action items generated from transcripts

If you haven’t run the migrations yet, please go back to the Quick Setup guide first. Alternatively, you can manually create the tables by running the SQL from this file in your Supabase SQL editor:

  • supabase/migrations/20240000000006_voice_to_notes.sql

The tables include Row Level Security (RLS) policies to ensure users can only access their own data.

Great job! The database setup is complete, and your app is ready to go. Visit http://localhost:3000/audio to see it in action.

​
App Structure

  1. app/api/(apps)/audio/* Key API routes:

    • app/api/(apps)/audio/upload/route.ts: Uploads audio recordings to Cloudflare R2, returns a public URL, and updates the recordings table. It also reduces the credits of the user if you’ve defined paywall === true in app/audio/toolConfig.ts
    • app/api/(apps)/audio/transcribe/route.ts: Transcribes the audio recording using incredibly-fast-whisper (Replicate) and stores the data in Supabase.
    • app/api/(apps)/audio/summarize/route.ts: Generates a short summary and a list of action points.
    • app/api/(apps)/audio/delete/route.ts: Deletes data from both Supabase and Cloudflare R2.

  2. /app/(apps)/audio/* Contains all front-end logic, including paywall checks and dynamic pages per recording.

  3. /components/audio/* Contains all front-end components unique to the audio app.

If you have any questions, feel free to reach out to me on Twitter or Discord!

Chat with PDFText to Speech