We’ll use Whisper, Llama 3, Cloudflare R2 storage & Supabase database to create our own Voice to Notes app. You’ll have it running in 5 minutes!

Voice to notes app using Whisper and LLaMA 3

Voice to notes app using Whisper and LLaMA 3

​
Pre-requisites

To build your own Voice to notes app you’ll need to have Supabase, Replicate, Groq and Storage set up. If you haven’t, please start by doing that.

Supabase

Set up user authentication & PostgreSQL database using Supabase

Storage

Set up audio, pdf and image storage using Cloudflare R2

Replicate

Set up Replicate & understand how it’s used throughout the app

Groq

Set up Groq & understand how it’s used throughout the app

Once that is ready, you can go ahead and follow the steps below.

​
Database tables

We’ll create 3 tables: one to store our transcriptions, one for our recordings and one for our summaries.

Copy paste the following into your Supabase SQL editor. Make sure to do this for both tabs!

Great job! The database setup is complete, and your app is ready to go. Visit http://localhost:3000/audio to see it in action.

​
App Structure

  1. app/api/audio/* Key API routes:

    • app/api/audio/upload/route.ts: Uploads audio recordings to Cloudflare R2, returns a public URL, and updates the recordings table. It also reduces the credits of the user if you’ve defined paywall === true in app/audio/toolConfig.ts
    • app/api/audio/transcribe/route.ts: Transcribes the audio recording using incredibly-fast-whisper (Replicate) and stores the data in Supabase.
    • app/api/audio/summarize/route.ts: Generates a short summary and a list of action points.
    • app/api/audio/delete/route.ts: Deletes data from both Supabase and Cloudflare R2.

  2. /app/audio/* Contains all front-end logic, including paywall checks and dynamic pages per recording.

  3. /components/audio/* Contains all front-end components unique to the audio app.

If you have any questions, feel free to reach out to me on Twitter or Discord!

Chat with PDFText to Speech