BlogImplementation Guide

Transcribe YouTube Videos with Whisper, and What Changes on a Server

Downloading the audio and running Whisper works on a laptop. On a server the download fails, Whisper wants a GPU, and most videos already had captions. One call that uses captions first.

On this page8 sections

The usual recipe for a YouTube transcript in Python is: download the audio, run OpenAI's open-source Whisper on it, done. On your laptop that works well. In a shipped app it runs into three problems: the download step fails from cloud servers, Whisper needs a GPU to be fast, and most videos you send it already had captions you could have used for a fraction of the cost. This guide covers the free route first, then what to change once it runs on a server.

The free route: download the audio, run Whisper

bash
pip install pytubefix openai-whisper
# Whisper also needs ffmpeg on the machine: brew install ffmpeg / apt install ffmpeg
python
import whisper
from pytubefix import YouTube

yt = YouTube("https://www.youtube.com/watch?v=jNQXAC9IVRw")
audio_path = yt.streams.get_audio_only().download(filename="audio.m4a")

model = whisper.load_model("base")  # tiny, base, small, medium, large: bigger is slower and better
result = model.transcribe(audio_path)

print(result["text"])
for segment in result["segments"]:
    print(f"{segment['start']:.1f}-{segment['end']:.1f} {segment['text']}")

For a script on your own machine, that's a good setup and it costs nothing. Keep it if that's where it runs. Moving it into an app is where it gets harder:

  • The download fails on the server. YouTube blocks or challenges a lot of traffic from cloud and datacenter IPs. The same code that downloads the audio from home fails from AWS, Render, Railway, Fly or a CI runner, before Whisper ever runs.
  • Whisper wants a GPU. The larger models are slow on CPU, so a long video can tie up a web server for minutes. In practice that means a GPU worker, a queue and a place to put the audio files.
  • Most videos already have captions. Running speech-to-text on a video that has uploaded or auto-generated captions spends compute to recreate text YouTube already has. Checking for captions first is cheaper and often more accurate on names and terms the uploader typed in.

The same thing with one API call

Importly takes the YouTube link and returns the transcript. With "mode": "auto" it uses the video's captions when there are some and runs speech-to-text when there aren't, so you only pay for speech-to-text on videos that need it. It works the same from a server as from your laptop, with no GPU or audio files on your side:

python
# transcript.py
import os
import time

import requests

API = "https://api.importly.io"
HEADERS = {"Authorization": f"Bearer {os.environ['IMPORTLY_API_KEY']}"}


class TranscriptError(Exception):
    def __init__(self, code, message):
        super().__init__(message)
        self.code = code


def _check(res):
    body = res.json()
    if not res.ok:  # the request itself was rejected: bad link, bad key, low balance
        raise TranscriptError(body.get("code"), body.get("error"))
    return body["data"]


def transcribe(url, language=None):
    data = _check(requests.post(
        f"{API}/transcribe",
        headers=HEADERS,
        json={
            "url": url,
            "mode": "auto",  # captions if the video has them, speech-to-text if not
            "wait": True,  # hold the request until the transcript is ready (up to 30s)
            "maxCostDollars": 1,  # fail instead of running over this on a long video
            **({"language": language} if language else {}),  # e.g. "es" for non-English media
        },
        timeout=90,
    ))

    # Long video with no captions: still running after the wait, so poll
    while data["status"] not in ("completed", "failed"):
        time.sleep(3)
        data = _check(requests.get(f"{API}/transcribe/status", params={"id": data["jobId"]}, headers=HEADERS, timeout=30))

    if data["status"] == "failed":
        raise TranscriptError(data.get("code"), data.get("message"))
    return data["result"]


result = transcribe("https://www.youtube.com/watch?v=jNQXAC9IVRw")
print(result["source"])  # "platform_captions" or "importly"
for segment in result["data"]["segments"]:
    print(f"{segment['start']:.1f}-{segment['end']:.1f} {segment['text']}")

The segments have the same start, end and text fields as Whisper's, in seconds, so code that consumed Whisper's output usually only needs result["segments"] changed to result["data"]["segments"]. source tells you whether the text came from captions or speech-to-text, and captionKind whether captions were uploaded (manual) or generated by YouTube (auto). The full shape is in the transcripts docs.

For speech-to-text, pass language for non-English media. Without it, speech-to-text transcribes as English (caption tracks default to English when the video has one).

Already paying for OpenAI? Bring your key

If you'd rather keep speech-to-text on your own OpenAI account, send "provider": "openai" and your key as "apiKey". With "mode": "auto", captions are still used first, and your key only runs on videos without them. Importly charges $0.005 per media minute for fetching the media and returning the transcript, and OpenAI bills you its own fee. See provider options.

For batches, use a webhook

For a backlog of videos, send webhookUrl instead of wait. Each request returns a jobId at once, and Importly posts the finished transcript (or the failure, with its code) to your endpoint, where the fields sit directly on data (data.text, data.data.segments). Deliveries aren't retried, so poll jobs you haven't heard back about. See webhooks.

What it costs

  • From captions, uploaded or auto-generated: $0.003 per video.
  • Speech-to-text, when a video has no captions: $0.01 per media minute, fetching included, with a $0.01 minimum. A 10-minute video is $0.10.
  • maxCostDollars caps each request, and failed transcriptions are refunded.

It's free to try, no card required. See pricing.

When to keep self-hosted Whisper

Keep running Whisper yourself if your audio isn't on the web (uploads, recordings, calls), if you already run GPU workers with spare capacity, or if the audio can't leave your infrastructure. For links to YouTube and other video sites, the download step and the captions check are where self-hosting costs you the most time.

Have your coding agent wire it in

Connect https://mcp.importly.io/mcp to Claude Code, Cursor or another coding agent and ask it to replace the download-and-Whisper step in your app. It gets working code for your stack, writes the integration and tests real calls with your API key. The steps are in Have your agent build Importly into your app.

FAQ

Why does my Whisper pipeline work locally but fail on the server?

Usually it's the download, not Whisper. YouTube blocks or challenges a lot of traffic from cloud and datacenter IPs, so fetching the audio fails on AWS, Render or a CI runner even though the same code works at home.

Is Whisper more accurate than YouTube's captions?

It depends on the video and the audio. Uploaded captions were written or checked by the creator and are usually the best text you can get. "mode": "auto" uses captions when they exist and speech-to-text when they don't.

Can I get the video file too?

Yes. Use POST /import with "includeTranscript": true to get the file and its transcript from one job. See Download YouTube videos in Python.

Related: YouTube transcripts in Python, YouTube video with no captions, Video transcription API, Chunk a YouTube transcript for RAG.

Keep reading

Put video and audio links to work in your app

One API for files, transcripts and metadata. Free to try, no card required.

Try free →