Seamless AI Dub
An English video goes in, a Portuguese dub comes out. No manual editing at any point along the way.
The problem
A lot of good technical material only exists in English. Subtitles solve part of it, but they compete with the screen: in a video where you need to watch what someone’s hands are doing, reading along the bottom gets in the way.
Dubbing by hand solves it better, but an hour of video turns into a day in a studio.
The decision
Build the whole path as a pipeline, with no manual step in the middle: the audio comes out of the video, Whisper transcribes it into timestamped segments, each segment is translated, the voice is synthesised, the cuts go back positioned by the original timestamp and the final video is rendered. The absence of a manual step is what gives the project its value: with a human in the middle it would just be a worse video editor than the ones that already exist.
Portuguese is longer than English
This problem only shows up the first time you run the pipeline. The translation of an English sentence usually comes out longer in Portuguese, and the synthesised voice does not fit in the original segment’s slot. Each segment spills a few tenths into the next, the error accumulates, and by minute three the dub is talking about something that left the screen a long time ago.
The way out was to speed up each voice cut individually, only as much as it needs to fit, without letting the pitch climb into cartoon territory. That way sync recovers segment by segment, instead of stretching the video or cutting a chunk out of the translation. The original audio sits underneath, very quiet, which keeps the room tone of the scene.
# o pydub encurta as ondas sonoras sem achatar a frequência
def acelerar_audio_sem_esquilo(arquivo_mp3, fator):
audio = AudioSegment.from_file(arquivo_mp3)
# speedup funciona perfeitamente mantendo o timbre humano grave
audio_rapido = audio.speedup(playback_speed=fator, chunk_size=150, crossfade=25)
audio_rapido.export(arquivo_mp3, format="mp3")
# ... dentro do laço, para cada segmento que o Whisper devolveu:
# Mede quanto tempo a IA gerou de áudio cru
duracao_ia = len(audio_temporario) / 1000.0 # Em segundos
# Descobre quanto tempo a pessoa original (em inglês) levou para falar
tempo_original_falando = segmento["end"] - inicio
if duracao_ia > tempo_original_falando:
fator_aceleracao = duracao_ia / tempo_original_falando
# O limite garante que a voz não fique ofegante ou robótica
fator_aceleracao = min(fator_aceleracao, 1.45)
acelerar_audio_sem_esquilo(nome_arquivo_voz, fator_aceleracao)
# Posiciona a voz já tratada no tempo exato
clip_voz = AudioFileClip(nome_arquivo_voz).with_start(inicio)Two voice providers, one fallback
Synthesis uses ElevenLabs when a key is configured and falls back to Edge-TTS when there is none. The two providers exist for an economic reason: one charges per character, the other is free, and the choice changes mid-use depending on what you are willing to spend. Anyone who just wants to watch the pipeline run does not have to put a card anywhere.
About the video above
It is someone else’s technical content, and it is here as a short demonstration of what the pipeline produces, not as material of mine. I put an excerpt in because this is the only project in the portfolio where no screenshot proves anything: the quality lives entirely in the audio track, so either you hear it or you cannot judge it.