Audio
fastrtc
FastRTC turns any Python function into a real-time audio or video stream over WebRTC or WebSockets
★ 4.6k starsMIT
View guide →// audio · jik876/hifi-gan
hifi-gan
hifi-gan is an open-source audio project by jik876 (jik876/hifi-gan on GitHub). It has 2,373 GitHub stars and 560 forks, is released under the MIT license, and was last updated on GitHub on 27 July 2024.
GAN vocoder that turns mel-spectrograms into high-fidelity speech audio, fast enough for real-time use.
★ 2.4k stars⑂ 560 forksMITupdated 2024-07
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
so here's a placeholder. The guide below is complete regardless.
// project facts
Project facts
- Repository
- jik876/hifi-gan
- GitHub stars
- 2,373
- Forks
- 560
- License
- MIT
- Last updated on GitHub
- 27 July 2024
- Category
- Audio
- Listing
- Full install guide
// how it works
How it works
- Clarify your goal
Decide whether you need inference with a pretrained checkpoint, fine-tuning on your own voice data, or training from scratch.
- Get the project
Clone the repository and install the Python requirements; for training, download the LJ Speech dataset and place its wavs as documented.
- Install and configure
Download a pretrained model folder that matches your use case, such as a universal model or one fine-tuned for Tacotron2 mel-spectrograms.
- Run your first task
Run inference from wav files or mel-spectrograms, or start training with one of the bundled V1, V2, or V3 configurations.
// features
Why people love it
High-fidelity waveform generation
A GAN generator designed around periodic patterns in audio, producing 22.05 kHz speech the paper reports as close to human quality in listening tests.
Pretrained models included
Checkpoints for LJSpeech, VCTK, and a universal model, including versions fine-tuned on Tacotron2-generated mel-spectrograms.
Two inference modes
inference.py reconstructs audio from wav-derived features, while inference_e2e.py synthesises speech directly from mel-spectrogram files for end-to-end TTS.
Fine-tuning workflow
Generate mel-spectrograms with a model such as Tacotron2, place them in the fine-tuning dataset folder, and continue training with the fine-tuning flag.
Three generator configurations
Bundled config_v1, config_v2, and config_v3 JSON files trade model size and speed against quality.
Transfer-learning base
The universal model ships with discriminator weights intended as a starting point for adapting to new datasets.
Advertisement
// run it
Run it yourself
- Clone and install requirements
Python 3.6 or newer is required; install the dependencies from requirements.txt.
git clone https://github.com/jik876/hifi-gan.git cd hifi-gan pip install -r requirements.txt - Train or fine-tune (optional)
Train with a bundled config, or fine-tune a checkpoint on your own mel-spectrograms.
python train.py --config config_v1.json python train.py --fine_tuning True --config config_v1.json - Run inference
Put wav files in test_files (or mel files in test_mel_files) and generate audio with a checkpoint.
python inference.py --checkpoint_file [generator checkpoint file path] python inference_e2e.py --checkpoint_file [generator checkpoint file path]
// use cases
What you can build
- Adding a fast, high-quality vocoder stage to a Tacotron2 or Glow-TTS pipeline
- Converting mel-spectrograms from an acoustic model into listenable wav files
- Fine-tuning a vocoder on a custom single-speaker dataset
- Researching GAN-based speech synthesis and vocoder architectures
Advertisement
// faq
Questions, answered
Who maintains it?
It is maintained by jik876 (Jungil Kong) and collaborators, as the official open-source release accompanying the HiFi-GAN paper.
Is it free to use?
Yes. It is MIT-licensed, including the provided pretrained models, so research and commercial use are permitted.
Does it need a paid API key?
No. HiFi-GAN runs entirely locally in PyTorch; there are no API keys or hosted services involved.
How does it compare to a well-known alternative?
WaveGlow and MelGAN are earlier neural vocoders the authors drew on; HiFi-GAN focuses on modelling periodic audio patterns to reach higher fidelity at much faster than real-time generation.
How do I get started?
Clone the repo, pip install -r requirements.txt, download a pretrained model, and run inference.py or inference_e2e.py with its checkpoint.
// keep exploring
Similar tools
Audio
Real-Time-Voice-Cloning
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
★ 60.2k starsNOASSERTION
View guide →Audio
cheetah
On-device streaming speech-to-text engine for real-time apps.
★ 670 starsApache-2.0
View guide →Audio
voxt
A macOS voice assistant that turns speech into actions and notes.
★ 853 starsCustom
View guide →Audio
MOSS-TTS-Nano
MOSS-TTS-Nano: a 0.1B open multilingual speech model that clones voices and streams in real time on CPU
★ 4.5k starsApache-2.0
View guide →Audio
gTTS
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
★ 2.6k starsMIT
View guide →// alternatives
Alternatives in Audio
The most-starred other Audio projects in the LibreAI catalogue — the realistic shortlist if hifi-gan is not the right fit.
- MoneyPrinterTurbo★ 128.3k
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
- Whisper★ 109.9k
Robust Speech Recognition via Large-Scale Weak Supervision
- GPT-SoVITS★ 62.3k
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
- Real-Time-Voice-Cloning★ 60.2k
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
- whisper.cpp★ 54.1k
Port of OpenAI's Whisper model in C/C++
// more in audio
More in Audio
Nearby Audio projects in the catalogue, closest in stars to hifi-gan — keep exploring the shelf either side of it.
- VoiceMem★ 2.4k
VoiceMem — streaming dual-brain memory for voice agents: a left brain for facts, a right brain for emotion
- IMS-Toucan★ 2.2k
ToucanTTS: fast, controllable text-to-speech for over 7,000 languages, from Stuttgart's IMS.
- pyttsx3★ 2.5k
Offline text-to-speech for Python — uses your operating system's own voices, no internet needed
- WaveRNN★ 2.2k
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
- marytts★ 2.6k
MaryTTS — open-source, multilingual text-to-speech platform written in pure Java
- transcribe.cpp★ 2.0k
Fast local speech-to-text across 16+ model families, powered by ggml.
Get hifi-gan
Free and open — grab the source, star the repo, and start creating.