An all-in-one tool for audio-video translation, dubbing, and voice cloning! Krillin AI, a one-stop solution for multi-language video conversion.

Project Overview

Krillin AI is an all-in-one audio-video localization and enhancement solution. This simple yet powerful tool integrates audio-video translation, dubbing, and voice cloning, supporting both horizontal and vertical screen formats to ensure seamless presentation across all major platforms (Bilibili, Xiaohongshu, Douyin, WeChat Channels, Kuaishou, YouTube, TikTok, etc.). Through an end-to-end workflow, Krillin AI transforms raw materials into polished, ready-to-use cross-platform content with just a few clicks.

Key Features and Functions

🎯 One-Click Launch: No complex environment setup required—dependencies are automatically installed, and the tool is ready to use immediately.
📥 Video Acquisition: Supports downloading via yt-dlp or uploading local files.
📜 Accurate Recognition: High-precision speech recognition powered by Whisper.
🧠 Intelligent Segmentation: Subtitle segmentation and alignment using LLM (Large Language Models).
🔄 Terminology Replacement: One-click replacement of domain-specific terms.
🌍 Professional Translation: Paragraph-level translations based on LLMs ensure semantic coherence.
🎙️ Dubbing and Voice Cloning: Offers CosyVoice-selected voices or custom voice cloning.
🎬 Video Synthesis: Automatic handling of horizontal/vertical video formats and subtitle layout.

Effect Showcase

Below is the result of importing a 46-minute local video and generating subtitles with one click. The output requires no manual adjustments—no missing or overlapping subtitles, natural sentence breaks, and high-quality translations.

Subtitle Translation Example

(Insert visual example here)

Dubbing Example

(Insert visual example here)

Language Support

Input Languages Supported: Chinese, English, Japanese, German, Turkish (continuously expanding).
Translation Languages Supported: English, Chinese, Russian, Spanish, French, and 56 other languages.

Interface Preview

(Insert interface screenshots or descriptions here)

🚀 Quick Start Guide

Basic Steps:

Download the executable file matching your device system from the Release section and place it in an empty folder.
Create a config folder within the folder, then create a config.toml file inside the config folder. Copy the contents of config-example.toml from the source code’s config directory into config.toml, and fill in your configuration details. (For users who want to use OpenAI models but don’t know how to obtain a key, you can join the group for a free trial.)
Double-click the executable file or run it in the terminal to start the service.
Open your browser and navigate to http://127.0.0.1:8888 (replace 8888 with the port number you configured).

For macOS Users:

Since the software is unsigned, after completing the basic steps above, you need to manually trust the application. Follow these steps:

Open the terminal and navigate to the directory containing the executable file (e.g., KrillinAI_1.0.0_macOS_arm64).
Execute the following commands sequentially:

   sudo xattr -rd com.apple.quarantine ./KrillinAI_1.0.0_macOS_arm64
   sudo chmod +x ./KrillinAI_1.0.0_macOS_arm64
   ./KrillinAI_1.0.0_macOS_arm64

The service will start.

Docker Deployment

This project supports Docker deployment. Refer to the Docker Deployment Instructions for details.

Cookie Configuration (Optional)

If you encounter video download failures, refer to the Cookie Configuration Instructions to configure your Cookie information.

Configuration Help (Must Read)

Fast and Easy Configuration:

Set transcription_provider and llm_provider to openai.
In the openai, local_model, and aliyun sections, only fill in openai.apikey for subtitle translation. (Optionally configure app.proxy, model, and openai.base_url as needed.)

Using Local Language Recognition Models (Not supported on macOS):

Set transcription_provider to fasterwhisper and llm_provider to openai.
In the openai and local_model sections, fill in openai.apikey and local_model.faster_whisper for subtitle translation. Local models will be downloaded automatically.

Alibaba Cloud Configuration (Required for Certain Use Cases):

If llm_provider is set to aliyun, configure aliyun.bailian for large model services.
If transcription_provider is set to aliyun or the “dubbing” feature is enabled during task startup, configure aliyun.speech for voice services.
If the “dubbing” feature is enabled and a local audio file is uploaded for voice cloning, configure aliyun.oss for cloud storage services.

Refer to the Alibaba Cloud Configuration Guide for more details.

Project Link

https://github.com/krillinai/KrillinAI/blob/master/README.md

Reproduction without permission is prohibited：AI LAB » An all-in-one tool for audio-video translation, dubbing, and voice cloning! Krillin AI, a one-stop solution for multi-language video conversion.