Choose your platform

Select the operating system and device that will run the model.

Platform	CPU	Acceleration
Linux	Docker Compose or native Python	NVIDIA CUDA, in Docker or natively
macOS	Native on Apple Silicon; Docker for CPU, including Intel Macs	MPS on Apple Silicon, native only
Windows	Linux containers in Docker Desktop or Linux under WSL 2	NVIDIA GPU through Docker Desktop with WSL 2

The project supplies Bash setup scripts and a Makefile for Unix environments. The Windows instructions use WSL 2 or Linux containers; the repository does not provide a native PowerShell installer. MPS is a macOS backend. The project's CUDA path targets NVIDIA GPUs, not AMD or Intel GPUs.

Before you begin

Native setup needs Git, make, uv, and Python 3.12. The Makefile creates a venv/ environment through uv, which can provision Python. Container setup needs Git and Docker with Compose; Python is included in the image.

The first model download requires internet access and free disk space. RAM or VRAM usage depends on the model, input lengths, and batch sizes. The project does not specify one minimum memory requirement for all configurations. After loading the model, test a real request and then a workload representative of your inputs.

Native setup stores the default model in models/user-bge-m3. Compose uses a persistent models volume. Run only one of these alternatives on port 8091 at a time.

After installation

Finish every installation with the readiness check and first embedding request. A running process alone is not enough: the model must successfully return a vector.

Configure your installation

See Configuration and models for all settings, environment variables, command-line tools, and instructions for installing a different model.