Skip to content

First request and configuration

Check readiness

After the model loads, run this from a second terminal:

curl --fail http://127.0.0.1:8091/ready

A successful response contains "status": "ok" and queue state. Connections may fail while the model is downloading or before the server starts. For Docker, inspect docker compose ps -a and the model-init and embedder logs.

Request an embedding

The commands below work in Bash on Linux, macOS, and Ubuntu under WSL:

curl --fail-with-body http://127.0.0.1:8091/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepvk/USER-bge-m3","input":["How do I add document search?"],"encoding_format":"float"}'

Expect JSON with object: "list" and an array of numbers in data[0].embedding. A successful readiness check followed by a real vector confirms that the service can run inference, not just listen on a port.

The request's model must match the server's model_id. To embed multiple texts, pass multiple strings in input; results contain corresponding index values. This endpoint returns embeddings, not a written answer to the question.

Send background work

For the full method reference and client examples, see API.

Mark document-indexing requests with the batch header:

curl --fail-with-body http://127.0.0.1:8091/v1/embeddings \
  -H 'Content-Type: application/json' \
  -H 'X-Dzen-Embedding-Priority: batch' \
  -d '{"model":"deepvk/USER-bge-m3","input":["First document chunk","Second document chunk"],"encoding_format":"float"}'

Without that header, the request enters the realtime queue. Priority applies when choosing the next work, without interrupting inference already in progress.

Configure a native deployment

Create or update local.yaml at the checkout root:

device: cpu
host: 127.0.0.1
port: 8091
dashboard_enabled: true

Settings are applied in this order: embedder/config.yaml → local.yaml → environment variables. Restart after changing settings. When switching from CPU to CUDA or MPS, repeat make setup with the intended DEVICE before starting the server.

Use model_id and model_dir to select a model. Choose a new directory when replacing it: make setup checks for modules.json and does not replace an existing downloaded model in place.

Configure Docker Compose

The standard Compose configuration does not mount the checkout's local.yaml. Set supported Compose parameters in .env next to compose.yaml, for example:

EMBEDDER_PORT=18091
MODEL_ID=deepvk/USER-bge-m3

Repeat your CPU or CUDA docker compose up -d command, then check the API on port 18091. Changing MODEL_ID selects a separate directory in the persistent volume. EMBEDDER_TAG selects the image tag; GPU_DEVICE_ID selects the CUDA GPU.

For other settings, add environment variables to the embedder service through a Compose override. To enable the queue dashboard, create compose.dashboard.yaml:

services:
  embedder:
    environment:
      DASHBOARD_ENABLED: "true"

For CPU, run docker compose -f compose.yaml -f compose.dashboard.yaml up -d. For CUDA, insert -f compose.cuda.yaml between those files. Keep using the same file set for subsequent updates.

Observe and connect

  • /live: the HTTP process responds.
  • /ready: the service is ready to accept work.
  • /metrics: Prometheus metrics.
  • /docs: built-in API examples.
  • /dashboard: queue details when dashboard_enabled is enabled.

The port is bound to localhost by default. Another service in the same Compose project can use http://embedder:8091.

Embedder has no built-in API-key validation. For network access, use a private network or an authenticated TLS proxy. The repository includes an nginx example. Include metrics and the dashboard in your access configuration.

Troubleshooting

Symptom Check
Connection refused Model download, server process, and port
unknown embedding model Request model matches server model_id
CUDA is unavailable / MPS is unavailable Device, driver, and runtime; MPS needs native macOS
HTTP 504 Queue load and input length; request_timeout_seconds defaults to 30
Out of memory Model size, input length, batch sizes, and processes per device

See the default settings and request handler for configuration and API behavior.