Models, runtimes, and accelerators

The Aptabit Platform uses external and local models side by side. Models are registered once in the installation and enabled per workspace; agents and flows choose the model each task uses.

External providers

In Settings > Models, register an external provider:

Type	Use
OpenAI	The OpenAI API.
Azure OpenAI	Your organization's Azure deployments.
Anthropic	The Anthropic API.
OpenAI-compatible	Any endpoint with the OpenAI API: corporate gateways, remote inference servers, and other vendors that offer this compatibility.

API keys are encrypted in the database and never reach the browser. Each provider has a connection test.

With an external provider, requests to the model go to that provider. For content that must not leave your organization's environment, use local models.

Managed local models

The platform deploys and operates local models as containers, on six runtimes:

Runtime	Hardware requirement
llama.cpp	Any CPU; the default runtime.
OpenVINO Model Server	Intel CPU or GPU.
IPEX-LLM	Intel CPU or GPU.
vLLM	NVIDIA GPU with CUDA.
SGLang	NVIDIA GPU with CUDA.
TensorRT-LLM	NVIDIA GPU with CUDA.

From the interface, each local model can be installed, started, stopped, restarted, or removed, with an event history. Containers are created by the docker-guard service, which allows only known runtime images and never privileged containers. See Installation security.

Check before deployment

Before deploying, the platform checks the hardware. It refuses vLLM, TensorRT-LLM, or SGLang without a CUDA GPU visible to Docker and refuses OpenVINO without Intel hardware. When the model's memory estimate exceeds the available memory, it warns and suggests a smaller or quantized model.

Formats and quantization

Format	Compatible runtimes
GGUF	llama.cpp
OpenVINO IR	OpenVINO Model Server
Hugging Face safetensors	vLLM, SGLang, TensorRT-LLM, IPEX-LLM, and OpenVINO

The platform offers only the runtimes able to serve each format. Quantization profiles: Q4_K_M, Q5_K_M, Q8_0, FP16, INT8, and INT4.

Managed runtimes download the model weights from Hugging Face before starting. On an isolated installation, prepare the runtime images and the weights with Aptabit, or use an inference server already available on the local network.

Existing inference servers

The platform detects running Ollama, vLLM, and LM Studio servers and connects their models as OpenAI-compatible endpoints. Internal addresses must be allowed in the SSRF protection (APTABIT_SSRF_ALLOWED_HOSTS).

Model catalog

The platform works with more than 1,000 open models. The curated catalog organizes them in ten modalities: text, code, reasoning, embeddings, vision, image, audio, rerank, video, and voice. It can be filtered by modality, family, quantization, and runtime, and it shows whether each model fits the machine.

The platform's features use text, code, reasoning, vision, embedding, rerank, and audio transcription models.

Per-workspace enablement and default models

A model is registered for the installation, but it appears in chat only in the workspaces where it is enabled.
The installation sets default models for embeddings, reranking, image reading, and audio and video transcription.
Embeddings and reranking use locally served models.

Accelerators

The Settings > System tab shows the machine's CPU, memory, disks, and GPUs, with the video memory of each. The platform uses:

CPU, with no accelerator.
NVIDIA GPUs, through CUDA, with the NVIDIA Container Toolkit.
Intel GPUs, in the OpenVINO and IPEX-LLM runtimes.
NPUs, with model execution available.

See Requirements and sizing.