Before you start
- Compatible NVIDIA GPU, CUDA 13 driver and device plugin
- ARM64 image requires NVIDIA SBSA hardware; ordinary ARM CPU nodes are unsupported
- Review model license and access terms
- Persistent model cache; image download is about 10 GB
- GPU execution has not been verified on the local development host
Configuration that needs your attention
Choose an actual model and a 40-character immutable Hugging Face revision. Supply inference-api-key and, only for gated access, an approved model-token.
The preset requests one NVIDIA GPU, 8 GiB system RAM and a 30 GiB model cache by default. Required VRAM depends on the model.
The pinned image targets CUDA 13; ARM64 requires NVIDIA SBSA hardware. Ordinary ARM CPU machines cannot run this GPU preset.
No model is downloaded during planning and remote model code is disabled. Local development has not verified GPU execution.
Required secrets
Configure these scoped secrets during deployment review in your Hakopod installation. Secret values stay out of the application TOML.
-
inference-api-key
Use this preset
- Open the Catalog in your own Hakopod dashboard and select vLLM.
- Choose the project, environment, and application name. Set the required values and secret references.
- Review the generated configuration, resource requests, networking, and storage plan.
- Deploy, then inspect readiness and logs. Configure backups for persistent data before relying on the service.
Read the upstream setup documentation ↗
View native TOML and setup notes ↗
Pinned container image
docker.io/vllm/vllm-openai:v0.29.0@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1
The catalog pins this image for reproducibility. Review upstream updates and compatibility before changing it.