kosa8 docs

Commands / kosa8 model

kosa8 model

Pull and serve local models (OpenAI- and Anthropic-compatible)

kosa8 pulls GGUF models from Hugging Face and serves them on a fixed local port, exposing both OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) compatible endpoints.

Serving uses llama.cpp when llama-server is on PATH and kosa8's own built-in engine otherwise, so a model that pulled can always be served. Pick one explicitly with model serve --backend llama|engine.

Subcommands

kosa8 model lsList locally stored models kosa8 model pullDownload a GGUF model from Hugging Face (org/repo[:quant]) kosa8 model pull-ociInstall a model published to an OCI registry kosa8 model pushPublish a local model to an OCI registry as an artifact kosa8 model rmDelete a local model kosa8 model serveServe a model on the local gateway until interrupted