Guides / kosa8 Cloud / Metrics, logs and labels
Metrics, logs and labels
Four things for running many sandboxes at once: what each one is using, what ran in it, a way to find it again, and a webhook when one ends without you deleting it or a command in it runs out of memory.
Metrics
curl -s https://api.kosa8.com/v1/sandboxes/sbx_…/metrics -H "Authorization: Bearer $KOSA8_API_KEY"
# {"metrics":{"id":"sbx_…","state":"running","cpus":2,"mem_mb":4096,"disk_gb":32,
# "cpu_pct":48.7,"cpu_seconds":312.4,"mem_bytes":1342177280,"mem_pct":31.3,
# "disk_bytes":2147483648,"net":{"out_frames":9120,"out_bytes":1203391,"in_frames":15003,"in_bytes":20311822},
# "egress_denied":0,"at":"2026-11-14T12:00:00Z"}}
- Read from outside the sandbox. The host reads the VM's own process and disk
files; nothing runs inside the sandbox to answer. So asking never wakes a sleeping sandbox and
never keeps one awake: a sleeping sandbox answers
{"state":"sleeping","sleep_bytes":…}. cpu_pctis CPU used over a quarter second, as a share of the sandbox's vCPUs (100 is every vCPU busy).cpu_secondsis the total since the VM booted or woke, so two readings give a rate over any period.mem_bytesis host memory the VM holds. Memory the guest has touched stays held, so it rises with use and seldom falls; it is not the guest's "free" figure.disk_bytesis the space the sandbox's disks take on the host. The disks are sparse: a 32 GB disk that is mostly empty takes far less.netcounts Ethernet frames and bytes (headers included) the guest sent that its egress policy passed, and those delivered to it.egress_deniedis packets the policy refused.- CPU and memory are
nullwhere the host cannot read them (a host whose VMs run inside the daemon, such as macOS). The call takes about a quarter second.
Logs
curl -s 'https://api.kosa8.com/v1/sandboxes/sbx_…/logs?limit=20' -H "Authorization: Bearer $KOSA8_API_KEY"
# {"id":"sbx_…","entries":[
# {"at":"…","cmd":["pip","install","-r","requirements.txt"],"exit":0,"ms":8312,"stdout_tail":"…Successfully installed …"},
# {"at":"…","cmd":["python","train.py"],"exit":137,"ms":40210,"stream":true,"stderr_tail":"Killed","oom":true}]}
Every command run in the sandbox, through the API or a direct exec, is logged on its host: when, the command (long arguments clipped), the exit code, how long it took, and the last 2 KiB of stdout and stderr. The newest 200 are kept, oldest first in the answer. The log lives on the sandbox's host, not in kosa8's database, and is removed with the sandbox. Commands inside a long-running process you started (a server's own requests) are not logged; only the commands you ran.
Labels
curl -s -X POST https://api.kosa8.com/v1/sandboxes \
-H "Authorization: Bearer $KOSA8_API_KEY" -H 'Content-Type: application/json' \
-d '{"labels":{"customer":"acme","run":"eval-42"}}'
curl -s 'https://api.kosa8.com/v1/sandboxes?label=run=eval-42&label=customer' -H "Authorization: Bearer $KOSA8_API_KEY"
curl -s -X PUT https://api.kosa8.com/v1/sandboxes/sbx_…/labels \
-H "Authorization: Bearer $KOSA8_API_KEY" -H 'Content-Type: application/json' -d '{"labels":{"run":"eval-43"}}'
- Up to 16 labels. Keys are 1–63 letters, digits,
.,_,/or-, starting with a letter or digit; values are strings up to 256 characters. ?label=key=valuematches that value;?label=keyany sandbox with the key. Repeat it and every one must match. It works withlimit,cursorandall=1.PUT …/labelsreplaces them all;{"labels": null}clears them. Labels are for your bookkeeping and are not secret: do not put credentials in them.
Webhooks
sandbox.killed{id, reason}: a sandbox ended without you deleting it. Reasons:ttl(itsttl_sordelete_after),egress(its egress policy could not be applied at create),not_on_host(its host no longer has it, for example after a host reboot),host_lost(its host was silent for 24 hours) andhost_removed(your own host was removed). It comes beside thesandbox.deletedorsandbox.errorevent you already get.sandbox.oom{id, exit, cmd, at}: a command ended with exit 137 and the guest's out-of-memory killer had just killed a process. The sandbox keeps running; the command did not. Give the sandbox more memory (a largersize), or use less. It reaches you with the host's next heartbeat, within about a minute. A kill with exit 137 that was not for memory (kill -9) is not reported, and an image withoutcatleavesoomunknown in the log rather than guessed.
Subscribe as for any other event: POST /v1/webhooks {"url": …, "events": ["sandbox.killed", "sandbox.oom"]}.
A new endpoint that names no events gets every event, these two included; an existing endpoint keeps
the events it chose. See Webhook recipes for checking signatures.