Coding agents against your own endpoint
By the end of this page, Continue, Cline and aider send their requests to a model on your own DGX Spark instead of a metered API. All three speak OpenAI's API shape, so each one needs only a base URL, a key and a model name. The code you paste into an agent then goes to a machine you rent, not to a third-party model provider.
You need:
- A running GPUwerk instance with an OpenAI-compatible endpoint: a console model preset such as Qwen3 Coder 30B, or your own engine on a base image.
- For a console preset, an API key with the
inferencescope, created in the console. - Continue, Cline or aider installed on your laptop.
- No SSH key. GPUwerk configures the machine, and the console's browser terminal works without one. Add your own key only if you want your own SSH client, scp or port forwarding.
Start with a running endpoint
Every agent below needs an OpenAI-compatible base URL ending in /v1 and the exact model name the server reports. Set that up first.
- Pick the endpoint. With a console model preset, copy the HTTPS endpoint from the instance page in the console (it looks like
https://app-<instance-id>.gpuwerk.com) and add/v1. Every request needs your inference-scope API key as a bearer token. With your own engine, start vLLM if several people will share it, or llama.cpp if it is just you, since it is faster for a single stream. In the samples below,<spark-host>is then your instance's hostname,<name>.gpuwerk.com. - Protect your own engine. Each instance has its own public IPv4 address, assigned by GPUwerk, so an engine listening on
0.0.0.0is reachable from the internet. Start it with an API key (vLLM's--api-key), or bind it to127.0.0.1and reach it through an SSH tunnel. The tunnel is the one step here that needs your own SSH key. - Confirm it answers before you touch any editor config. For your own engine:
curl http://<spark-host>:8000/v1/models
For a console preset:curl https://app-<instance-id>.gpuwerk.com/v1/models \ -H "Authorization: Bearer $GPUWERK_API_KEY"
You should see JSON whosedatalist contains the model ID, for exampleqwen3-coder-30bon the Qwen3 Coder 30B preset. Use that ID as the model name in every config below.
Continue (VS Code / JetBrains)
- Open Continue's
config.yaml(older installs useconfig.json). - Add an
openaiprovider entry pointed at your instance:
models:
- name: My Spark
provider: openai
model: Qwen/Qwen3.8-27B-FP8
apiBase: http://<spark-host>:8000/v1
apiKey: not-needed-unless-you-put-a-gateway-in-front
- Set
modelto the ID from/v1/models. For a console preset, setapiBaseto the preset's HTTPS endpoint plus/v1andapiKeyto your inference-scope API key. With LiteLLM in front for per-user keys and budgets, use the gateway's URL and a virtual key instead. - Reload the editor window and pick My Spark in Continue's model selector. A test prompt should get a streamed answer from your instance.
Cline (VS Code)
- Open Cline's settings and choose the OpenAI Compatible provider.
- Set the base URL to your endpoint's
/v1URL. - Paste your inference-scope API key for a console preset. For your own engine without a key, any non-empty string works.
- Enter the model ID exactly as
/v1/modelsreports it. A first task should show Cline reading files and answering from your instance.
Cline runs longer tool-use chains and edits more files per turn than Continue, so it uses more context. Serve the model with a context window that fits your repository, not a short --max-model-len.
aider (terminal)
aider reads the endpoint from environment variables, so the simple case needs no config file. Set them and start aider; it should open a chat prompt and answer from your instance:
export OPENAI_API_BASE=http://<spark-host>:8000/v1 export OPENAI_API_KEY=not-needed-unless-you-put-a-gateway-in-front aider --model openai/Qwen/Qwen3.8-27B-FP8
For a console preset, set OPENAI_API_BASE to the preset's HTTPS endpoint plus /v1, OPENAI_API_KEY to your inference-scope key, and use --model openai/qwen3-coder-30b. Keep the openai/ prefix in every case: it tells aider's LiteLLM client which API format to speak, even though no OpenAI service is involved.
Which model to run
Use a coding model rather than a general chat model. Qwen3 Coder 30B is the console preset for this: it is a mixture-of-experts model with about 3B active parameters, so it answers quickly and leaves most of the Spark's 128 GB free. The preset serves it as qwen3-coder-30b with a 32,768-token context window. If your agent needs more context than that, run the model yourself on a base image with a larger --max-model-len. See which models fit in 128 GB for the other options.
The preset does not start vLLM with tool-call parsing, so agents that work through OpenAI tool calls, such as pi and OpenCode, get their requests rejected by it. Running a coding agent on a rented Spark shows the llama.cpp setup that returns tool calls, with configs for both.
What actually breaks
- Requests fail once a task grows. Cause: the agent read several files and the prompt passed the server's context limit. Fix: raise
--max-model-lenon your own engine, or keep tasks smaller on the preset's 32,768 tokens. - Every response seems to hang, then arrives at once. Cause: a gateway or proxy between the agent and the engine buffers Server-Sent Events. vLLM and llama.cpp stream by default. Fix: turn off response buffering on the proxy.
- A 404 that looks like a network problem. Cause: the model name in the agent config does not match what the server has loaded. Fix: copy the ID from
/v1/modelsexactly. - 401 or 403 from a preset endpoint. Cause: missing or wrong key. Fix: send an API key with the
inferencescope from the account that owns the instance.