How to Run Cactus Needle 3 on Local Hardware

Cactus Needle 3 is a free, Apache 2.0-licensed model that performs tool calling, structured extraction, and embeddings on phones, laptops, and single-board computers without internet. Cactus reports the full model file at roughly 36 MB and CPU response times measured in milliseconds.

What Is Cactus Needle 3?

Cactus Needle 3 is a 121-million-parameter on-device model from Cactus, an open-source AI inference project, that performs tool calling, structured extraction, and embeddings without internet access. The video coverage from September 2026 describes it as a shift toward models that act rather than talk, and Cactus positions it exactly that way: it can fill in a function call from a rambling spoken sentence, but it will not hold a conversation because there is no tool for that. The project ships under the Apache 2.0 license, which permits commercial use, modification, and redistribution.

The size is the first thing that separates it from typical small models. Cactus reports the full model file at roughly 36 MB with the engine included, with individual slices between 8 and 29 MB depending on how many layers you run. For comparison, a 7-billion-parameter model compressed to 4-bit still occupies about 4 GB, an order of magnitude larger. The Cactus repository on GitHub is the canonical source for the engine and model files.

Cactus Needle 3 Benchmarks: What Was Actually Measured

Cactus published its own benchmark results across six suites, and every number in this section is vendor-reported rather than independently reproduced. The suites covered mobile phone commands (DroidCall, with 961 rows scored on exact Android actions), multi-call phone requests (200 requests, some requiring two ordered calls), and the Berkeley Function Calling Leaderboard BFCL v4 with 3,641 rows, which also checks whether the model correctly refuses when no tool fits. Three extraction suites followed: DSTC8 with 1,813 dialogue turns, SNIPS gold with 700 rows against a given schema, and SNIPS 7-way with 700 rows where the model must pick the correct schema out of seven before filling it.

Two configuration details matter when reading these results. First, Cactus states the comparison models ran at full precision under vLLM while Needle 3 ran as its shipped 2-bit compressed file, meaning the smaller model was arguably handicapped and still won its category. Second, Cactus reports that the 121-million-parameter model beats models roughly ten times its size on mobile tool calls and matches models two to three times larger on extraction. Fine-tuning on DroidCall lifted every sub-network by 18 to 36 points, and from four layers up, Cactus says the tuned sub-network passed DeepSeek V4 Flash on that narrow task. Treat these as the vendor's own measurements on specific suites, not proof of general superiority across all agent tasks.

Speed, Size, and Supported Hardware

Cactus reports a single tool call returning in 66 milliseconds on a plain CPU with no graphics card and no internet, and two function calls completing in under 97 milliseconds. On a Raspberry Pi 5, the vendor reports decoding at 400 to 4,000 tokens per second and input processing at 1,000 to 10,000 tokens per second. These are component-level, vendor-measured figures for structured tasks, not end-to-end application latency guarantees.

The platform list explains why the model is small. Per Cactus, supported targets include macOS, Linux (x86, ARM, ARMv7), Windows, Android on three architectures, iOS, iPadOS, watchOS, and the browser via WebAssembly, each shipping a pre-built engine under one megabyte that loads weights at startup. The design intent is a watch, a TV, a car, or a browser tab with no server behind it. One practical caveat from the repository itself: telemetry is on by default in the binary, and Cactus documents that you can disable it with two environment variables, a detail worth checking before any privacy-sensitive deployment.

Intelligence Laddering: One Model, Nineteen Sizes

The most distinctive design choice in Needle 3 is what Cactus calls intelligence laddering. The model has 20 layers, and Cactus trained it so that every depth from 2 layers to 20 works as a complete model on its own. One set of weights therefore contains 19 stacked sub-networks: a 2-layer slice runs at roughly 13 MB on a cheap phone, while the full 20 layers deliver maximum capability. You pick the depth that matches your hardware rather than shipping a different model per device.

The efficiency comes from where the parameters live. Most of them sit in what Cactus calls an engram, effectively a lookup table of common word patterns rather than computed attention. Cactus says the 121-million-parameter version therefore does roughly the compute work of a 50-million-parameter one, spending over twice fewer operations per token than a standard transformer of the same shape. Fine-tuning is also shallow by design: Cactus describes a two-command process where you supply an example file and a pass count, receive an adapter, then build a platform-specific file for your chosen layer count.

Grammar-Constrained Output and Confidence Scores

Needle 3 eliminates the parsing layer most agent stacks need. In a conventional setup, the model writes prose one token at a time, and a separate step extracts the JSON, validates it, and converts it into an action. That extraction step fails whenever the model inserts a sentence before the JSON or mangles a field. Needle 3's output is constrained at the byte level by a grammar generated from your own schema, so every token must fit the requested shape. Cactus states the model cannot hallucinate a field that does not exist and cannot return output that fails validation.

Every response also carries a confidence score from a trained scoring head, which Cactus describes as a calibrated value rather than raw probability. The engine blocks anything under 0.1 automatically, holding those calls back into a separate list. Above that floor, Cactus recommends a three-bucket pattern: run high-confidence results automatically, confirm middling ones with the user, and reject the rest. A triggers feature lets you attach text patterns, such as words like turn or switch, to specific tools, forcing the model to use that tool even at low confidence so critical intents are never silently dropped.

Known Limitations Reported in Testing

Four weaknesses surfaced when people put the model through real use, and each matters more than the benchmark wins for production planning.

Conversation history leaks without resets

Firing several different requests at the model without resetting in between can produce wrong calls. In one reported test, a request to set an office to 21 degrees, then order a pizza, then ask for the capital of France, with no reset, caused the model to invent a pizza call despite no pizza tool existing. The low confidence score on that call shows the safety layer working, but the invented call still appeared. Cactus has shipped an engine update addressing state on reset, so this is being actively worked on as of the September 2026 release coverage.

Missing default values produce empty results

If you define a function without sensible argument defaults, the model can call the right function and still return nothing useful. Cactus documents this directly: a required argument with no default and no evidence in the request gets withheld rather than guessed. Adding defaults to your schema fixes it.

Messy input text degrades extraction

Clean invoice descriptions extract cleanly, but rambling chat-style text can produce empty fields. The shape of your input matters, so preprocessing counts.

Classification is a weaker fit

Extraction generalizes to classification using enums, but if sorting items into categories is your main workload, Needle 3 is not the strongest option; the video notes a larger, classification-focused model from a separate vendor as better suited to that job.

What This Means for Automation Workloads

The practical takeaway survives even if you never install the model. Jobs that need the same small judgment repeated a thousand times a day, such as pulling fields out of every incoming inquiry, do not need the largest available model. A small model fine-tuned on a few hundred of your examples handles them offline, instantly, and at zero marginal cost, while the expensive frontier model gets called only when general reasoning is genuinely required. That division of labor, rather than Needle 3 specifically, is the shift the September 2026 release points to.

For developers, the entry path is straightforward: the Cactus GitHub repository hosts the engine and weights under Apache 2.0, and the vendor reports training used 360 billion tokens of proprietary structured data focused on one kind of task done properly rather than broad chat ability. Reset state between independent calls, define defaults for every argument, and use the confidence buckets before wiring any output directly into production systems.

FAQ

• Is Cactus Needle 3 free to use commercially? Yes. The model ships under the Apache 2.0 license, which permits commercial use, modification, and shipping inside products. The engine and weights are available from the Cactus GitHub repository.

• How fast is Cactus Needle 3? Cactus reports a single tool call in 66 milliseconds on a CPU with no GPU and no internet, and two function calls in under 97 milliseconds. On a Raspberry Pi 5 the vendor reports 400 to 4,000 tokens per second decoding. These are vendor-measured figures for structured output tasks.

• Can Needle 3 answer general questions like a chatbot? No. Cactus gave up general conversational ability by design. Ask for the capital of France and it returns nothing because no tool covers it. It is built for tool calls, structured extraction, and embeddings only.

• Does Needle 3 require an internet connection? No. It runs entirely on local hardware including phones, laptops, Raspberry Pi, and browsers via WebAssembly. Note that telemetry is on by default in the binary and is disabled via two environment variables documented by Cactus.

• Is the benchmark evidence independent? No. The published results come from Cactus itself across six suites including BFCL v4 and DroidCall. The comparison models ran at full precision under vLLM while Needle 3 ran as its shipped 2-bit file, but the measurements have not been independently reproduced.

Source video