A 120-billion-parameter AI model, running privately on one ordinary PC, faster than the standard tool.

The one-paragraph version

Neural runs OpenAI's gpt-oss-120B, a very large AI model with about 117 billion adjustable settings (called parameters), on one gaming PC with a 12 GiB graphics card and 64 GB of RAM (the computer's main working memory). It runs on this PC with no account or subscription. In a fresh October 1 comparison using four identical API requests in alternating order, Neural generated answers at a weighted 24.69 tokens per second; llama.cpp reached 14.32. Neural's measured rate was 1.72 times higher in this engine comparison. A separate, matched Neural software change improved decode speed by 1.74% with normal adaptive placement; a frozen-placement control measured 2.89% but is not a recommended setting. The comparison used the existing model files and did not change their precision.

What was achieved

The October 1 figures below use fresh Neural and llama.cpp sessions in Neural / llama.cpp / llama.cpp / Neural order, with the same four API request bodies, four answers capped at 512 tokens per session, and two sessions per engine. They are the latest comparison. The September 30 figures farther down are retained as historical measurements; they are not the current headline.

For these exact requests, Neural's weighted generated-answer rate was 24.6910 tokens per second; llama.cpp's was 14.3189. The first-answer rates were 23.6047 and 13.0566; follow-up rates were 25.0756 and 14.7957. The first request's client wall time was 30.38 seconds for Neural and 176.82 for llama.cpp; on follow-ups it was 22.835 and 39.405 seconds. First prompt-processing medians were 8.66645 and 137.552015 seconds (15.87 times faster for Neural in this fresh-session test). That is not the separate warm-prompt study. Prompt settings were chosen for answer speed, not tuned separately for llama.cpp prompt reading.

Speed is counted in tokens per second, but the programs count slightly differently: Neural times 511 tokens of a 512-token answer, while llama.cpp reports 512. Their weighted rates use each program's reported count. Same API bodies and answer lengths do not prove that the two backends rendered identical internal prompt tokens.

The new CPU worker change improved Neural's measured weighted decode rate by 1.74% with normal adaptive placement; all corresponding answer token IDs matched in the two-session CPU-only comparison. A frozen-placement control measured 2.885%, with matching outputs, but freezing placement made the server slower overall and is not recommended. The v10 launcher uses this CPU worker with prompt grouping limited to 1,024–4,096 new tokens and releases unused graphics-memory cache after prompt processing. In a separate first-pass study, that profile lowered warm processing time for a 3,000-token file prompt by 7.03% (3.69655 to 3.43665 seconds) and released about 1.11 GiB of graphics-memory reservation after a long prompt. That is allocator space returned to the system, not a smaller model. These are separate measurements, not extra gains in the engine comparison above.

Earlier September 30 measurements (kept for context, not current results):

The September 30 first-answer advantage ranged from 1.84 to 2.08 times across runs, and llama.cpp was slower than usual in that round. That older run's 1.57 times follow-up result is superseded by the October 1 comparison above.

Both October 1 Neural sessions produced matching prompt and generated-token IDs for all four answers. The comparison used the same PC, alternating fresh sessions, and the same request bodies. First requests still depend on run order and on what the PC's file cache held, and each program's own timer is used. See the Trust section for the scope of the output checks.

October 2 batch-execution research

A separate experiment processed 16 independent coding requests together. A clean split-graph screen produced 42,639 tokens in 1,454.035 seconds, or 29.3246 aggregate tokens per second. That is the total across the cohort, including backend prompt processing; it is not the speed of one conversation or a measurement of the editor-to-answer experience.

The same-output historical batch runs reached 23.595 and 23.623 aggregate tokens per second. The new screen was about 24.2% higher in throughput, calculated from the measured clocks. It is one later run against historical results, not a fresh repeated alternating comparison. The historical adaptive runs used different outputs and token counts, so they are excluded from this same-output chart and do not support a confirmed quality-equivalent gain.

The strict coding-task quality gate is not cleared. Frozen static review found paired rubric regressions in the final coding responses. That review did not execute the generated code and is not a general model-quality evaluation. Matching a numerical reference preserves the reference's answers without proving their quality. No production or end-to-end improvement is confirmed, no research change is promoted to the normal server. The October 1 production comparison above remains separate.

The grouped version's full natural numerical audit passed 42,623 decode checks across 16 natural finals. Its separate timing did not complete: the first screen was interrupted by a Windows reboot, and an identical retry failed before generation at the host-allocation RAM-reserve guard. No grouped timing result is available, and the split timing repeat was deferred. Research is paused. See the separate research chart and raw evidence or the method and limits. The model and its precision were not changed.

A very large model, privately, on one PC

The model takes 61 GiB on disk (about 65 GB), roughly the size of a large game install. The graphics card holds 12 GiB, so the model is about five times too big for it (calculated from 61 and 12). Neural writes at 23 to 25 tokens per second on the latest version, several times faster than most people read.

It speaks the OpenAI chat-completions interface (streaming and tool calls), so tools such as Continue in VS Code can connect to it. Not everything the cloud offers is supported (no images, no Responses API, one tool call per response), and Cursor's built-in agent cannot use local models. It is designed to keep everything on the machine; its network traffic has not been audited.

We have not compared running costs against a cloud service, so we make no cost claim here. What the work shows is a capability: a model of this size, at these speeds, on hardware many people already own.

Send the work to the data, not the data to the work

The model is built from many small specialist networks, called “experts”. Picture a hospital with 36 floors (the model's “layers”) and 128 specialists on each floor. Each token, roughly three quarters of a word, is handled by only 4 per floor. That makes 4,608 specialists in total and 144 used per token, about 3% (a calculation).

The busiest specialists sit in the graphics card's own memory: about 500 of them on the current selected profile, a configuration choice. That is roughly 1 in 10 of the 4,608 (a calculation). The rest stay in the PC's main memory (RAM). The PC's main chip (the processor) uses its eight cores to consult them there, while the card handles its own share at the same time.

The alternative is to copy each missing specialist to the card, about 13 megabytes each time, roughly a large photo. Instead, for each layer, the per-token messages between card and processor are about 23 kilobytes, and the processor does the work where the data sits. A few popular experts are still copied to the card in the background, and long prompts stage experts too. The processor reads RAM at about 36 GB per second on the September 30 build, against a measured memory ceiling of about 38 to 40 GB per second.

For long inputs, Neural works through the model one floor at a time, so each specialist not on the card is fetched once per floor instead of once per block of the prompt. Earlier changes cut a 13,000-token prompt from 15.0 to 10.7 seconds, then to 7.7 seconds in like-for-like runs; the published September comparison recorded 6.8 seconds. The current selected profile adds bounded prompt grouping only for 1,024 to 4,096 new tokens and releases unused graphics-memory cache after prompt processing. Grouping is off for longer prompts. These prompt changes preserve the prompt result bit for bit.

How Neural differs from llama.cpp

llama.cpp is mature and runs on many kinds of computers, which Neural does not (see Limits). The October comparison used llama.cpp build 10361 (revision 14e78ddef) with -ngl 99 -ncmoe 31 -t 8 -c 16384 -fa on --jinja, a setting tuned for decode speed on this PC (-ncmoe 31 keeps the specialists of 31 of the 36 layers in RAM), and it computes them there on the processor. That setting was not tuned for prompt reading. Neural does the same, but it decides for each individual specialist, not for whole layers. It also keeps the ones used most recently on the card, so the choice follows the current conversation.

The gain differs by activity. In September's warm-prompt measurements, llama.cpp took 93 to 102 seconds for the same prompt. The October 1 fresh-session median was 137.55 seconds versus Neural's 8.67; this uses a different method. We did not profile llama.cpp's prompt path or try a prompt-tuned setting, so the prompt ratio needs particular care. Answers gain less, because both are limited by how fast RAM can be read.

In August our research version lost to llama.cpp, at 13.65 against 13.90 tokens per second. That version copied 1.84 times less data than llama.cpp, but it copied it over the connection between the PC and the graphics card, which is slow. llama.cpp avoided that trip by computing the data where it already sat. The design then changed to a hybrid: for every token, the processor computes the experts held in RAM while the card computes the resident ones. Three research steps took that hybrid from 0.82 times llama.cpp to about 1.5 times on short prompts, and the server versions started from there.

How we checked it

• The October 1 CPU-only and frozen-placement comparisons matched every corresponding answer token ID. The final selected Neural repeats also matched. One earlier combined-profile run differed in the last two outputs of one session, so its strict all-answer identity check failed.
• A few changes are not bit-identical: the larger expert pool from the rolling K/V cache, the halved admission rate, and the fused core's different summation order. Answers can therefore differ from earlier versions, as this server's answers already can between two runs, because which specialists sit on the card depends on timing.
• Neural uses OpenAI's published 4-bit (MXFP4) expert weights with the same values (the packed store re-encodes only the scale bytes, losslessly), and nothing is re-quantized or pruned.
• New checks covered 432 native-kernel cases and 192 full-model decode steps per implementation across three prompt sizes; the tested logits, hidden states, and K/V cache matched. The selected repository test run passed 103 tests. These checks cover the named cases and settings; they are not proof of universal answer equivalence.
• The fresh engine comparison used the same API request bodies, but llama.cpp does not expose its internal prompt-token IDs in this test. We make no claim that different model backends are quality-equivalent.
• Ratios are tied to the specific machine, settings, workload and date stated beside them.
• Where a measurement was not clean, the row says “not measured” instead of showing a number.
• Failed experiments are kept on record, and reviews corrected our own earlier claims.

What it does not do

• It is one PC and one model: Windows, an NVIDIA card with 12 GB or more, and a recent Intel or AMD processor that supports a set of fast maths instructions called AVX-512.
• It serves one conversation at a time.
• It can hold 16,384 tokens (roughly 12,000 words, calculated at 0.75 word per token) of conversation at once, and it takes about 2 minutes to start.
• The expert store (about 55–57 GiB) only just fits in 64 GB next to other programs; the first answer touches some experts for the first time and can also hit disk stalls when free RAM is low (about 55 GiB free is recommended). First-answer speed therefore varies more than follow-ups.
• The advantage was measured only with the expert store held in RAM. In an earlier research build under RAM pressure, Neural fell to 0.31× llama.cpp (un-replicated). On Sep 5 an earlier build was 2.92× slower than llama.cpp on fresh prompts under background load.
• Prompt-reading speed on this PC also drifted between days on an unchanged build in September, for reasons not established. October's prompt measurements are a separate experiment and do not remove that uncertainty.
• It is a personal server, not a product, and the code is private.

The story in brief

1. In August the research said no: it reached 0.98 times llama.cpp's speed, and the best it could theoretically reach was 1.17 times (a calculation). The project's own minimum worth building was 1.20 times.
2. The design changed to a hybrid: for every token, the processor computes the experts held in RAM while the card computes the resident ones.
3. Over Sep 29 and 30, six passes of measured changes improved prompt processing; those values remain historical.
4. On Oct 1, a matched fresh-session comparison measured 1.72 times llama.cpp's weighted answer rate, and a separate CPU software change measured a 1.74% decode gain with normal adaptive placement.
5. Prompt and decode speed are different measurements. The new prompt grouping helps selected 1k–4k prompts; longer prompts and first-answer speed remain open areas for measurement.

What would move it further

The October 1 CPU result shows that software still had measurable room: a persistent worker team reduced some per-token overhead and improved decode by 1.74% in the normal adaptive comparison. More software opportunities remain, but they are hypotheses until tested in serving workloads. The selected prompt profile helps 1,024–4,096-token inputs; longer prompts did not benefit from grouping. Faster memory or a graphics card with more memory may also help, but no hardware purchase is needed for the software gains reported here and this page makes no purchase recommendation.

For the full measurements, see the October 1 decode follow-up, the optimization review, the decode evidence and reports, and the selected-profile launcher.