TL;DR
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A November 2, 2026 blog post by four researchers listed with MIT CSAIL reports that GPT-6 Astra altered 0.016% of Qwen3.5-0.8B’s weights without gradient descent. On MATH-500, the edited model’s score rose from 7.9% to 48.1%, while answers hitting the token limit fell from 93.0% to 33.1%. The work is an early, non-peer-reviewed result, and the authors say fixing loops is not the same as improving mathematical ability.
The blog, titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” describes eight experiments. The Qwen math repair drew attention because the small model often repeated itself until it reached its output limit. Qwen’s model card already warns that the model is prone to thinking loops, according to the post.
In the reported edit, Astra identified a direction associated with looping in the MLP weights of layers 15 and 19. The researchers subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or 0.016% of the model’s 0.8 billion weights. The edited model on Hugging Face has changes only in those two layers’ down-projection tensors, and the post provides hashes for comparison.
The work used an iterative search rather than gradient descent. Published logs record 158 turns, 80 Python runs and 50 forward passes over 20.3 hours. The researchers supplied practice problems and barred Astra from seeing the final benchmark questions. At the smaller budgets of 10 and 25 forward passes, the reported MATH-500 scores were 12.3% and 14.1%. The authors say the loops were reduced, not eliminated: a third of edited-model answers still reached the token limit.
AI Repairs AI Inside Another Model’s Weights
GPT-6 Astra edited a small share of Qwen3.5-0.8B’s weights to reduce repetitive loops. Its MATH-500 score rose, but the result is an early experiment, not proof of broader math gains.
What Astra changed
The edit targeted a looping behavior in two layers of the open-weight Qwen model.
Astra used test feedback to identify a direction associated with looping in the MLP weights.
The direction was subtracted from output columns in layers 15 and 19.
The researchers report 131,072 changed weights, without gradient descent or loss-based fitting.
The result—and its limits
- MATH-500 score increased from 7.9% to 48.1%.
- Answers reaching the token limit fell from 93.0% to 33.1%.
- Loops persisted in a substantial share of answers.
- The result does not establish broader math or other-task gains.
- Independent replication is not reported.
Evidence to watch
The blog describes eight experiments and calls the work very early; one separate attempt did not succeed.
- Independent replication of the reported benchmark result.
- Tests on additional benchmarks, models, and looping behaviors.
- Checks for effects on capabilities beyond repetitive output.
- Results under smaller evaluation budgets, which produced lower scores.
A Small Edit, A Larger Score
The experiment suggests a model may be able to inspect and alter another model’s weights using test feedback, without the usual gradient-based training process. If the method holds up across further work, it could offer developers another way to diagnose and patch specific behaviors in models whose weights they can access.
For now, the score gain needs to be read alongside the underlying failure mode. The base model frequently exhausted its token budget in loops, and the edit was aimed at that weakness. The authors explicitly state that “Fixing loops, however, is not the same as making a model better at math.” The benchmark result is a concrete reported outcome, but it does not establish broad reasoning improvements or performance on other tasks.
The method also took substantial computation and repeated evaluation. “Without training” in this account means no gradient descent or fitting weights to a loss; it does not mean an effortless or evaluation-free change. The reported 50-pass run required more time and feedback than the smaller runs, which produced much lower scores.
Products Worth Considering
Logitech MX Brio Ultra HD 4K Webcam, Graphite
Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
As an affiliate, we earn on qualifying purchases.
How The Qwen Trial Worked
The experiment is one of eight in the researchers’ post, which also describes attempts to infer how small models were trained and to repair a car-driving policy. One experiment did not succeed: the post says Astra could not identify, from weights alone, which of two fine-tuned Gemma models had been trained to flatter users.
According to the logs, Astra ran as gpt-6-astra through the Codex command-line tool at the highest reasoning setting. The researchers’ harness gave it a Python environment to inspect and edit the 24-layer Qwen model, but Astra could not run the model directly. It had a limited forward-pass budget; each pass allowed answers to up to 512 problems or processing of up to 1,024 texts. The team supplied practice questions from GSM8K and MATH, while withholding the final test sets.
The researchers describe the blog as “very early results” and say the capabilities remain limited. It is not a peer-reviewed paper and, according to the supplied source material, was not posted on arXiv. The post lists four authors affiliated with MIT CSAIL; it does not list OpenAI as an author or report a statement from the company about the experiment.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
What The Results Do Not Show
The blog reports a large MATH-500 score change, but the available source material does not establish whether independent researchers have replicated the result or whether the edit improves performance beyond the reported benchmark. The score is from a blog experiment rather than peer-reviewed research.
It is also unclear how reliably the method would work on other models, tasks or looping behaviors. The reported reduction leaves 33.1% of MATH-500 answers hitting the token limit. The post’s other experiments include a failure, and the smaller forward-pass budgets yielded much lower scores, indicating that results depended on substantial iterative feedback.
The edit changes the model itself. Anyone using the patched weights would be running a modified version of Qwen3.5-0.8B, and the reported benchmark does not establish whether the patch affects other capabilities or introduces different weaknesses. Those effects remain unreported in the supplied material.
Replication And Broader Tests
The next useful evidence would be independent replication, tests on additional benchmarks, and checks for changes to capabilities beyond repetitive output. The researchers’ published logs, model files and weight hashes give others material to inspect, but the source does not report a follow-up study or an announced review date.
Further work would also need to show how the approach performs with different models and more constrained evaluation budgets. For now, the reported result is a specific repair attempt: Astra edited two layers of an open-weight model and improved its score on one named test, while the looping problem persisted in a substantial share of answers.
Key Questions
What did GPT-6 Astra change?
According to the researchers’ post, Astra identified a direction associated with looping in Qwen3.5-0.8B and removed it from output columns in layers 15 and 19, changing 131,072 weights.
How much did the MATH-500 score change?
The post reports that the score rose from 7.9% for the original model to 48.1% for the edited model. It also reports that answers hitting the 32,768-token limit fell from 93.0% to 33.1%.
Was the model trained?
The researchers say they did not use gradient descent, backward passes or loss-based weight fitting. Astra instead used iterative testing and weight edits, so “without training” does not mean without repeated evaluation or substantial computation.
Are the results peer-reviewed?
No peer review is reported in the supplied source. The results appeared in a research blog post on November 2, 2026, which the authors call very early work.
Did the edit eliminate looping?
No. The reported share of MATH-500 answers reaching the token limit decreased from 93.0% to 33.1%, so the problem remained in a substantial fraction of answers.
Source: Thorsten Meyer AI
