The short answer
- Weights are the numbers inside a model that set how it turns input into output, and training normally changes them.
- A November 2026 arXiv preprint reports a 27B model that diagnosed and edited its own weights, gaining 6.6 points on unseen problems.
- The authors say the gain comes without gradient updates or a training run, and that it adds no cost per answer.
- The paper tests one model family at four sizes, and only the largest size showed the effect.
- The preprint has not been peer reviewed, and critics dispute the related MIT CSAIL result.
Language models rewriting their own weights means a model reads the numbers that define how it works, proposes changes to them and tests whether the edited version performs better. A preprint posted in mid-November 2026 reports that a 27-billion-parameter model did this and improved on problems it had never seen, which is why it is drawing attention. As of November 21, 2026, the paper has not been peer reviewed, and its claims are still being tested.
Why it's trending
The trigger is an arXiv preprint, listed as version 2 posted on 18 Nov 2026, by Jiajun Fan and colleagues. The listed affiliations include UIUC, CMU, Rutgers, Princeton, Stanford and Yale. The authors claim, to their knowledge for the first time, that a language model rewrites its own weights through its own reasoning and becomes better, with no gradients and no training. They call this "self-rewriting."
Coverage so far comes mainly from AI news sites and blogs. As of November 21, 2026, no wire service, national broadcaster or major newspaper coverage had been identified.
Interest also builds on an earlier MIT CSAIL blog, published 2 Nov 2026, titled Sparks of Weight-Space Programming. A LessWrong critique posted on 17 Nov 2026 says that blog post has received wide attention, and that a tweet by its lead author has been viewed nearly 10 million times as of November 2026.
What it means for a language model to rewrite its own weights
Weights are the numbers inside a model that determine how it turns an input into an output. Training, including fine-tuning, adjusts them. Other methods, such as better prompts and extra computing at answer time, leave the weights unchanged.
Self-rewriting adds a loop in which the model reads its own weights, diagnoses its own mistakes, edits the numbers and tests the result. The authors describe the model as its own optimizer.
How the 27B experiment worked
Edit, test, submit
The main setup gives an open 27B model its weights, its graded answers and the ability to run itself. The model diagnoses where it fails, proposes an edit to the weights behind that failure and tests each edit on practice problems. It submits only edits that work. The headline result is a 6.6-point gain in accuracy on problems it never saw.
The damaged-copy repair
In a second test, the authors damaged a copy of the model so that it solved almost nothing. The model then repaired the copy without retraining and without reading the original weights. The paper reports that the repaired copy reached over 80% of the original accuracy, but that figure is for image tasks the model had not seen, not for language tasks.
Repeated sessions
The paper reports that over eight consecutive sessions, accuracy only rose. The figure caption says each session starts from the base weights, but every submitted edit replaces the model in use, so any build-up happens through those edits. A blog summary of the preprint describes each session as restarting from the base weights and calls the process "not cumulative." The two accounts differ, so readers should treat the paper's description of its own setup with care.
Why scale matters
The ability appeared only at the largest size. The paper tested four sizes from one model family: 2B, 4B, 9B and 27B parameters. The 2B model did not rewrite its weights, and the 4B and 9B models degraded their performance because of random changes. On held-out problems, the self-edited 9B model solved none, against 47.5% for its base model. Because the finding rests on one family at four sizes, it should not be generalised to other models.
Why researchers are trying it
The reasons below are the authors' arguments, not settled findings.
- Cost per answer: the authors argue that, unlike test-time scaling, self-rewriting writes its gain into the weights at no extra cost per answer.
- Training: they say that, unlike training, the method runs only forward passes, as serving does.
- Long-term aim: they describe a model that does not separate training from inference and could keep upgrading itself, which they frame as a route to recursive self-improvement.
An O'Reilly Radar article from 19 Nov 2026 describes recursive self-improvement as AI improving itself, with each better version improving itself again. The fetched page was cut off, so these details come from search-result excerpts of the article. The article says self-improvement works best in code and math, where a test can say right or wrong. It also warns that methods that modify a model's own brain or code can bake in mistakes and break the checks meant to catch them.
Earlier work and critiques
The Sparks result (MIT CSAIL, Nov 2026)
In the MIT CSAIL blog, a frontier model called GPT-6 Astra edited about 0.016% of the weights of a small 0.8B-parameter model. The blog reports that its MATH-500 score rose from 7.9% to 48.1%, and it attributes the gain to fixing a reasoning-loop problem rather than broad new math skill. In a separate experiment, Astra edited 3.5% of a failing control policy's weights, taking it from 0% to 100% success.
The result is disputed. A LessWrong critique by researchers at UC San Diego argues that Astra's edit is a special case of activation steering combined with synthetic data, not native weight editing. In their reproduction, accuracy was 12.3% with an 87.0% loop rate without steering, compared with 57.7% accuracy and a 29.5% loop rate with Astra's steering. Critics also point out that the Sparks setups blend tool-free and tool-using results, since the authors allowed a Python kernel in one math experiment.
What the self-rewriting paper does and does not show
The paper tests a 27B model in a controlled setup. It does not show general self-improvement in deployed systems. Its claim to be the first of its kind is the authors' own, made to their knowledge.
What happens next
As of November 21, 2026, no scheduled events had been confirmed. Peer review, a code release and independent replication have not been confirmed either. Readers can watch for the following:
- Peer review: the preprint has not been peer reviewed, and no publication date has been confirmed.
- Replication: independent tests of the 6.6-point result and the damaged-copy repair would show whether the findings hold.
- Responses to the activation-steering critique, which could clarify whether the Sparks result is native weight editing.
The arXiv listing is the place to check for new versions of the paper.
Questions people ask
What does it mean for a language model to rewrite its own weights?
It means the model reads its own weights, proposes edits to the numbers that shape its behaviour, tests those edits and keeps the ones that help. The paper says this happens without gradient updates or a training run. The authors describe the model as its own optimizer.
Why would researchers try this?
The authors argue that the gain is written into the weights, so it adds no cost per answer, unlike extra reasoning at answer time. They also say it uses only forward passes, the same computation used in serving, and they frame it as a route to recursive self-improvement.
Has the result been confirmed?
As of November 21, 2026, not yet. The self-rewriting paper is an arXiv preprint, version 2 posted on 18 Nov 2026, and it has not been peer reviewed. The earlier MIT CSAIL result behind it is disputed by critics who argue it reflects activation steering rather than native weight editing.
Does this mean AI models are now improving themselves in everyday products?
The paper tests a 27B model in a controlled setup, and its effect appeared only at the largest of four sizes in one model family. It does not show general self-improvement in deployed systems.
Sources
- Language Models Can Rewrite Their Own Weights — arXivarxiv.org
- Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights — Research blog (Takehi, Teoh, Zhang, Khattab)rikiyat.github.io
- A closer look at Sparks of Weight-Space Programming — LessWronglesswrong.com
- Who Builds the Next Model? — O'Reilly Radaroreilly.com



