AIHOT 于 2026-08-17 收录了“当模型持续学习：测试时训练如何改变 AI 的记忆与成本”这一公开动态。以下先呈现从来源页面抓取的正文，再给出 AIHOT 摘要与 TopoReduce 编辑解读。
PUBLIC SOURCE CONTENT
已抓取公开正文公开原文内容
When Models Learn | Tomasz Tunguz
In short : Explains test-time training through the analogy of a GPS learning a persistent shortcut around daily traffic rather than a one-time reroute: the model takes a gradient step on the prompt it's answering, so its weights change as it works. Traces three implications, flat memory instead of a linearly growing KV-cache, the provider cost of serving a separate model per user, & faster inference, then states the tension as a tradeoff between serving long context and serving many people, & grounds it in concrete use cases, a coding agent that earns back its per-user cost over a long session versus a one-off query a shared frozen model handles just as well.
Every model you’ve ever used froze the day its training ended. The answers are the same even if you have used it every day.
What if a model kept learning as you use it?
A GPS learns a persistent shortcut around daily traffic on northbound Highway 101, not just a one-time reroute. Test-time training does that to a model as it works.1 As you use the AI, the model changes its weights, changes how it thinks about its memories, to answer you better.
The changes are more profound than finding an off-ramp to an access road past a highway junction chokepoint.
Memory requirements plummet. A standard transformer keeps a KV-cache, a running record of every earlier token, so its memory grows linearly with context, every additional token adds to the running record. Test-time training folds that history into a fixed-size set of weights instead of a growing cache, so memory stays flat no matter how long the conversation runs.
The model provider now has to serve a separate model to each person. Once a model updates on your prompt, it is no longer the model that answered your neighbor’s, so a single checkpoint serving millions of users becomes millions of slightly different models, each shaped by the person using it. That divergence is the provider’s problem to solve: a GPU provider needs a copy in flight per user instead of one shared copy for everyone, which means more compute, more chips, to serve the same number of people.
It’s much faster. Stanford research on small models indicates it can be up to 2.7 times faster, because a test-time trained model’s inference latency stays constant no matter how long the context runs, the way a standard transformer’s does not.2 In-Place TTT also ships drop-in, lifting a 4b model to competitive 128k-context performance with no retraining.3
Here is the tension. Standard AI is limited by memory, test-time AI is limited by compute & chips, so a provider picks based on whether it’s serving long context or serving many people.
That cost is only worth paying where personalization earns its keep. A coding agent that learns your codebase’s conventions, the resilient persistent bugs, ultimately should provide some form of lock-in via memory, so the per-user cost pays for itself. A one-off customer support question doesn’t need any of that. A shared, frozen, potentially fine-tuned model answers it just as well & costs the provider far less to serve.
Test-time training will be a key part of the discourse throughout the end of 2026 & beyond. It has the potential to change the current economics of AI.
-
Sun et al., Learning to (Learn at Test Time): RNNs with Expressive Hidden States ↩︎
-
End-to-End Test-Time Training for Long Context ↩︎
-
In-Place Test-Time Training ↩︎
Get the next one in your inbox
The 1-minute read that turns tech data into strategic advantage.
Read by 150k+ founders & operators.
Related Posts
Top 10 Posts of 2025
DECEMBER 3, 2025
Google's Cloud Revenue Converges to NVIDIA's Growth Rate
JULY 22, 2026
8.9 Million AI Users
JULY 9, 2026
GP at Theory Ventures. Former Google PM. Sharing data-driven insights on AI, web3, & venture capital.
Bloomberg
•
WSJ
•
Economist
Every model you’ve ever used froze the day its training ended. The answers are the same even if you have used it every day.
What if a model kept learning as you use it?
A GPS learns a persistent shortcut around daily traffic on northbound Highway 101, not just a one-time reroute. Test-time training does that to a model as it works.1 As you use the AI, the model changes its weights, changes how it thinks about its memories, to answer you better.
The changes are more profound than finding an off-ramp to an access road past a highway junction chokepoint.
Memory requirements plummet. A standard transformer keeps a KV-cache, a running record of every earlier token, so its memory grows linearly with context, every additional token adds to the running record. Test-time training folds that history into a fixed-size set of weights instead of a growing cache, so memory stays flat no matter how long the conversation runs.
The model provider now has to serve a separate model to each person. Once a model updates on your prompt, it is no longer the model that answered your neighbor’s, so a single checkpoint serving millions of users becomes millions of slightly different models, each shaped by the person using it. That divergence is the provider’s problem to solve: a GPU provider needs a copy in flight per user instead of one shared copy for everyone, which means more compute, more chips, to serve the same number of people.
It’s much faster. Stanford research on small models indicates it can be up to 2.7 times faster, because a test-time trained model’s inference latency stays constant no matter how long the context runs, the way a standard transformer’s does not.2 In-Place TTT also ships drop-in, lifting a 4b model to competitive 128k-context performance with no retraining.3
Here is the tension. Standard AI is limited by memory, test-time AI is limited by compute & chips, so a provider picks based on whether it’s serving long context or serving many people.
That cost is only worth paying where personalization earns its keep. A coding agent that learns your codebase’s conventions, the resilient persistent bugs, ultimately should provide some form of lock-in via memory, so the per-user cost pays for itself. A one-off customer support question doesn’t need any of that. A shared, frozen, potentially fine-tuned model answers it just as well & costs the provider far less to serve.
Test-time training will be a key part of the discourse throughout the end of 2026 & beyond. It has the potential to change the current economics of AI.
-
Sun et al., Learning to (Learn at Test Time): RNNs with Expressive Hidden States ↩︎
-
End-to-End Test-Time Training for Long Context ↩︎
-
In-Place Test-Time Training ↩︎
Get the next one in your inbox
The 1-minute read that turns tech data into strategic advantage.
Read by 150k+ founders & operators.
Related Posts
Top 10 Posts of 2025
DECEMBER 3, 2025
Google's Cloud Revenue Converges to NVIDIA's Growth Rate
JULY 22, 2026
8.9 Million AI Users
JULY 9, 2026
GP at Theory Ventures. Former Google PM. Sharing data-driven insights on AI, web3, & venture capital.
Bloomberg
•
WSJ
•
Economist
AIHOT 摘要
测试时训练（Test-time training）让模型在使用中持续更新权重，而非训练结束后冻结。相比标准 Transformer，其内存需求从随上下文线性增长变为恒定，斯坦福研究显示推理速度最高可提升 2.7 倍，且 In-Place TTT 无需重训即可将 4B 模型提升至 128k 上下文性能。但代价是每个用户需独立模型副本，服务成本转向算力与芯片，更适合编码助手等个性化场景。
为什么值得关注
TTT 把长上下文的内存压力转成按用户服务的算力压力，个性化只在编码代理这类能形成记忆黏性的场景才划算。
工程化解读
从 TopoReduce 的工程视角看，这条信息属于“模型与训练”主题。它的价值不只在于一个新产品或新观点本身，还在于说明 AI 系统正在如何影响模型接入、智能体协作、研发流程、基础设施和团队决策。实际采用前，应结合原文确认版本、适用范围、价格和运行条件。
- 发布时间：2026-08-17；AIHOT 分类：模型与训练。
- AIHOT 标签：
- AIHOT 判断：TTT 把长上下文的内存压力转成按用户服务的算力压力，个性化只在编码代理这类能形成记忆黏性的场景才划算。
- AIHOT 评分：48；评分用于站内排序，不等同于独立评测结论。
TopoReduce 编辑观察
当 AI 动态进入真实生产环境，团队需要同时关注能力边界、数据来源、调用成本、权限控制和可回滚性。把单条新闻放回完整工程链路中阅读，比只看标题更有助于判断它是否适合自己的产品和工作流。