Tencent Hy4 Preview: The Open Model Built for Long, Messy Work

Imagine handing an AI assistant an entire code repository, several bug reports, a product brief, and a spreadsheet, then asking it to carry the work through several revisions. Producing one plausible answer is not the hard part; preserving the plan and evidence as the task grows is. Tencent Hy4 preview, released on August 28, 2026, is aimed at that long-running kind of work across coding, office analysis, game development, and scientific research.

Hy4 preview is a large language model (LLM), software trained to generate text from learned patterns. Its headline figures are 770 billion total parameters, 49 billion active parameters per token, and a context window of about one million tokens. The 770B number is loud, but the way the model uses that capacity is more interesting. (tencent.com)

Start with the numbers, not the hype

A parameter is a learned numerical weight inside a model. During training, those weights are adjusted until the system becomes better at predicting useful responses. A large parameter count signals capacity, but it does not guarantee that every answer will be correct.

A token is a small piece of text. It might be a whole short word, part of a longer word, punctuation, or a space. A context window is the amount of text a model can consider in one request, including instructions, documents, tool results, and its own response.

What does a one-million-token context window change for a developer? It can keep a substantial codebase, logs, design notes, and reference documents in one working context instead of forcing you to summarize and reload them repeatedly. The 49B active-parameter figure describes computation: Hy4 does not use every component for every token. That helps control the work per step, but the full model still requires serious memory to run locally.

Why Hy4 is a committee of specialists

Hy4 uses a mixture-of-experts (MoE) architecture. In an MoE model, a router directs each token to a few specialist sub-networks, called experts, instead of sending every token through the entire model. Its published design has 78 layers, including 77 MoE layers; each of those layers contains 256 routed experts and one shared expert, with eight routed experts selected for each token.

Think of a large workshop. It may employ hundreds of specialists, but a repair order does not summon everyone to the same workbench. The router picks the people whose skills fit that particular step. Hy4 keeps broad capacity while concentrating computation on a smaller working group.

Under the hood, the model uses a gated sparse attention design. Attention is the mechanism that helps a language model decide which earlier pieces of text deserve focus; sparse attention avoids treating every old token as equally important. Hy4 also uses IndexCache to reuse some indexing work across layers, choices intended to make long-context processing more practical. (github.com)

Built for work that sprawls

Software engineering is the clearest target. Hy4 is designed to understand a large codebase, make a plan, debug failures, and check whether a proposed fix holds together. For front-end work, Tencent also points to better visual quality and interaction design, not only code that passes a parser.

The same focus appears in office and analysis tasks. Hy4 is meant to gather information scattered across documents, analyze data and financial models, then turn the result into a document, spreadsheet, or presentation. An AI coding agent is more than a chatbot: it can inspect files, call tools, make changes, and continue from the results.

Game development and scientific research round out the target areas. The model can create a playable prototype from a natural-language request and continue refining it with a game engine. Its research focus includes artificial intelligence, molecular dynamics—the computer simulation of atoms and molecules—condensed-matter physics, and fundamental mathematics.

The self-improvement claim needs careful reading

Tencent says Hy4 participated in automated experiments involving training methods, data strategies, evaluation frameworks, and low-level operators. The model proposed changes, ran experiments, examined logs and feedback, and helped decide what to try next. That forms an early recursive optimization loop: a model is placed inside a controlled process where experiment results guide the next round.

This is different from a model independently rewriting its own goals or weights without oversight. The more useful lesson is that language models can assist with parts of model development when experiments are repeatable and their results can be measured.

Tencent also reports a 31.8% increase in end-to-end inference throughput over a baseline. Inference means generating an answer after training, while throughput measures how much work a system completes over time. The reported gains came from operator fusion, which combines several computation steps, and communication optimization. Actual performance will depend on hardware, context length, batch size, and serving software.

What running Hy4 looks like

For a first experiment, hosted access is more practical than local deployment. Tencent is exposing the model through products such as WorkBuddy and CodeBuddy, while developers can reach it through hosted application programming interfaces (APIs), including Tencent Cloud TokenHub and OpenRouter.

The open release includes model weights, code, and deployment guidance under the Apache License 2.0, a permissive open-source license. Local serving is still a serious infrastructure project. Tencent’s published recipes use vLLM or SGLang, which are inference engines, an FP8 quantized checkpoint, and tensor parallelism across eight GPUs. FP8 stores values in an 8-bit floating-point format, while tensor parallelism splits the model’s work across several GPUs.

Hy4 includes a native multi-token prediction (MTP) layer for speculative decoding. In that technique, a draft step suggests several upcoming tokens and the main model verifies them together, reducing some repeated work. Once a provider or local endpoint is configured, the call pattern looks like this:

from openai import OpenAI

client = OpenAI(api_key='YOUR_PROVIDER_KEY')

response = client.chat.completions.create(
 model='hy4-preview',
 messages=[
 {'role': 'user', 'content': 'Review this failing test and propose a fix.'}
 ],
 temperature=0.9
)

print(response.choices[0].message.content)

The published setup defaults to a higher-effort reasoning mode for difficult coding, mathematics, and planning tasks. A no-think option is available for shorter, more direct responses, which can reduce delay and output cost. (tencent.com)

Promising results, with preview-stage trade-offs

In Tencent’s blind internal evaluation, 163 experts rated outputs on 203 engineering tasks. Hy4 preview averaged 2.99 out of 4.00, compared with 2.92 for GLM-5.3 and 2.94 for Kimi K3. That is a useful signal about the work Tencent cares about, but it is not an independent public leaderboard, so it should be treated as directional rather than universal.

The listed API prices are $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million tokens for cache hits. A cache hit is repeated input that a provider can reuse instead of processing from scratch. At those rates, 10 million input tokens plus 2 million output tokens would cost about $13.34 before cache savings.

The preview label matters. Tencent’s model notes acknowledge that Hy4 can spend longer than necessary reasoning through difficult tasks and may over-verify its own work. A million-token context is a large desk, not a guarantee that every document on the desk will be read correctly.

Hy4 preview is interesting because it points open models toward sustained work: code that changes across files, documents that become finished artifacts, and research tasks that require connected steps. The lasting test will not be the model’s size alone, but whether it can keep its footing while the work becomes messy, tool-filled, and real.