{"id":425,"date":"2026-02-17T02:53:19","date_gmt":"2026-02-16T18:53:19","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=425"},"modified":"2026-02-17T02:53:19","modified_gmt":"2026-02-16T18:53:19","slug":"alibaba-qwen-team-releases-qwen3-5-397b-moe-model-with-17b-active-parameters-and-1m-token-context-for-ai-agents","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=425","title":{"rendered":"Alibaba Qwen Team Releases Qwen3.5-397B MoE Model with 17B Active Parameters and 1M Token Context for AI agents"},"content":{"rendered":"

Alibaba Cloud just updated the open-source landscape. Today, the Qwen team released Qwen3.5<\/strong>, the newest generation of their large language model (LLM) family. The most powerful version is Qwen3.5-397B-A17B<\/strong>. This model is a sparse Mixture-of-Experts (MoE) system. It combines massive reasoning power with high efficiency.<\/p>\n

Qwen3.5 is a native vision-language model. It is designed specifically for AI agents. It can see, code, and reason across 201<\/strong> languages.<\/p>\n

\n
\"\"
https:\/\/qwen.ai\/blog?id=qwen3.5<\/figcaption><\/figure>\n<\/div>\n

The Core Architecture: 397B Total, 17B Active<\/strong><\/h3>\n

The technical specifications of Qwen3.5-397B-A17B<\/strong> are impressive. The model contains 397B<\/strong> total parameters. However, it uses a sparse MoE design. This means it only activates 17B<\/strong> parameters during any single forward pass.<\/p>\n

This 17B<\/strong> activation count is the most important number for devs. It allows the model to provide the intelligence of a 400B<\/strong> model. But it runs with the speed of a much smaller model. The Qwen team reports a 8.6x<\/strong> to 19.0x<\/strong> increase in decoding throughput compared to previous generations. This efficiency solves the high cost of running large-scale AI.<\/p>\n

\n
\"\"
https:\/\/qwen.ai\/blog?id=qwen3.5<\/figcaption><\/figure>\n<\/div>\n

Efficient Hybrid Architecture: Gated Delta Networks<\/strong><\/h3>\n

Qwen3.5 does not use a standard Transformer design. It uses an \u2018Efficient Hybrid Architecture.\u2019 Most LLMs rely only on Attention mechanisms. These can become slow with long text. Qwen3.5 combines Gated Delta Networks<\/strong> (linear attention) with Mixture-of-Experts (MoE)<\/strong>.<\/p>\n

The model consists of 60<\/strong> layers. The hidden dimension size is 4,096<\/strong>. These layers follow a specific \u2018Hidden Layout.\u2019 The layout groups layers into sets of 4<\/strong>.<\/p>\n