Shanghai AI Lab’s 397B Open Model Tops MMLU-Pro — and Beats GPT-5.5 on Code

A new open-weight model from Shanghai AI Lab has just claimed the top spot in general reasoning among open-source systems — and its benchmark numbers against closed frontier models are raising eyebrows across the research community. InternS2-Preview-397B is a multimodal Mixture-of-Experts model released under the Apache 2.0 license, built specifically for scientific intelligence and long-horizon agentic tasks. The numbers are hard to dismiss: 89.75 on MMLU-Pro versus GPT-5.5's 88.20, and 61.56 on SWE-Bench Pro versus GPT-5.5's 58.60.

What makes InternS2 unusual isn't just the benchmark performance — it's the design decisions that got it there. Three distinct training pillars underpin the model, each targeting a different layer of capability that general-purpose LLMs typically sacrifice in favor of language fluency.

Architecture: Built for Science, Not Just Language

InternS2-Preview-397B uses a Qwen3.5-MoE text backbone. Despite the "397B" branding, Hugging Face model card counts report 403B total tensors — a minor discrepancy typical of how parameter counts are marketed versus what's actually serialized. The model activates roughly 17 billion parameters per token, routing through 10 of 512 available experts at inference time.

Key architectural details:

• 256K native context window (reported as 262K in some configurations with speculative decoding enabled)
• Multi-Token Prediction (MTP) for speculative decoding, reducing latency on long outputs
• Separate encoders for vision, scientific time-series data, and general image/video inputs
• Native multimodal support covering images, video, and scientific time-series — not bolted on after training

The separate encoder approach for time-series data is particularly notable. Most multimodal models treat all non-text input through a single visual encoder. InternS2 treats scientific instrument data, lab outputs, and experimental time-series as a distinct modality — which maps more accurately to how researchers actually work with data.

Training: Raw Documents and Reinforcement at Scale

InternS2 was trained on raw scientific documents — papers, datasets, and supplementary materials preserved with their original formatting. Diagrams, equations, tables, and figures were kept structurally intact rather than converted to text descriptions. Shanghai AI Lab's argument: stripping scientific documents to plain text destroys the spatial and relational context that gives those documents meaning.

On top of the pretraining base, the team scaled reinforcement learning across more than 20 scientific domains simultaneously — biology, chemistry, materials science, mathematics, physics, and others trained jointly rather than sequentially. The joint RL approach is designed to prevent the kind of catastrophic forgetting that tends to happen when models are fine-tuned domain by domain.

The third training layer is what Shanghai AI Lab calls "black-box agentic reinforcement learning." The model is connected to large-scale sandboxed environments — compute environments where the model can execute actions, run code, and observe outcomes — and trained on the quality of those outcomes rather than on step-by-step supervision. This is a more realistic training signal for long-horizon tasks than supervised imitation of human trajectories.

Benchmark Performance: What's Verified and What Isn't

The headline numbers come directly from Shanghai AI Lab, and the usual caveats around vendor-reported benchmarks apply. That said, the figures are specific enough to be falsifiable, and the MMLU-Pro score in particular has already drawn independent attention.

• MMLU-Pro: 89.75 — above GPT-5.5 (88.20), currently leading among open-source models
• SWE-Bench Pro: 61.56 — above GPT-5.5 (58.60), but below Claude (69.20)
• Biology and materials generation: Strong reported wins, though independent reproduction of scientific generation benchmarks is harder to verify quickly

The SWE-Bench Pro gap relative to Claude is worth flagging. On agentic software engineering — arguably the most commercially relevant agentic benchmark right now — InternS2 trails by nearly 8 points. That gap matters for developers evaluating whether this model is ready to drop into coding agent workflows.

InternS2 is not a general-purpose frontier killer. Mixed results on agentic and coding benchmarks outside of its scientific sweet spot suggest this is a specialized model that happens to generalize well, rather than a generalist that's also good at science.

Availability: Open Weights, But Not Truly Open Source

InternS2-Preview-397B is available under Apache 2.0 — meaning the weights can be used commercially, modified, and redistributed. An official chat interface and API are live at chat.intern-ai.org.cn. The GitHub repository is hosted at github.com/InternLM/Intern-S1.

What's not yet available: the model has no presence on OpenRouter, Together AI, Fireworks, or Groq as of this writing. Self-hosting is technically possible but practically out of reach for most — the model requires at minimum eight H100 or H200-class GPUs to run at full precision, putting it firmly in the "institutional hardware only" category for local deployment. If you're interested in what it actually takes to run frontier-scale open models on consumer hardware, AirLLM's layer-sharding approach for 70B models on 4GB VRAM illustrates just how far the gap still is at the 397B scale.

For users who want to run an InternLM model locally, the InternS2-Preview-35B variant released in May is the more practical option — capable enough for most research workflows and runnable on a single high-end consumer GPU.

Where It Fits in the Competitive Landscape

InternS2 sits in an interesting position relative to the current wave of large open-weight models. Kimi K3 from Moonshot AI remains the largest open-weight model by parameter count at 2.8 trillion, with a strong showing on coding and general reasoning tasks. DeepSeek V4 has drawn attention for running on Huawei Ascend silicon, removing the Nvidia dependency entirely. InternS2 carves out a different niche: scientific reasoning at depth, with multimodal inputs that actually understand the structure of experimental data.

The competitive comparison Shanghai AI Lab leads with — beating Qwen3-235B-A22B and GPT-5.5 on reasoning — is calibrated to land a specific message: that open-weight science-focused models have crossed a threshold. Whether that threshold matters for any given use case depends entirely on whether your workload looks more like biology and materials design or general software development. xAI's still-training 2-trillion-parameter successor to Grok 4.5 suggests the frontier closed-model ceiling is also continuing to rise — so the relative advantage of any open-weight release has a short shelf life.

What's Next

Shanghai AI Lab's "Preview" label on InternS2 signals this isn't the finished article. Preview releases in this space typically precede a training-complete version with improved benchmark numbers and potentially broader API availability. Given the strong scientific domain performance, the most likely trajectory is an expanded presence on third-party inference platforms and, eventually, dedicated scientific research partnerships.

For now, the official chat interface and the 35B local variant are the two accessible entry points. Whether the full 397B model reaches OpenRouter or Together AI in the near term will be a good signal of how seriously Shanghai AI Lab is pursuing Western developer adoption beyond the research community.

Related Reading

• Kimi K3 Is Official: The 2.8T Open Model That Built Its Own Compiler and Designed Its Own Chip
• 70B LLMs on 4GB VRAM — AirLLM's Layer-by-Layer Trick That Rewrites the Rules
• DeepSeek V4 Runs on Zero Nvidia Silicon — and Huawei's Biggest Chip System Debuts Tomorrow
• xAI's Next Model Outperforms Grok 4.5 in Every Metric — and It Hasn't Finished Training Yet