NVIDIA released Nemotron 3.5 Lightning and NeMo Switchyard on August 11, 2026, pairing a new open 30-billion-parameter agentic model with an open-source routing library that dynamically selects the cheapest capable model for each step of a multi-agent workflow. Early-access partners including LangChain, Ramp, and Cognition are reporting cost reductions between 28% and 74%, holding performance near frontier-model quality. For enterprise teams running always-on AI agents, the combination directly addresses the cost problem that has kept agentic deployments in pilot mode.
The Cost Problem NVIDIA Is Solving
Enterprises building multi-agent systems face a compounding economics problem. Routing every task to a frontier model like Claude Opus 4.8 or GPT-5.6 Sol produces high-quality results but becomes expensive at the token volumes that sustained agentic workflows generate. Building custom routing logic to direct simpler tasks to cheaper models solves the cost problem but creates a new engineering burden: the router has to be maintained every time a workflow changes, a model updates, or a new provider enters the mix.
NVIDIA’s answer is to solve both problems simultaneously. Nemotron 3.5 Lightning is a purpose-built model for the cheap, fast end of an agent workflow. NeMo Switchyard is the routing layer that decides when to send work to Lightning versus a frontier model, based on live signals from the agent’s state rather than a static configuration set at design time.
The pairing matters because a cheaper model alone does not solve the routing problem, and a smarter router alone has nothing efficient to route to. Owning both under one open license is what distinguishes this release from model-only or router-only competitors.
Nemotron 3.5 Lightning: What Enterprises Are Getting
Nemotron 3.5 Lightning extends the hybrid Mamba-Transformer, latent mixture-of-experts architecture that NVIDIA introduced with the Nemotron 3 family in late 2025. The model has 30 billion total parameters but activates only around 3 billion per token, a design that produces high efficiency for the type of high-volume, specialized tasks that dominate production agentic workflows.
According to NVIDIA’s official release, the model delivers up to 4x faster output speed and completes agentic tasks roughly 30% faster than Qwen3.6-35B at comparable accuracy on PinchBench, a real-world agent benchmark covering coding, research, and file management tasks. It also outperforms Gemma 4 26B’s accuracy at a similar completion time.
The model supports post-training with NVIDIA NeMo on custom domain data, which is where NVIDIA’s early-access partners are seeing the most significant gains. CodeRabbit built a working router agent using the NeMo Auto model recipe in approximately two hours for roughly $85 in compute cost. CrowdStrike customized Lightning for malicious-content recall in cybersecurity workloads. Harvey and Trajectory applied it to legal task completion. Lila Sciences used it for energy simulation work in physical and life sciences.
The distribution is broad. Nemotron 3.5 Lightning is available on Hugging Face, ModelScope, OpenRouter, and build.nvidia.com as an NVIDIA NIM microservice. It can run on NVIDIA RTX PCs, DGX Spark, DGX Station, Jetson devices, and across cloud environments, giving enterprises the flexibility to start with hosted inference and move to on-premises deployments as workloads scale.
NeMo Switchyard: Dynamic Routing at the Agent Step Level
NeMo Switchyard is an open-source Rust proxy that routes LLM traffic across providers and translates between the OpenAI Chat, Anthropic Messages, and OpenAI Responses API formats. The library sits between an agent framework and its model backends, evaluating each request and routing it to the most appropriate model without requiring developers to rewrite the agent application.
Switchyard offers four routing strategies:
| Strategy | Routing Signal | Best For |
|---|---|---|
| Random Routing | Fixed strong/weak probability split | A/B testing, cost baselines |
| LLM Classifier Routing | Request content drives tier selection | Content-varying agent tasks |
| Stage-Router Routing | Agent state and tool-result signals | Multi-step agentic workflows |
| Escalation-Router Routing | Starts weak, escalates on judge failure | Conservative cost management |
The stage router is the most relevant for enterprise agentic deployments. An agent’s state changes as tools return results, errors appear, or a step turns out to be routine rather than complex. Static model assignments cannot adapt to those transitions. Switchyard evaluates live signals from the agent, including tool-use outcomes and agent state markers, to route each request dynamically rather than applying a single model choice across the entire workflow.
Cost awareness is built into the routing decision. Switchyard can evaluate model verbosity, predicting how many tokens a given model tends to generate for a task type, and steer work toward the cheaper option before the API call is made. That prediction layer is what allows the LangChain result, routing only 7% of calls to a frontier model while preserving task performance.
The library integrates with agent frameworks (LangChain, Cognition, Nous Research) and LLM gateways (Kong AI Gateway, LiteLLM, OpenRouter), which means enterprises can adopt Switchyard without replacing their existing infrastructure. Kong ships Switchyard natively inside Kong AI Gateway. OpenRouter has integrated the routing algorithm into its Auto mode. For teams already running infrastructure from those providers, Switchyard becomes available without a new deployment surface.
Partner Results: The Enterprise Cost Picture
NVIDIA shared specific benchmark figures from five early-access partners:
| Company | Use Case | Cost Reduction | Performance Trade-off |
|---|---|---|---|
| LangChain | Multi-turn Deep Agents (145 tasks) | 74% | 6% accuracy reduction |
| Ramp | Ramp SWE-Bench | 58% | None (matched frontier) |
| Cognition (Devin Desktop) | FrontierCode Main | 28% | Near-frontier performance |
| CodeRabbit | Code review routing | Custom router built for $85 | N/A |
| NVIDIA (internal) | vs Opus 4.8 alone | ~66% (cost to one third) | Frontier-level task completion |
The Ramp result is the most significant for enterprise planning. Matching frontier performance at 58% lower cost on an engineering benchmark suggests the routing approach can preserve output quality on real production workloads, not just synthetic tests. The LangChain result is the most aggressive: 7% of calls going to a frontier model means 93% of requests are served by Lightning or another efficient model, with the quality maintained by the routing layer’s ability to escalate when it detects a task needs more capability.
What This Means for Enterprise AI Infrastructure
For enterprises building agentic production infrastructure, the Nemotron 3.5 Lightning and Switchyard release represents a structural shift in how to think about model selection. The question is no longer which single model to default to across a workflow. It is how well the routing layer matches models to individual steps in production, a harder problem to benchmark and a more durable competitive advantage to build.
Three patterns are emerging in how enterprises are approaching this:
Routing becomes infrastructure. Dynamic per-step routing, driven by live agent state rather than static configuration, is becoming a core infrastructure concern. Enterprises that designed agent pipelines around a single default model are being pushed toward routing architectures as the cost pressure of multi-step agentic workloads compounds.
Open source at both layers. The Lightning plus Switchyard combination is fully open under NVIDIA’s standard NIM licensing and GitHub open-source terms. That means enterprises can deploy the entire cost-reduction stack without per-token API charges to NVIDIA, using their own hardware or cloud infrastructure, a different economics model from closed inference APIs.
Specialization through post-training. The partners seeing the largest cost reductions are the ones post-training Lightning on domain-specific data. A general model fine-tuned on cybersecurity telemetry, legal documents, or code review patterns outperforms a general model on those tasks at lower cost. The NeMo post-training stack makes this accessible at a reported cost of under $100 for a working router agent.
This dynamic connects directly to the broader enterprise AI pricing revolt that has been building across the market. Enterprises are scrutinizing token costs as agentic workloads scale, and the pressure to find infrastructure that delivers frontier-quality output at lower per-task cost is real. NVIDIA’s bet is that owning the model and the routing layer under one open license is the approach that actually moves the cost needle, not a single cheaper model or a smarter router built on someone else’s stack.
The Competitive Context
Switchyard is not entering an empty space. Not Diamond already powers OpenRouter’s Auto mode, and RouteLLM, the open-source routing framework from UC Berkeley and LMSYS, gives developers a foundation for building their own routers. Neither ships its own model. NVIDIA’s argument is that controlling both sides of the routing decision, under one open license, provides a coherence that model-only or router-only alternatives cannot match.
For enterprises evaluating the broader AI gateway market, Switchyard’s integration posture is notable: it plugs into existing gateways and frameworks rather than replacing them. That means adopting Switchyard does not require a forklift migration of the agent infrastructure already in production.
NVIDIA’s release arrives as the enterprise AI infrastructure layer continues to fragment into specialized components: models for specific task types, gateways for governance, routers for cost management, and runtime environments for deployment. Nemotron 3.5 Lightning and NeMo Switchyard are a direct response to that fragmentation, offering a matched model-plus-router pair that can slot into existing stacks without requiring enterprises to rebuild from scratch.