What is Gimlet Labs building?

Gimlet Labs is building the first multi-silicon inference cloud: a platform that decomposes AI model workloads across different chip architectures (GPUs, CPUs, near-memory compute, and dataflow accelerators) to deliver faster, more power-efficient inference than homogeneous GPU clusters.

Why does multi-silicon inference matter for enterprise AI?

Agentic workloads execute dozens of sequential model calls per task. Latency compounds across those calls. A multi-silicon approach can deliver 5-10x speedups within the same power budget, which matters both for responsiveness and for controlling infrastructure cost at scale.

How does Gimlet's disaggregation work?

Gimlet traces and decomposes an LLM into components. Each component runs on the chip architecture best suited to it: compute-intensive prefill on one type of silicon, memory-bandwidth-bound decode on another. The platform also supports speculative-decode and attention/FFN disaggregation. Developers see a single inference API.

Who are Gimlet's investors and customers?

The $300M Series B was led by Andreessen Horowitz and joined by Arm, M12 (Microsoft's venture fund), Samsung Ventures, Sapphire Ventures, Menlo Ventures, Tiger Global, and twelve others. Gimlet has confirmed a top-three AI lab and a major cloud provider as customers, though it has not named them publicly.

How does Gimlet fit into the broader enterprise AI infrastructure picture?

Gimlet addresses the power ceiling that is now the primary bottleneck for enterprise AI scaling. As GPU capacity is rationed and data-center power triples in cost and competition, infrastructure that extracts more intelligence per kilowatt becomes a competitive differentiator rather than a niche optimization.

Gimlet Labs Raises $300M for Multi-Silicon AI Inference

On September 4, 2026, Gimlet Labs announced a $300 million Series B round that brought its valuation to $3 billion, less than six months after raising an $80 million Series A. Andreessen Horowitz led the round, joined by Arm Holdings, M12 (Microsoft’s venture fund), Samsung Ventures, Sapphire Ventures, Menlo Ventures, Tiger Global, and a dozen additional backers. The company is building what it calls the first multi-silicon inference cloud: infrastructure that runs AI model workloads across GPUs, CPUs, near-memory compute systems, and dataflow accelerators simultaneously, rather than relying on a homogeneous pool of a single chip type.

The round is one of the largest AI infrastructure raises of 2026 and signals a shift in where the frontier of enterprise AI scaling is now located. It is no longer in training. It is in inference, and specifically in how to extract more intelligence from each kilowatt of constrained data-center power.

The Compute Ceiling Is a Power Ceiling

The framing in a16z’s investment thesis is direct: the AI industry is running out of watts. AI data centers consumed approximately 18 GW of capacity in 2025; that figure is projected to triple by 2030. Capital expenditure on AI infrastructure is running near $1 trillion per year and is expected to reach a cumulative $7 trillion by 2030. GPU capacity is being leased at record prices, with rate limits and latency degradation showing up across major providers as they trade interactivity for throughput.

Gimlet’s founding proposition is that inference infrastructure was never designed for inference. The GPU clusters powering today’s production AI were originally built for training, then repurposed. Training and inference have fundamentally different requirements, and even within inference, different workloads pull in different directions: a voice assistant lives or dies on latency, batch jobs optimize throughput, a research agent targets cost per token, and a coding agent must balance all three.

Monthly token generation has grown 6x in 12 months according to Gimlet, with some projections estimating another 20x increase by 2030. Serving that volume efficiently from constrained power grids requires more than additional GPUs. It requires rethinking what inference infrastructure is.

How Multi-Silicon Disaggregation Works

A large language model is not a monolithic workload. It consists of components with very different hardware requirements. The prefill phase, where the model interprets an input prompt, is compute-intensive. The decode phase, where the model generates tokens autoregressively, is memory-bandwidth-bound. Running both phases on the same chip type leaves significant performance on the table.

Gimlet’s software traces and decomposes models into components, then schedules each component onto the silicon architecture best suited to it. According to SiliconANGLE, the platform can route prefill to a compute-optimized accelerator and decode to a memory-optimized system, dynamically rebalancing load as hardware availability shifts. The software also supports more granular disaggregation: speculative-decode disaggregation (a lightweight “drafter” model on one chip, a frontier model on another) and attention/FFN disaggregation.

Converge Digest’s technical breakdown notes that Gimlet uses a combination of AI agents and a custom compiler to optimize model code for target hardware, covering generic and chip-specific optimizations. The result is presented to developers through a single inference API; the heterogeneous scheduling is invisible above that interface.

Gimlet claims this approach delivers 5-10x speedups for frontier workloads within the same power footprint, or corresponding throughput improvements at a given latency level. These are company-reported performance figures. The company has confirmed a top-three AI lab and a major cloud provider as customers, which lends some weight to the claims, though independent benchmarks have not been published.

Gimlet Labs: Series B at a Glance
Round size	$300 million Series B
Valuation	$3 billion
Lead investor	Andreessen Horowitz
Notable co-investors	Arm, M12 (Microsoft), Samsung, Sapphire, Menlo, Tiger Global
Total funding raised	$392 million
Time since Series A	~5 months (Series A: $80M, March 2026)
Claimed speedup	5-10x on frontier workloads at same power footprint
Disaggregation types	Prefill/decode, speculative-decode, attention/FFN, custom
Infrastructure scale	Scaling toward hundreds of megawatts of managed capacity
Confirmed customer types	Top-three AI lab, major cloud provider (unnamed)

Why Agentic Workloads Make This Urgent

The timing is not coincidental. As enterprise AI moves from single-shot queries to agentic workflows, the infrastructure requirements change significantly. An agent executing a multi-step task might call a model dozens of times in sequence, retrieve documents, execute code, and call external tools. Latency compounds across every step. A 200ms response time that feels acceptable for a chat query becomes 10 seconds for a 50-step workflow, which is too slow for production use cases.

Context windows have also grown: from roughly 100,000 tokens in 2023 to over 1 million tokens today. Frontier model sizes have moved from hundreds of billions of parameters to 3-10 trillion. Both trends increase the memory bandwidth pressure on decode, making the case for heterogeneous disaggregation stronger with each model generation.

This is the same infrastructure pressure that drove Callosum’s $100M raise for heterogeneous AI routing in August, and that sits underneath the broader enterprise AI infrastructure expansion now reshaping the market. The constraint is not intelligence. It is the physical cost of delivering intelligence at low latency and high throughput simultaneously.

What Enterprise AI Teams Should Watch

For enterprise teams building agentic workflows, the Gimlet story carries a few practical implications.

Inference cost is infrastructure cost. The tokenomics pressure enterprises have been navigating does not go away when token prices fall. It shifts to the infrastructure layer, where power and compute efficiency determine total cost of ownership for large-scale agentic deployments.

Multi-silicon is becoming a supply-chain choice. Arm’s participation in the Gimlet round alongside M12 reflects a deliberate effort to make heterogeneous inference viable across multiple chip architectures, not just NVIDIA GPUs. Enterprises that route workloads through platforms like Gimlet gain some insulation from single-vendor GPU supply constraints.

Latency budgets are the new SLA. As agents compound model calls, the tolerance for high latency shrinks. Enterprise architecture decisions made today around inference providers will shape whether agentic product experiences are fast enough to be useful, or too slow to replace manual workflows.

Gimlet sells its platform in two modes: a serverless edition and as a managed service deployable on enterprise infrastructure. The company is also developing an inference-optimized server without a traditional motherboard, intended for deployment outside conventional data centers, which suggests an edge inference roadmap beyond the current cloud-first offering.

The $300M round positions Gimlet to scale from its current gigawatt-scale pipeline to hundreds of megawatts of active managed capacity, and to expand into custom hardware. For the enterprise teams now designing the infrastructure layer beneath their agentic AI stacks, it is one of the more technically specific bets on what the next generation of inference infrastructure needs to look like.