Gimlet Labs' $300M Bet: AI Inference Stops Picking a Chip

Zain Asgar spent years watching AI infrastructure get built around one assumption: that a single chip family, the graphics processor, would carry everything. By the time he and four co-founders took Gimlet Labs out of stealth in October 2025, that assumption had hardened into the industry's most expensive bottleneck.

Their answer is a cloud that refuses to pick one chip.

The round lifts its total funding to $392 million, five months after an $80 million Series A led by Menlo Ventures. Arm and Microsoft's venture fund, M12, joined as new backers, alongside returning investors Factory and Sapphire Ventures.

TIMELINE: Gimlet Labs, stealth to $3B
─────────────────────────────────────────────────────────────
  2025 ──────── Oct 2025 ──────── Mar 2026 ──────── Sep 2026
  🔬            🧪                🏭                ◉ NOW
  founded       $12M seed          $80M Series A     $300M Series B
                leaves stealth     Menlo-led         a16z-led, $3B

The architecture bet: stop picking a winning chip

Every large language model is a chain of jobs with different appetites. Prefill reads the prompt and is compute-bound. Decode writes the answer one token at a time and is memory-bound. Tool calls are network-bound. A single processor has to be mediocre at all three.

Its software slices a model apart and routes each phase to the hardware that handles it best: graphics processors, central processing units, accelerators that keep data in on-chip memory, and dataflow architectures. The company says the approach delivers up to 10X gains in throughput and interactivity on frontier models within the same power envelope.

"We're able to deliver unprecedented performance because Gimlet software intelligently slices and orchestrates workloads across different types of hardware, from both mainstream and emerging chipmakers," Asgar said.

The vendor list matters. It supports chips from NVIDIA, AMD, Intel, Arm, Cerebras, and d-Matrix, and sells the layer that makes them interchangeable. Arm's participation in the round is the tell: a chip designer investing in the software that decides when its chips get used.

AI demand is growing exponentially, while data centers and silicon can't keep pace. The answer isn't just more infrastructure — it's a better architecture.— Raghu Raghuram, managing partner, Andreessen Horowitz

Where the $300 million goes

It sells two things. Gimlet Cloud is a serverless inference service built for AI agents. A second product runs as a managed service inside a customer's own data center. Underneath both sits a compiler that uses AI agents to find the best way to adapt model code to each chip, then tests the result before shipping it. A companion tool, kforge, generates optimized kernels from PyTorch, the framework most models are trained in.

The new capital is aimed at physical capacity. The company says it has secured billions of dollars in contracted revenue and is scaling to hundreds of megawatts (MW) of managed, heterogeneous infrastructure, with a gigawatt-scale data center pipeline. It also plans an inference-optimized server that can sit outside a traditional facility.

The demand signal behind that buildout is the shift from training to serving. Inference is now measured in quadrillions of tokens a month, and Goldman Sachs estimates AI capital spending will reach $765 billion in 2026 alone, or $7.6 trillion cumulatively through 2031. Most of that money is still poured into homogeneous racks.

Gimlet is not alone in the category. Inferact raised $150 million in seed funding in January, Baseten raised $300 million in February, and Groq has rebuilt itself as an Nvidia-powered neocloud after a licensing deal stripped its founding team. The money is plentiful. The question is which layer captures it.

The turning points

Three moments set up this round. The first was December 2025, when a non-exclusive licensing deal between Nvidia and Groq showed how fast the hardware layer can be reshuffled by a single contract. The second was March 2026, when the company said it had tripled its customer base and added one of the top three frontier labs and one of the top three hyperscalers.

The third is the round itself. The $3 billion valuation is a statement about where the margin sits. It has no fabrication plant, no chip of its own, and describes its revenue only as "billions contracted".

As we wrote in August, Etched's $21 billion bet on single-purpose inference silicon is a one-trick wager: it wins only if one model architecture keeps winning. Gimlet is making the mirror-image bet. If model architectures keep fracturing, the orchestration layer becomes the durable asset.

What could break the thesis

The obvious risk is that one architecture gets good enough. If Nvidia's next generation closes the gap between prefill and decode efficiency, the premium for heterogeneity shrinks. The less obvious risk is operational. Running many chip types in one facility means different power densities, cooling needs, and network topologies. Heterogeneous compute is sometimes a compiler problem and sometimes a plumbing problem.

Its answer is that it has already moved into power, cooling, and data-center construction. That is a capital-heavy business with thin margins and long lead times, and it is exactly where the constrained AI buildout is heading. The company is betting that whoever solves orchestration owns the layer above the chips, and that this layer is where the next decade of infrastructure margin accumulates.

Sources