The race for faster AI inference has pushed labs toward custom silicon, but a French startup argues the cheapest speedup is already sitting in servers companies bought. Kog says it can reach roughly thirty times faster LLM inference on standard data-center GPUs using software alone.
Kog demonstrated three thousand tokens per second per request on AMD MI300X and NVIDIA H200 GPUs in May, running a small two-billion-parameter model it has since open-sourced as Laneformer 2B. Chief executive Gael Delalleau told TechCrunch the preview drew about two hundred qualified business leads.
The first target is software engineering. Claude Code users already wait hours for long runs, and Anthropic charges a premium for Fast Mode, so speed is something teams pay for. Kog is pitching shops that run AI workflows for paid work and cannot absorb those delays.
The bigger promise is thirty x on large models, but today it rests on a tiny demo. Delalleau expects to show a major model running at ten x speed in September, a milestone he says will unlock customer traction and a Series A raise. The approach is deliberately low-level.
Delalleau, a former white-hat hacker and Ecole Polytechnique physics graduate, says his team reverse-engineers each GPU down to assembly and binary to exploit unused memory bandwidth. The cost is time: weeks to months per chip with a team of eleven, which caps how many accelerators Kog can support.
Kog is not alone. France's ZML ships hardware-agnostic inference software that bypasses CUDA, and Stanford's Hazy Research explores similar GPU acceleration. Kog compares itself to Hazy Research but goes deeper. Backers include Scaleway, Bpifrance, and French Tech 2030, giving it a European sovereignty tailwind.
The open question is whether hand-tuned, per-chip optimization scales to the models customers actually want. If Kog hits ten x on a frontier model in September, the assumption that GPUs are already maxed out gets a serious challenge.
Sources
TechCrunch: https://techcrunch.com/2026/08/14/kog-is-going-deeper-to-squeeze-more-inference-out-of-gpus/
Kog tech preview: https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/
Laneformer 2B (Hugging Face): https://huggingface.co/kogai/laneformer-2b-it
