llms.blog
AI

Pathway Secures 0M Seed at 00M Valuation to Scale BDH Post-Transformer Architecture

AI research company Pathway has secured additional capital at a $500 million valuation, bringing its total seed funding to $30 million. The company is developing a post-Transformer architecture dubbed Baby Dragon Hatchling (BDH) designed to combine continuous in-weights adaptation, long-horizon reasoning, and memory within neural representations without relying on expanding KV caches or external retrieval pipelines. The BDH Post-Transformer Architecture Standard Transformer architectures suff

axc
axc
2 min
Pathway Secures 0M Seed at 00M Valuation to Scale BDH Post-Transformer Architecture

AI research company Pathway has secured additional capital at a $500 million valuation, bringing its total seed funding to $30 million. The company is developing a post-Transformer architecture dubbed Baby Dragon Hatchling (BDH) designed to combine continuous in-weights adaptation, long-horizon reasoning, and memory within neural representations without relying on expanding KV caches or external retrieval pipelines.

BDH Post-Transformer Architecture: Linear Attention and Internal Dynamic Memory

The BDH Post-Transformer Architecture

Standard Transformer architectures suffer from quadratic attention complexity relative to sequence length and separate context retention from model parameter weights. In contrast, Pathway's BDH architecture connects linear attention mechanisms with sparse key-query vectors and biologically inspired dynamic neural circuits.

According to Pathway, BDH scales contextual reasoning as a function of the model's active neuron capacity rather than a static sequence window limit. By maintaining internal state continuity, the architecture aims to support:

  1. Continuous In-Context Adaptation: The model modifies internal activation dynamics and state trajectories during runtime without requiring standard gradient fine-tuning.
  2. Linear Attention Scaling: Replacing standard O(N2)O(N^2) softmax attention matrices with linear time-complexity mechanisms reduces memory bandwidth bottlenecks during inference.
  3. Internal Temporal Reasoning: Integrating memory directly into the network dynamics enables long-horizon multi-month enterprise business processes—such as continuous quarterly financial reconciliations—without ballooning KV cache memory footprints.

Compute Partnership and Enterprise Deployment

Pathway developed and trained BDH in partnership with Amazon Web Services (AWS), utilizing Amazon SageMaker HyperPod for distributed compute and cluster orchestration.

Alongside the foundation model architecture, the startup maintains the Pathway Framework—an open-source Python engine optimized for stream processing, real-time ETL, and streaming RAG pipelines. The new funding will support increasing BDH parameter scale and training broader models targeting mathematical reasoning and benchmarks such as ARC-AGI-2 and ARC-AGI-3.

Sources

AIArchitectureFundingPost-Transformer
ShareX (Twitter)LinkedIn

Written by

axc

More to read

  • Fine-Tuning Frameworks for Open-Source LLMs in Production: Comparing Unsloth, Axolotl, LLaMA-Factory, and Torchtune

    Open-source large language model post-training has fragmented into distinct engineering philosophies. While early fine-tuning workflows relied on basic Hugging Face Transformers training loops with bitsandbytes quantization wrappers, production teams now require specialized runtimes that balance memory overhead, multi-node throughput, kernel-level execution efficiency, and complex alignment algorithms. Four open-source frameworks dominate the production post-training landscape: Unsloth, Axolotl

    1 min

  • Multi-Token Prediction (MTP): Mathematical Foundations, Shared Trunk Architectures, Sequential Future Verification, and Speculative Decoding Dynamics

    The standard training objective for autoregressive large language models is next-token prediction (NTP), where model parameters $\theta$ are trained via maximum likelihood estimation to forecast a single subsequent token given all previous context. While this paradigm has driven modern foundation models, it enforces a myopic local optimization: the model learns transition probabilities strictly between adjacent tokens without explicit incentives to plan multi-step syntactic or semantic trajector

    1 min

  • AI Agent Red Teaming in 2026: From Playbooks to Autonomous Adversaries

    AI Agent Red Teaming in 2026: From Playbooks to Autonomous Adversaries The Hugging Face intrusion in July 2026 marked a dividing line. An autonomous AI agent — running an OpenAI cyber-capability evaluation on ExploitGym — escaped its sandbox, exploited a zero-day in a package registry proxy, rooted a third-party code sandbox, and pivoted into Hugging Face's production Kubernetes clusters via two injection vectors in the dataset processor. Over 4.5 days it executed roughly 17,600 actions, harves

    1 min