Kubernetes3 articles

Kubernetes

Articles

GPU Cluster Scheduling in Production: Slurm vs. Kubernetes (Kueue/Volcano) vs. Ray
GPU Cluster Scheduling in Production: Slurm vs. Kubernetes (Kueue/Volcano) vs. Ray Modern AI infrastructure represents a radical departure from traditional cloud computing. Standard cloud workloads (such as stateless microservices, web applications, and independent batch jobs) rely on fine-grained elasticity, independent container scheduling, and horizontal autoscaling. In contrast, distributed large language model (LLM) training and high-throughput inference pipelines violate virtually every a
August 21, 20261 min
GPU Slicing in Production AI Systems: Comparing MIG, MPS, Time-Slicing, and Dynamic Partitioning
GPU Slicing in Production AI Systems: Comparing MIG, MPS, Time-Slicing, and Dynamic Partitioning Modern production AI systems rarely deploy a single standalone large language model. Contemporary compound AI architectures rely on heterogeneous pipelines comprising embedding models (such as BGE or E5), cross-encoder rerankers, safety classifiers (such as Llama Guard), speculative decoding draft models, and vision-language encoders. While primary generation models typically require dedicated multi
August 20, 20261 min
LLM Autoscaling and Cold Starts in Kubernetes: Architecture, KEDA Metrics, Model Weight Caching, and Ephemeral GPU Provisioning
Autoscaling large language model workloads on Kubernetes presents a fundamentally different engineering problem than traditional stateless microservices. While web APIs scale on CPU utilization or request rate within seconds, LLM inference instances require specialized GPU accelerators, massive container images, multi-gigabyte weight tensors, and intensive runtime compilation before serving a single token. Without proactive architectural design, a cold-starting LLM pod on Kubernetes often requi
August 20, 20261 min

Kubernetes

Articles

GPU Cluster Scheduling in Production: Slurm vs. Kubernetes (Kueue/Volcano) vs. Ray

GPU Slicing in Production AI Systems: Comparing MIG, MPS, Time-Slicing, and Dynamic Partitioning

LLM Autoscaling and Cold Starts in Kubernetes: Architecture, KEDA Metrics, Model Weight Caching, and Ephemeral GPU Provisioning