When you open the download page for a popular open model, you do not find one file. You find dozens that are all "the model": a stack of multi-gigabyte shards with a config.json, a flat list of files ending in .gguf with quantization levels in the names, and sometimes a folder marked MLX. They contain the same trained weights. They are not interchangeable, and picking the wrong one costs you a 60 GB download or a loader error.
This post is a map of that distribution layer: what each format actually is, why the split happened, and which file you should click.
The archive: safetensors
The base artifact of modern open-weights release is the safetensors file. Its design is deliberately boring: a small JSON header naming every tensor, its dtype and its shape, followed by raw weight bytes. There is no executable code path in the format, and the header lets a loader memory-map the file instead of parsing it. Hugging Face released it in September 2022, and in 2023 an external security audit commissioned jointly by Hugging Face, EleutherAI and Stability AI cleared it to become the default save format across those organizations.
The reason a new format was needed at all: PyTorch checkpoints historically shipped as pickle, and pickle is a serialization protocol that can run arbitrary Python during deserialization. A malicious "model" file could take over the machine loading it. Hugging Face's own security documentation states the plainest mitigation, "don't use pickle," and the audit cleared the replacement. The migration is real but not complete: the 2025 PickleBall academic study still found pickle-based models prevalent on the Hub, and a February 2025 ReversingLabs writeup documented malicious pickle models that evaded Hugging Face's Picklescan scanner.
What safetensors does not do: it does not shrink the model, does not carry the tokenizer or architecture metadata in the file itself, and is not directly runnable by most inference engines at consumer hardware sizes. Those gaps are what the other formats fill.
The local format: GGUF
GGUF is the container llama.cpp speaks. Georgi Gerganov developed it, it replaced the GGML format in August 2023, and it is now the de facto distribution standard for consumer inference on llama.cpp, Ollama, LM Studio, Jan and most of the local-LLM tool chain.
Architecturally it is the opposite bet from safetensors. A GGUF is one file that carries everything: the quantized tensors plus a key-value metadata header with the architecture, tokenizer and hyperparameters. A loader from an older version of the spec can open a newer file and skip fields it does not recognize. The quantization is baked in, not applied at load time, and the method is encoded in the filename. The spec's type table runs from plain F16 and BF16 through block quantizations like Q8_0 and the super-block k-quants (Q4_K_M lands at about 4.5 bits per weight) to the importance-scaled i-quants, where IQ1_S squeezes weights to 1.56 bits per weight. It compiles and runs on Metal, CUDA, AMD, Vulkan, CPU and more.
The adoption numbers from Hugging Face's own Summer 2026 ecosystem report put the scale of this layer: over the January to August 2026 window, repositories declaring the gguf library grew 464%, against 16% for transformers and peft. Qwen's models alone pulled 39.6 million GGUF downloads per month, nearly twice Gemma's 20.8 million and more than five times Llama's 7.5 million. The format's ceiling moved with llama.cpp itself: by July 2026 there were GGUF builds of roughly 284-billion-parameter and 2.8-trillion-parameter models, meaning "local inference" now includes frontier-scale mixture-of-experts spread across a few consumer machines.
The uncomfortable part is provenance. Of 28,531 GGUF conversions of Qwen models on the Hub, Qwen itself published 54. The quantization layer that makes large open models usable at home is almost entirely community labor, and Hugging Face's report explicitly nudges labs to publish and sign official conversions rather than leaving a gap between the weights tested by model creators and the files most users actually run.
The Apple lane: MLX
MLX is Apple's array framework for Apple silicon, and its "format" is a misnomer worth correcting. An MLX model is a directory of safetensors files plus a tokenizer and config, produced by mlx_lm.convert from the original Hugging Face checkpoint. Quantization is a representation inside that directory: group-wise, default 4-bit with a group size of 64, with kernels written specifically for Apple's unified memory rather than ported from NVIDIA paths.
Practical consequences. MLX builds run only on Apple silicon (macOS 14 or later), and the gguf tag on Hugging Face numbers roughly eight times the mlx tag (counts taken 2026-10-19). There is no clean GGUF-to-MLX conversion path either: you convert from the source safetensors, because dequantizing a GGUF back to 16-bit and requantizing loses quality. In return, on Macs the generation throughput is often the best available, and MLX is the option if you want to fine-tune locally or build native Swift apps. Comparisons on identical 4-bit builds give mixed verdicts, with some Mac benchmarks favoring MLX on generation and GGUF on prefill latency, which is why the honest advice for Mac users is to test both.
The engine room: everything else
Between the archive and the two consumer lanes sits the serving layer, and it mostly reads safetensors directly. vLLM, TGI and SGLang load safetensors checkpoints, including LoRA adapters for multi-adapter serving. Weight-only quantizations of the same archive exist as AWQ, GPTQ and FP8 safetensors variants for data-center GPUs. ONNX and TensorRT are the compiled, engine-specific formats for framework portability and maximum NVIDIA throughput. Google's and NVIDIA's own guidance on model formats summarizes the whole landscape in one line that this post is built around: format follows runtime, not preference.
Same weights, different artifacts
The numbers make the trade-offs concrete. For one 27-billion-parameter model in November 2026, community trackers list: GGUF Q4_K_M at 16.81 GB, MLX 4-bit at 16.08 GB, an AWQ INT4 safetensors build at 21.04 GB, and the lab's own FP8 safetensors at 30.89 GB. The GGUF file and the FP8 archive are the same trained model, and they are not the same thing you can run on the same hardware.
Four pitfalls worth knowing before you click:
- Download counts are not usage for GGUF repos. A single model repository ships a dozen quantized files, and Hugging Face's download statistics count requests against those files unevenly. Treat GGUF download numbers as file pulls.
- A community quant is not the tested artifact. If the model's quality claims were measured at BF16 and you are running an unsigned IQ3 build at 3.44 bits per weight, the eval table does not describe your model.
- The filename is the product. Q4_K_M and IQ1_S are both "the 4-bit model" only to a person reading loosely. The level is already applied; you are not switching it at runtime.
- A GGUF is not a compressed safetensors. It is a different file whose weights were already rounded. You cannot expand it back to full precision.
The decision in three lines
If you are serving on data-center GPUs, take the safetensors archive and let your engine apply quantization. If you are running locally on mixed hardware, take the GGUF at the largest quant that fits your memory, preferring signed or lab-published conversions when they exist. If you are all-in on Apple silicon and care about throughput or local fine-tuning, take the MLX build and test it against GGUF on your specific model before committing.
The distribution layer grew faster than the models underneath it in 2026: repositories declaring gguf rose 464% against 21.5% growth for model repositories overall. That is the sound of open weights becoming actual software: the research artifact and the runnable artifact are now different files, and someone other than the lab usually made yours.

Sources
- Hugging Face, State of Open Models: Summer 2026 Observations (GGUF repository growth, monthly GGUF downloads, Qwen conversion counts, trillion-parameter GGUF builds): https://huggingface.co/blog/state-of-open-models-summer-2026
- Hugging Face Hub documentation, GGUF format (metadata design, quantization type table): https://huggingface.co/docs/hub/en/gguf
- Hugging Face, Safetensors security audit announcement: https://huggingface.co/blog/safetensors-security-audit
- Hugging Face Hub documentation, Pickle Scanning (pickle threat model and mitigation guidance): https://huggingface.co/docs/hub/en/security-pickle
- PickleBall: Secure Deserialization of Pickle-based Machine Learning Models, arXiv (format release dates, residual pickle prevalence): https://arxiv.org/abs/2508.15987
- Safeguard.sh, Hugging Face Pickle Backdoor Research 2025 (ReversingLabs nullifAI writeup, PyTorch weights_only mode): https://safeguard.sh/resources/blog/huggingface-pickle-backdoor-research-2025
- Google Developer forum, "Choosing the right format for your AI model" by Ekaterina Sirazitdinova (NVIDIA) and Ivan Nardini (Google Cloud): https://discuss.google.dev/t/choosing-the-right-format-for-your-ai-model-a-comprehensive-guide-to-ai-inference-formats/276691
- vdf.ai, GGUF Quantization levels table (Qwen3.8-27B cross-format size comparison, November 2026): https://vdf.ai/blog/gguf-quantization-levels
- Contra Collective, GGUF vs MLX Quantization Formats on Apple Silicon (MLX group-quantization mechanics, conversion asymmetry): https://contracollective.com/blog/gguf-vs-mlx-quantization-formats-apple-silicon-2026
- outlier.host, GGUF vs MLX timed comparison (gguf-tag vs mlx-tag model counts, machine-specific throughput): https://outlier.host/vs/gguf-vs-mlx-model-formats
- Ertas AI, GGUF vs SafeTensors comparison (serving-side format expectations, multi-LoRA): https://www.ertas.ai/compare/gguf-vs-safetensors
- AI Craft Journal, How to Read Hugging Face Trending Models (GGUF download-count semantics caveat): https://aicraftjournal.com/articles/hugging-face-trending-models
