How to Run an AI Model Locally on Your Phone
Free apps can run open AI models on your iPhone or Android phone with no internet. How to set up AI Edge Gallery or PocketPal AI and pick a model.

Running AI on phones, laptops and embedded devices. Posts compare small language models like SmolLM2 and Llama 3.2, on-device runtimes such as ExecuTorch and MLC-LLM, and local tools like llama.cpp and Ollama, plus edge hardware like Nvidia Jetson.
Free apps can run open AI models on your iPhone or Android phone with no internet. How to set up AI Edge Gallery or PocketPal AI and pick a model.

Phi is Microsoft's family of small open AI models built to run on your own device. Here is each Phi-4 model, its MIT license, and how to run it locally.

MiniCPM is OpenBMB's family of open-weight models for phones and PCs, from MiniCPM5 and MiniCPM-V 4.7 to MiniCPM-OCR 1.0. Sizes, licenses, and how they differ.

Zyphra's ZONOS3-Mini is a small open voice-cloning model that runs in a browser. What you can control, how fast Zyphra says it is, and what it costs.

Gradium's Phonon is a family of small open text-to-speech models that run on your device. Which model to pick, licenses, benchmarks and how to try it.

Underdog Woof is an English text-to-speech model of 31.7 MB or 40.6 MB that runs on a laptop CPU or in a browser, offline. Accuracy claims and license.

Gemma is Google's family of open AI models you can download and run. How it differs from Gemini, its variants, how to try it and what the license allows.

Liquid AI's LFM3 is six open-weight 2B-class models for text, vision, audio, omni input and decisions, built on one 1.7B base. Specs, scores and license terms.

Open-weight models come in files labeled Q8_0, Q4_K or IQ2_XXS. They are the same model stored with fewer bits per weight. Here is what quantization is, how GGUF labels work, what you give up, and how to pick a file that fits your computer.

Nanbeige4.3-33B is a new Apache 2.0 open-weight model for coding agents with a 512K context. Its own table puts it ahead of Qwen3.8-27B and Gemma4-31B on most agent tests. Here is what it is, what the numbers say, and what hardware it needs.

Recursive's Flint is a 40B MoE coding model with 2.5B active parameters, released under Apache 2.0. Here is what it scores in Recursive's own tests, how fast it runs on a Mac or RTX Spark, the hardware you need, and what the Eureka Machine built.

A new paper replaces Gemma 3 270M's 168M-parameter embedding table with frozen 512-bit codes. Trained on the same 2.05B tokens, it ties or beats the learned table on LAMBADA, HellaSwag and PIQA, and a ternary version is a 46 MB file running 615 to 740 tokens per second on a laptop CPU.

RTX Spark puts an NVIDIA Blackwell RTX GPU, a Grace CPU and up to 128GB of unified memory in one chip for Windows laptops. Here are the specs, the laptops, what NVIDIA says each memory size runs locally, Nemotron 3.5 Bolt, and retailer prices from $2,599.99.

Nvidia's Nemotron 3.5 Bolt 74B-A9B packs a model compressed from Super VL into 24GB with 2-bit experts for RTX Spark and DGX Spark. Here is what it needs to run, how its speed compares in Nvidia's own tests, and when Bolt E4B arrives.

OpenBMB's MiniCPM-V 4.7 is an Apache 2.0 vision model in 35B-A3B and 1B sizes. A new expert-scheduling method streams MoE weights from flash, and OpenBMB reports up to 20.51 tokens per second on an iPhone. Here is how it works, how it scores, and how to run it.

Deploying language models directly on client hardware, edge gateways, and low-latency routing layers requires balancing parameter count, memory footprint, and task-specific reasoning capability. The sub-3B parameter class has shifted from stripped-down toy models to highly capable engines trained on tens of trillions of tokens or distilled from frontier teachers. In production architectures, models in the 1B to 3B range serve three primary roles: speculative decoding draft models that accelerat

The deployment landscape for large language models is bifurcating. While datacenter workloads rely on high-throughput continuous batching engines such as vLLM and TensorRT-LLM, local and edge deployments operate under fundamentally different physical constraints. On developer workstations, embedded hardware, and private office clusters, inference is rarely bound by compute saturation across thousands of concurrent requests. Instead, it is constrained by memory bandwidth, local VRAM capacity, hos

Activation-Aware Weight Quantization (AWQ): Mathematical Foundations, Salient Weight Protection, and INT4 Tensor Core Execution Large language models have transformed AI applications, but their deployment remains constrained by memory and compute barriers. A 70B parameter model in FP16 occupies ~140 GB of VRAM — exceeding even the 192 GB of NVIDIA's flagship B200 GPU, let alone edge devices. Quantization addresses this by reducing weight precision from 16-bit floats to 4-bit integers, shrinking

NVIDIA has announced the Jetson Orin Nano 2, an updated entry-level robotics and edge AI computer designed to double inference throughput over the Jetson Orin Nano Super while maintaining the identical physical form factor. The module delivers up to 78 trillion operations per second (TOPS) of AI compute and reduces power consumption by 40% when matched against its predecessor's performance baseline. Targeted at robotics, autonomous delivery drones, and edge computer vision deployments, the hard

Xiaomi has unveiled details of its custom semiconductor roadmap, introducing two specialized AI processors alongside its next-generation mobile system-on-chip: the 6-nanometer Xring O100 near-memory AI accelerator and the 3-nanometer Xring D100 autonomous driving chip. Both processors are manufactured by TSMC and have completed hardware validation ahead of planned commercial rollouts. Xiaomi has invested more than 20 billion yuan (about $3 billion) in Xring chip development, Reuters reported,
