An AI engineer builds products on top of models that someone else already trained. The job is less about inventing new neural networks and more about wiring models into software that real people use: prompts, retrieval, tools, evaluation, cost and reliability. That puts it within reach of working software developers, career changers and students who are willing to build things.
This guide explains what the role is, how it differs from machine learning engineering and research, what the public labor data does and does not say about it, and a staged roadmap with free resources we checked in October 2026. Every resource below links to its own page.
What an AI engineer does
The roadmap.sh AI Engineer roadmap gives a short definition: an AI engineer "uses pre-trained models and existing AI tools to improve user experiences," and focuses "on applying AI in practical ways, without building models from scratch." It contrasts this with AI researchers and ML engineers, who focus more on creating new models.
The title itself is recent. swyx argued for it in The Rise of the AI Engineer, published on Latent Space on June 30, 2023. His case was simple. Foundation models had turned tasks that once needed a research team into tasks that need API docs and an afternoon, and someone has to turn those capabilities into products. He estimated "~5000 LLM researchers in the world, but ~50m software engineers," and predicted that AI engineering "will likely be the highest-demand engineering job of the decade." That is a prediction from 2023, not a measured result, so treat it as one person's forecast.
The same post quotes Andrej Karpathy: "One can be quite successful in this role without ever training anything." That line sums up the main difference from older ML jobs.
In practice, the daily work looks like this:
- Choosing a model and calling it through an API (OpenAI, Anthropic, Google, or an open-weight model you host).
- Writing and testing prompts, including system prompts and structured output.
- Building retrieval so the model can answer from your own documents (retrieval-augmented generation, or RAG).
- Giving models tools and turning them into agents that take actions.
- Writing evaluations so you know when a change made things better or worse.
- Shipping: latency, cost, logging, security and a feedback loop from real users.
Chip Huyen's book AI Engineering (O'Reilly) is organized around nearly the same list. Its README says the book helps answer questions such as how to evaluate an application, what causes hallucinations, why RAG works, what an agent is, when to finetune, and how to make a model "faster, cheaper, and secure." The README also notes that it is "NOT a tutorial book," so pair it with hands-on courses.
AI engineer vs ML engineer vs AI researcher
The lines blur, and many teams use the titles loosely. A useful working split:
- AI researcher: creates new models and methods, runs training experiments, publishes papers. BLS says computer and information research scientists typically need at least a master's degree.
- ML engineer: trains, tunes and serves models, often on the company's own data. Strong on data pipelines, training infrastructure and classic ML.
- AI engineer: builds applications on top of foundation models. Strong on software engineering, prompting, retrieval, tool use and evaluation. May finetune, but usually starts from a pretrained model.
Swyx's post makes the same point about the boundary: AI engineers can "go left to finetune/host models," and research engineers can build on APIs too. Huyen's README draws a similar line between her two books. Her earlier book covers "tabular data annotations, feature engineering, and model training," while AI Engineering covers "prompt engineering, context construction, and parameter-efficient finetuning."
Is AI engineering a good career? What the data says
The US Bureau of Labor Statistics (BLS) Occupational Outlook Handbook has no separate "AI engineer" profile, so any single salary figure you see for "AI engineer" comes from job boards or surveys with their own methods. The closest BLS categories give a grounded baseline:
- Software developers, quality assurance analysts and testers: median pay of $134,040 per year (2025), with a bachelor's degree as the typical entry-level education, according to the related-occupations table on the BLS computer and information research scientists page.
- Data scientists: median pay of $120,230 (2025), bachelor's degree typical, from the same table.
- Computer and information research scientists: median annual wage of $140,300 in May 2025. BLS projects employment to grow 22 percent from 2025 to 2035, "much faster than the average for all occupations," with about 2,900 openings a year on average. BLS says these workers "typically need at least a master's degree," and names AI as one source of demand: "computer scientists' expertise will be needed in the creation of new technologies related to artificial intelligence (AI)."
By the roadmap.sh definition, AI engineering is applied software work, so the software developer line is the closer comparison for these roles. That has two practical effects. BLS lists a bachelor's degree, not a PhD, as the typical entry-level education for software developers. And you should plan to be a solid software engineer first, with the AI skills on top.
Entry-level hiring at AI companies has tightened in at least one public case. Anthropic reports that the proportion of junior technical hires there fell from 27% to 9%. We covered what Anthropic's hiring data means for entry-level AI jobs earlier today. If you are starting from zero, plan for a portfolio that proves you can ship, and do not count on a certificate alone.
Do you need a degree?
Not always. BLS lists a bachelor's degree as typical for software developers and a master's for research scientists. AI engineering job posts vary by company. Swyx's 2023 post listed well-known builders of AI products and wrote "Not a single PhD in sight." That describes a handful of people, not the whole market, but it reflects how the role is framed: shipping skill counts for more than credentials.
What a degree gives you is a fast filter past résumé screens and a math base. If you do not have one, a public portfolio of working projects, readable code and written evaluations does the same job more slowly.
The roadmap, stage by stage
The stages below build on each other. Move on when you can do the "you are ready when" test for a stage, not when a calendar says so. Each stage has a name so you can find it again; the diagram uses the same names.
Foundations: Python, Git and the web
AI engineering is software engineering first. You need comfortable Python, Git, HTTP and JSON, and enough command-line skill to run a project.
- The Python Tutorial from the official Python docs is free and covers the language from the start.
- Kaggle Learn offers free short courses, including tracks on Python and pandas.
- Google's Machine Learning Crash Course gives you the ML vocabulary. Its modules include neural networks, embeddings, an introduction to large language models and production ML systems.
You are ready when you can write a small Python script that calls a public web API, parses the JSON and saves the result, all in a Git repo.
Model APIs and Prompting
Next, learn to call models and control their output. Read one provider's docs end to end: the Claude developer platform docs, the Gemini API docs, or the OpenAI Cookbook.
- Anthropic's courses on GitHub contain five courses in a suggested order: API fundamentals, an interactive prompt engineering tutorial, real world prompting, prompt evaluations, and tool use. The README says the courses "often favor our lowest-cost model, Claude 3 Haiku, to keep API costs down," so expect some API cost.
- Microsoft's Generative AI for Beginners has 21 lessons on building generative AI applications.
You are ready when you can get a model to return valid structured output (for example, JSON that matches a schema) and explain why a prompt change helped.
RAG and Data
Many business AI apps need to answer from private data. That means chunking documents, creating embeddings, storing them, retrieving the right pieces and passing them to the model. Start with our plain-English explainer, What Is RAG?, then build one.
- The Hugging Face LLM Course says it is "completely free and without ads." It covers Transformers, Datasets and Tokenizers, fine-tuning a pretrained model, curating datasets, and a newer chapter on building reasoning models.
- For practice data, see our list of free datasets for machine learning projects.
You are ready when your app answers questions from a set of documents and shows which passages it used.
Tools and Agents
Agents are models that call tools in a loop to finish a task. Learn function calling first, then a framework.
- The Hugging Face AI Agents Course is free. It teaches smolagents, LlamaIndex and LangGraph, has a unit on agentic RAG, and ends with a final project to "Create, Test, and Certify Your Agent." You can follow it as a self-audit or take the certification path.
- The Model Context Protocol (MCP) docs describe MCP as "an open-source standard for connecting AI applications to external systems." Building one small MCP server is a good way to learn how tools get exposed to models.
You are ready when your agent can use at least two tools, recover from a failed tool call and stop when the task is done.
Evals
Evaluation is how you know a change made your app better instead of just different. Hamel Husain's post Your AI Product Needs Evals (March 29, 2024) lays out three levels: unit tests, human and model evaluation, and A/B testing. Anthropic's prompt evaluations course, listed in the Model APIs and Prompting stage, is a hands-on companion.
You are ready when you have a test set for your app, a script that scores it, and a before-and-after number for your last change.
Ship and Portfolio
Put projects online, log what users do, and track cost and latency. Then write about what you built.
Strong portfolio projects are narrow and real. A few patterns that show the full stack:
- A RAG assistant over a public document set (a city's council minutes, a product manual), with citations and an eval set.
- An agent that does one multi-step job end to end, such as triaging GitHub issues, with logs of every tool call.
- A cost and quality comparison of two or three models on the same task, using your own eval script.
For each project, include a README with what it does, how you evaluated it, what failed, and what you would fix next.
Optional: go deeper into how models work
You do not need to train models to be an AI engineer, but understanding them helps you debug. Two free options:
- Neural Networks: Zero to Hero by Andrej Karpathy starts with backpropagation and builds up to modern deep neural networks "like GPT." Its stated prerequisites are "solid programming (Python), intro-level math."
- Practical Deep Learning for Coders from fast.ai is a free course for people with some coding experience. Part 1 has 9 lessons of around 90 minutes each.
For a broader list of free options, see our best free AI courses in 2026.
How to get your first AI engineering job
- Start from where you are. If you already work as a software developer, one practical path is an internal move: volunteer for the AI feature on your team. In 2023, swyx predicted that informal AI Slack channels at startups would "turn from informal groups into formal teams."
- Show, then tell. Link two or three shipped projects with evals at the top of your résumé.
- Write in public. A short write-up of what broke in your RAG app and how you measured the fix shows the judgment the role needs.
- Pick a niche if it helps. Security is one example. We covered how AI security engineer postings rose while overall cybersecurity postings fell.
- Read the duties, not just the title. Companies use "AI engineer" loosely, so check whether a posting is really applied product work, ML training or research.
FAQ
How long does it take to become an AI engineer? It depends on your starting point. A working software developer can add the AI skills faster than someone learning to code from zero, because the Foundations stage is already done. Measure progress by the "you are ready when" tests above, not by weeks.
Is prompt engineering the same job? No. Prompting is one skill inside AI engineering. Swyx's 2023 post argued that AI engineers need to "go beyond the Prompt Engineer and write software."
Do I need to know PyTorch? Not to start. You need it if you move toward finetuning or ML engineering. The optional deeper stage above covers it.
Are certificates worth it? Free certificates, such as the Hugging Face Agents Course path, are a fine way to structure study. A certificate does not show that you can ship, so treat it as a bonus next to your projects, not the goal.
Sources
- roadmap.sh: AI Engineer roadmap
- Latent Space: The Rise of the AI Engineer (swyx, June 30, 2023)
- Chip Huyen: AI Engineering book repo and README
- Anthropic: Employment at Anthropic
- US Bureau of Labor Statistics: Computer and Information Research Scientists
- The Python Tutorial
- Kaggle Learn
- Google Machine Learning Crash Course
- Claude developer platform docs
- Gemini API docs
- OpenAI Cookbook
- Anthropic courses on GitHub
- Microsoft Generative AI for Beginners
- Hugging Face LLM Course
- Hugging Face AI Agents Course
- Model Context Protocol: What is MCP?
- Hamel Husain: Your AI Product Needs Evals
- Andrej Karpathy: Neural Networks: Zero to Hero
- fast.ai: Practical Deep Learning for Coders
