Subscribe
AI Voice
6 min read

What Is Text-to-Speech? How AI Voices Work, Explained Simply

Text-to-speech turns written text into a synthetic voice. This explainer walks through how modern neural TTS works, from text to phonemes to spectrograms to a vocoder, plus how to choose a voice and control pace, pauses and pronunciation.

Olivetti-style cover reading Text-to-Speech Explained, with a text block, an arrow and sound wave bars on cream paper
Olivetti-style cover reading Text-to-Speech Explained, with a text block, an arrow and sound wave bars on cream paper

Text-to-speech, usually shortened to TTS, is software that reads written text out loud in a synthetic voice. Microsoft's Azure documentation says text to speech "enables your applications, tools, or devices to convert text into human like synthesized speech," and notes that it "is also known as speech synthesis." The vendors list uses such as voice assistants, chatbots, car navigation, audiobooks and accessibility apps.

Older voices joined small pieces of recorded speech. Newer neural voices generate the sound with a model. This explainer covers what changed, how a TTS system turns letters into sound, and what you can control as a user, in plain English and without any code.

What text-to-speech does

The input is text. The output is an audio file or a live audio stream. Google's Cloud Text-to-Speech guide calls the process of turning text into audio "synthesis" and the result "synthetic speech." It says the service "converts text or Speech Synthesis Markup Language (SSML) input into audio data like MP3 or LINEAR16 (the encoding used in WAV files)."

Amazon describes its TTS service, Polly, in similar terms: a "cloud service that converts text into lifelike speech," according to the Polly developer guide.

TTS is the reverse of speech-to-text. Speech-to-text listens to audio and writes down the words. TTS takes words and speaks them.

Where you already hear it

The vendors list many of the same uses:

  • Voice assistants and chatbots. Azure lists making "interactions with chatbots and voice assistants more natural and engaging." Google's example is a voice assistant app that says out loud, "I've added the event to your calendar."
  • Audiobooks and long reads. Azure lists converting "digital texts such as e-books into audiobooks." ElevenLabs, in its text to speech overview, lists producing "audiobooks in multiple languages with complex emotional delivery."
  • Navigation. Azure lists enhancing "in-car navigation systems."
  • Accessibility and learning. Amazon lists newsreaders, games, eLearning platforms and "accessibility applications for visually impaired people."
  • Ads and media. ElevenLabs lists narrating "global media campaigns & ads."

How a TTS system turns text into a voice

Modern TTS works in a few stages. The exact design differs between companies, but Amazon's description of its neural engine is in its developer guide, so this section follows it.

How text-to-speech works: text goes in, is prepared and turned into phonemes, a neural network makes spectrograms, a vocoder makes the waveform, and audio comes out

Stage 1: Text goes in

You send plain text, or text marked up with SSML (more on that below). Amazon's How Polly works page says you choose a voice, provide the text and specify an audio output format.

Stage 2: The text is prepared

Written text is full of things that are not spoken the way they are spelled: numbers, abbreviations, acronyms, odd brand names. The system has to decide how to say them. Amazon gives examples in its guide to pronunciation lexicons. A TTS engine, it says, "reads the text literally, pronouncing the name exactly as it is spelled," so a stylized phrase like "g3t sm4rt" may come out wrong unless you give it an alias. The same applies to an acronym such as W3C.

Numbers follow the voice. Amazon says numbers written as digits "are synthesized in the language of the voice and not the text." Amazon also notes that Polly "is not a translation service," so the speech is in the same language as the text.

Stage 3: Text becomes phonemes

Next the words are mapped to phonemes. Amazon defines phonemes as "the most basic units of language," meaning the individual sounds that make up spoken words.

Stage 4: A neural network predicts the sound

Amazon's neural engine has two parts. The first is a neural network that converts a sequence of phonemes into a sequence of spectrograms. Amazon explains that spectrograms "are snapshots of the energy levels in different frequency bands." Think of a spectrogram as a picture of sound: which pitches are loud or quiet at each moment.

This part does more than look up one sound per letter. Amazon describes it as a sequence-to-sequence model that "doesn't create its results solely from the corresponding input but also considers how the sequence of the elements of the input work together." In other words, it looks at a whole stretch of input together rather than one sound at a time.

Microsoft describes the same goal from another angle. Its neural voices "overcome the limits of traditional speech synthesis regarding stress and intonation in spoken language," and "Prosody prediction and voice synthesis happen simultaneously, which results in more fluid and natural-sounding outputs." Put simply, the stress and intonation are worked out together with the sound itself.

Stage 5: A vocoder makes the audio

The second part of Amazon's engine is a vocoder, which converts spectrograms into "a nearly continuous audio signal." A vocoder is the step that turns the picture of sound into an actual waveform your speakers can play.

Stage 6: Audio comes out

The result is saved or streamed in a format such as MP3 or WAV. Amazon lists MP3 and Ogg Vorbis for web and mobile apps, PCM for devices and telephony, and Mu-law or A-law for telephony.

Why older voices sounded robotic

An older method, still used by Amazon's standard voices, is concatenative synthesis: joining together small pieces of recorded human speech. Amazon says its standard engine "concatenates phonemes of recorded speech, producing very natural-sounding synthesized speech," but that "the inevitable variations in speech and the techniques used to segment the waveforms limits the quality of speech."

Neural TTS replaces the joined recordings with a model that generates the sound. Amazon says its neural engine "doesn't use standard concatenative synthesis to produce speech." Microsoft says its approach uses "deep neural networks to make computer voices nearly indistinguishable from human recordings."

Choosing a voice

Every TTS request needs a voice. Google's guide says "you must specify a voice that 'speaks' the words," and that voices "differ by language, gender, and accent (for some languages)." Its example converts the same text into a British English voice and then an Australian English voice.

Beyond the built-in voices, the vendors offer three other ways to get a voice:

  • Custom brand voices. Microsoft says you can "create a custom voice that's unique to your product or brand," with "limited access for responsible use."
  • Voice cloning. ElevenLabs lists Instant Voice Cloning "for quick voice replication" and Professional Voice Cloning "for highest-fidelity replicas."
  • Designed voices. ElevenLabs also offers Voice Design "to generate custom voices from text descriptions." Google's Gemini 3.8 TTS goes a step further with voice remixing, which edits a saved voice with a text prompt.

What you can control

You can usually change how a voice speaks without changing the voice itself.

  • Basic settings. Google says its service "supports configuring the speaking rate, pitch, volume, and sample rate hertz."
  • SSML. Speech Synthesis Markup Language is a set of tags you add around text. Microsoft describes it as "an XML-based markup language used to customize text to speech outputs," and says it lets you "adjust pitch, add pauses, improve pronunciation, change speaking rate, adjust volume, and attribute multiple voices to a single document." Google says SSML lets you "insert pauses, acronym pronunciations, or other additional details." Support is not universal: Google notes it "does not support all SSML elements for all available languages."
  • Pronunciation dictionaries. A lexicon is a list of words with the way you want them said. Amazon applies a lexicon "to the input text before the synthesis begins." ElevenLabs' Studio guide describes pronunciation dictionaries for "character or brand names" and for how "acronyms should be read."
  • Sound effects on the voice. ElevenLabs offers audio effects that make speech sound "like it was recorded somewhere specific," such as a phone line, an old radio, a hall or a café.

Real-time speech versus long recordings

TTS is used in two different ways.

Real time. The voice has to start speaking almost immediately, as in a phone agent or a voice assistant. Microsoft lists "Real-time speech synthesis" as a feature. ElevenLabs lists streaming "real-time audio from text." In a cascaded voice agent, TTS is the last link in a chain: speech-to-text, then a language model, then TTS. We explain the trade-offs of that design in speech-to-speech vs cascaded voice agents, and look at one real-time voice product in our Eleven Realtime explainer. Some small voices run on the device itself with no cloud service; we compare two of them in Klein TTS X1.5 and Linnet: Tiny On-Device Voices.

Long form. For an audiobook or lecture, speed matters less than consistency. Microsoft offers "Asynchronous synthesis of long audio" for files "longer than 10 minutes (for example, audio books or lectures)," where you send the request and download the audio when it is ready.

Where TTS still goes wrong

  • Names and jargon. Because an engine can read text literally, unusual names, brand spellings and acronyms can come out wrong. A lexicon or an alias lets you set how to say them.
  • Uneven features by language. Google says SSML support varies by language, so a tag that works in one language may not work in another.

How to try it yourself

You do not need to write code to hear modern TTS. Microsoft points no-code users to the "Audio Content Creation tool in Speech Studio." Amazon says you can hear example Polly voices in your browser on its product overview. ElevenLabs links to a step-by-step product guide for using text to speech in ElevenLabs. Paste a paragraph, try two or three voices, then change the speaking rate or add a pause and listen to the difference.

Sources

AI VoiceExplainersGenerative AI