# Microsoft’s speech synthesis patent: combining sounds, words and sentence structure

US12094447B2 explains how phoneme or character information can be combined with word embeddings and grammatical structure to generate speech.

Source: https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts

HOME / NEWS & INSIGHTS NEWS & INSIGHTS Microsoft’s speech synthesis patent: combining sounds, words and sentence structure US12094447B2 explains how phoneme or character information can be combined with word embeddings and grammatical structure to generate speech. Artificial Intelligence 2026.10.28 published IPLEX 8 min read A spoken customer notice needs clear pronunciation and a coherent rhythm. Microsoft’s US12094447B2 describes a text-to-speech architecture that brings together information at several linguistic levels. A fictional dispatch notice helps explain how those inputs are aligned and ultimately converted into an audible waveform. 1. The same message can sound different Imagine an online shop reading: ‘Your order will be dispatched tomorrow.’ On a screen, the message is immediately apparent. In speech, its clarity also depends on how the words connect, where the voice pauses and which parts receive emphasis. A fixed text does not determine a single possible delivery. Text-to-speech, or TTS, converts an existing written passage into synthesized speech. It must produce more than individual sounds. Pitch, duration, stress and pauses contribute to prosody: the rhythm and intonation that shape an utterance. The patent supplements phoneme or character information with word-level and sentence-level information. Pronunciation units are therefore processed alongside representations of linguistic context and structure. The origins of those representations and the points at which they are combined help define the technical approach. This is a synthesis process for a given message. Selecting the answer or writing a new customer notice is a separate operation. The question here is how the system calculates the sound of the text already supplied. 2. Three levels of information Phonemes distinguish the sounds of speech, whereas character-level information follows the written character sequence. The disclosed approach can start from phonemes or characters; the explanation does not require both types of sequence to be supplied together in every implementation. At word level, the model uses representations learned from language data to capture information about words and their relationships. A dispatch notice supplies a different linguistic setting from another kind of passage. The words are treated as more than arbitrary identifiers. Sentence-level information describes grammatical organization. It can identify the phrases to which a word belongs, whether it lies at a phrase boundary and its relative position within a phrase. These features carry the organization of smaller groups within the sentence into the synthesis process. None of these inputs is finished audio. They represent pronunciation units, word information and sentence structure in forms that a model can process. Keeping the levels distinct makes the need for alignment and combination easier to understand. 3. Preparing word and grammatical representations A word embedding represents a word numerically for use by a neural network. The patent describes obtaining word information through a pretrained embedding model, allowing knowledge learned from language material to contribute to speech synthesis. Grammatical parsing supplies another representation. A parse tree organizes a sentence into phrases and words. From that structure, the system can extract such features as phrase type, whether a word marks a boundary and the word’s relative position within a phrase. For illustration, the dispatch message can be viewed as a group referring to the ordered goods and another describing tomorrow’s dispatch. This is a simple reading aid for the fictional example. An actual parser’s output depends on the language and parsing method. The grammatical structure is converted into computational features associated with individual words. It is not an additional explanatory passage appended to the message. The resulting representation supplies structural information that the speech model can use. 4. Aligning sequences of different lengths A word sequence and a character or phoneme sequence generally have different lengths: one word can contain several characters or phonemes. Simply placing the sequences side by side would leave their correspondence unclear. One disclosed approach repeats each word’s information to match the number of corresponding phonemes or characters. Word-based grammatical features can be aligned in the same way. This up-sampling places information on the finer-grained sequence used by the model. In the source’s simplified Korean example, one word meaning ‘goods’ comprises two written syllable blocks. The same word-level information is associated with both positions. The word is not spoken twice; each relevant position can consult information about the word to which it belongs. This alignment concerns textual representations. It does not lengthen a recording or change playback speed. The temporal structure of the eventual speech is generated later, after the text-derived information has been combined. 5. From combined information to a waveform The prepared inputs enter a multi-input encoder. An encoder transforms input data into representations that subsequent neural processing can use. Here, phoneme or character information and contextual information contribute to text features. The description includes different combination points. Separate networks can process the inputs before their results are concatenated, or inputs can be combined before neural processing. Identifying both the inputs and the point of combination helps distinguish the architectures. Attention and a decoder connect text features to acoustic features. Attention helps associate relevant text information with the sound being generated. This is a different correspondence from the earlier alignment between word features and character or phoneme positions. Acoustic features are intermediate representations, for example describing the frequency-related characteristics of sound. They are not yet the final signal for a loudspeaker. A vocoder uses those features to generate the speech waveform, completing the path from linguistic inputs to audible output. 6. Following a fictional dispatch notice The online shop has already selected the message ‘Your order will be dispatched tomorrow.’ That sentence is the synthesis input. Looking up the actual dispatch date and checking the truth of the notice are separate from this illustrative TTS operation. The system prepares a phoneme or character sequence, obtains word information from the same sentence and derives grammatical groupings. These are different representations of one notice, not three newly written messages. Word and grammatical features are then associated with the corresponding phoneme or character positions. With a shared positional basis, the encoder can process a pronunciation unit together with information about its word and sentence structure. Combined text features lead to acoustic features, which the vocoder turns into a waveform. The operator can listen to how names or product names are pronounced and whether the phrasing communicates the notice clearly. The illustration concerns the generated result and its input processing; it is not an Azure interface, recording or measured product result. Fictional shop example: three levels of information from one dispatch notice are aligned and combined, then a vocoder generates speech. This is not an Azure screen, recording or measurement of a particular voice. 7. Azure Speech and the design of speech services Microsoft’s Azure Speech provides text-to-speech capabilities. Its official documentation describes neural voices, synthesis through an SDK or REST API, and tools for creating audio content. Such functions can support customer notices and the reading of written documents. Speech Synthesis Markup Language, or SSML, lets a service user specify aspects of delivery, including sentence structure, pauses, pronunciation, speaking rate and pitch. These output controls should be distinguished from the patent’s internal processing of word and sentence information. The documents do not establish that Azure implements the particular patented architecture. For a development team, the useful distinctions concern the levels of input and their combination points. Record whether the model receives characters or phonemes, whether learned word representations are added, how grammatical structure is encoded and how sequences of different lengths are aligned. A technical design record is more informative when it follows input, intermediate representation, alignment, combination, acoustic features and waveform generation. Merely stating that an AI system reads naturally does not identify the processing change. The sequence helps separate improvements to pronunciation inputs from improvements that use contextual information. A team seeking to improve spoken delivery while keeping the message unchanged can begin by examining how word and sentence information reaches the speech model. Describing the intermediate steps between visible text and audible output provides a concrete basis for understanding the technology and considering its patent implications. The original figure labels and forms are preserved; the text explains the source material. Related service: AI patents Explore IPLEX’s AI patent work and discuss the technical features of your speech service. Original Tistory article · Patent publication: US12094447B2 Related AI analyses: Naver’s image-translation patent: keeping translated text aligned with a moving camera · Baidu’s RAG approach to conflicts between model answers and retrieved knowledge Read the Korean source This article reflects the information available when it was published. Contact us to discuss your circumstances. Discuss this topic ↗ All articles ON THIS PAGE 1. The same message can sound different 2. Three levels of information 3. Preparing word and grammatical representations 4. Aligning sequences of different lengths 5. From combined information to a waveform 6. Following a fictional dispatch notice 7. Azure Speech and the design of speech services TALK TO IPLEX Discuss your IP questions We consider your technology and business needs together. ↗ Contact us Newer HK Acharya & Company visits IPLEX ↗ Older When a design registration is invalidated: loss of legal interest in a scope-of-rights action ↗ Related insights Artificial Intelligence 2026.10.29 SwiLA: organizing memory around context How Switching Linear Attention combines linear regressors within a fixed recurrent state, what its recall experiments establish, and where capacity and computational limits remain. ↗ Read article Artificial Intelligence 2026.10.27 GBLA: reading long recommendation histories efficiently A technical and patent-practice reading of gated bidirectional linear attention, its hybrid encoder and the limits of the reported speed and retrieval results. ↗ Read article Artificial Intelligence 2026.10.22 DFlare: widening the information path to a speculative drafter Layer-specific feature fusion, separate KV projections and a progressive training loss increase the capacity of a block-diffusion drafter, with measured gains and important limits. ↗ Read article

- https://www.iplexlaw.co.kr/en
- https://www.iplexlaw.co.kr/en/blog/category/ai
- https://www.iplexlaw.co.kr/en/ai-patents
- https://iplexlaw.tistory.com/13
- https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/12094447
- https://www.iplexlaw.co.kr/en/blog/naver-papago-ar-image-translation
- https://www.iplexlaw.co.kr/en/blog/baidu-rag-knowledge-conflict-resolution
- https://www.iplexlaw.co.kr/ko/blog/microsoft-multilevel-tts
- https://www.iplexlaw.co.kr/en/contact
- https://www.iplexlaw.co.kr/en/blog
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-0
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-1
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-2
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-3
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-4
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-5
- https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts#article-13-en-6
- https://www.iplexlaw.co.kr/en/contact
- https://www.iplexlaw.co.kr/en/blog/visit-hk-acharya-2026
- https://www.iplexlaw.co.kr/en/blog/brunch-case-1888
- https://www.iplexlaw.co.kr/en/blog/swila-switching-linear-attention
- https://www.iplexlaw.co.kr/en/blog/swila-switching-linear-attention
- https://www.iplexlaw.co.kr/en/blog/gbla-generative-retrieval
- https://www.iplexlaw.co.kr/en/blog/gbla-generative-retrieval
- https://www.iplexlaw.co.kr/en/blog/dflare-block-diffusion-drafting
- https://www.iplexlaw.co.kr/en/blog/dflare-block-diffusion-drafting