The 5 Phases of Natural Language Processing

After looking at what NLP is and how it’s applied, it’s time to learn how it actually works. There are five distinct phases that allow a machine to process and understand human language. Let's look at each one.

Lexical Analysis is also known as Morphological Analysis. This is the very first step where the sentence is broken down into structured components i.e lexemes or morphemes. Example: Can / you / pass / the / glass. Lexeme: Breaking down a given sentence into individual sub-words, words, or streams of characters. Morpheme: Identifying the smallest unit of a word that still holds distinct meaning. Goal: To convert an entire sentence into an organized stream of lexemes or morphemes.

This phase acts as a structural validation system. Goal: It focuses heavily on checking the grammar, evaluating structural word arrangements, and establishing formal relationships between the words within a sentence.

While syntax focuses on structure, semantics focuses entirely on the internal data logic. Goal: Used strictly to isolate and map out the literal, factual meaning of individual words, phrases, or full sentences.

This phase involves looking at groups of sentences where the meaning of one sentence is dependent on the one preceding it. Example: In the sequence "Ramu is in 4th standard. Ramu is a good boy," the system learns to swap out the repetitive subject for the pronoun "He" while successfully tracking that "He" still directly references "Ramu"

This is the final and most advanced step. Sometimes, humans say things that mean something completely different from their literal definitions. Pragmatic analysis helps the computer figure out the real-world context and hidden intent behind a sentence so it can take the right action. Example: If you ask a computer, "Can you pass the glass?" * Without pragmatics, it would think you are asking a literal yes/no question and just say "Yes" or "No" without moving. With pragmatics, it understands that your true intent is a polite request, and it will actually perform the action of passing the glass to you.