Lexical analysis

Glossary > Lexical analysis

The lexical analysis takes a source text into its elementary pieces that are the lexemes, which are interpreted as token. This process precedes the parsing of the text at least by one step. The parsing analyses the grammatical connections of the token.

This page belongs to the TextTransformer Documentation