Tokens

Examples > Calculator > Tokens

The tokens for the operators and the parenthesis are defined directly inside of the productions.

"+", "-", "*","/","(", ")"

A complex token for the recognition of numbers is defined as a regular expression on the token page:

number  = \d+(\.\d*)?|\.\d+

"\d" a single digit

"\d+" a repeat of digits, at least one

"\." the dot (without backslash '\' the dot has a special meaning inside of a regular expression).

"\d*" a repeat of digits, or no digit

"?" an optional occurrence of the content of the preceding parenthesis

"|" an alternative

So a number is defined either as a sequence of digits, optional followed by a dot followed by null or more digits ( e.g: "123" or "123." or "123.45"). Or a number begins with a dot followed by a sequence of digits (e.g.: ".45")

This page belongs to the TextTransformer Documentation