Lexical Analysis and Tokens
Tokens, lexemes and patterns with a reference table, the two disambiguation rules (longest match, then the earliest rule) and how keywords are separated from identifiers. Script-counted token totals for C fragments including a+++b, a--->b, string literals with escapes and comments. A transition diagram for relational operators shows lookahead and retraction. Also covers input buffering with sentinels, lexical errors and their recovery, the structure of a Lex specification, and turning a regular expression into a minimal DFA.
