This chapter enables developers to design a new C# language feature and implement various compiler phases for this language feature. On a high level, the C# compiler has following important phases:
- Lexical Analysis (https://en.wikipedia.org/wiki/Lexical_analysis): This transforms the sequence of characters in a source file to into tokens such as keywords, identifiers, operators, and so on.
Lexer.Lex
(http://source.roslyn.io/#Microsoft.CodeAnalysis.CSharp/Parser/Lexer.cs,5ad0cc36317d33e7) is the primary entry point into the C# lexer that fetches the next token and increments the current position within the source text. For example, consider the following source code:
class C { void Method(int x) { x = x + 1; } }
This gets transformed into the following sequence of tokens during lexical analysis (white space and newline trivia excluded for brevity):
class (Keyword) C (IdToken) { (OpenBraceToken) void (Keyword) Method (IdToken) ( (OpenParenToken) int (Keyword) x (IdToken) ) ...