Let's look at how macro processing happens. We did explore table creation in Chapter 1, Introduction to SAS Programming, where we discussed the compiler and execution phases. Macro processing is related to the compiler phase. The whole macro processing starts from the input buffer stage that we discussed earlier. The one aspect we didn't discuss in detail earlier was the word scanner. The word scanner is a component that reviews the characters from the input buffer and segregates them into tokens. Tokens are like atoms, the smallest pieces of information that can be held in the SAS processing engine. The process of breaking this information is called tokenization. The word scanner determines which part of SAS processing each token should be sent to.
The types of tokens are as follows:
- Literals: A string of characters enclosed in single or double...