-
Book Overview & Buying
-
Table Of Contents
Practical System Programming for Rust Developers
By :
The tokenizer is the module in our system design that reads one or more characters from an arithmetic expression and translates it into a token. In other words, input is a set of characters and output is a set of tokens. In case you are wondering, examples of tokens are Add, Subtract, and Num(2.0).
We have to first create a data structure for two things:
In the following section, we will delve into how to determine the right data structures for the tokenizer module.
To store the input arithmetic expression, we can choose among the following data types:
We will choose the &str type, as we do not need to own the value or dynamically increase the size of the expression. This is because the user will provide the arithmetic expression once, and then the expression won't change for...