Book Image

Build Your Own Programming Language

By : Clinton L. Jeffery
Book Image

Build Your Own Programming Language

By: Clinton L. Jeffery

Overview of this book

The need for different types of computer languages is growing rapidly and developers prefer creating domain-specific languages for solving specific application domain problems. Building your own programming language has its advantages. It can be your antidote to the ever-increasing size and complexity of software. In this book, you’ll start with implementing the frontend of a compiler for your language, including a lexical analyzer and parser. The book covers a series of traversals of syntax trees, culminating with code generation for a bytecode virtual machine. Moving ahead, you’ll learn how domain-specific language features are often best represented by operators and functions that are built into the language, rather than library functions. We’ll conclude with how to implement garbage collection, including reference counting and mark-and-sweep garbage collection. Throughout the book, Dr. Jeffery weaves in his experience of building the Unicon programming language to give better context to the concepts where relevant examples are provided in both Unicon and Java so that you can follow the code of your choice of either a very high-level language with advanced features, or a mainstream language. By the end of this book, you’ll be able to build and deploy your own domain-specific languages, capable of compiling and running programs.

Related Content you might be interested in

Current Title:

Small book image
Build Your Own Programming Language
Clinton L. Jeffery
Dec, 2021 | 494 pages
No titles found
Table of Contents (25 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Code in Action
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Section 1: Programming Language Frontends
Section 1: Programming Language Frontends
Free Chapter
2
Chapter 1: Why Build Another Programming Language?
Chapter 1: Why Build Another Programming Language?
So, you want to write your own programming language…
Language versus library – what's the difference?
Applicability to other software engineering tasks
Establishing the requirements for your language
Case study – requirements that inspired the Unicon language
Summary
Questions
3
Chapter 2: Programming Language Design
Chapter 2: Programming Language Design
Determining the kinds of words and punctuation to provide in your language
Specifying the control flow
Deciding on what kinds of data to support
Overall program structure
Completing the Jzero language definition
Case study – designing graphics facilities in Unicon
Summary
Questions
4
Chapter 3: Scanning Source Code
Chapter 3: Scanning Source Code
Technical requirements
Lexemes, lexical categories, and tokens
Regular expressions
Using UFlex and JFlex
Writing a scanner for Jzero
Regular expressions are not always enough
Summary
Questions
5
Chapter 4: Parsing
Chapter 4: Parsing
Technical requirements
Analyzing syntax
Understanding context-free grammars
Using iyacc and BYACC/J
Writing a parser for Jzero
Improving syntax error messages
Summary
Questions
6
Chapter 5: Syntax Trees
Chapter 5: Syntax Trees
Technical requirements
Using GNU make
Learning about trees
Creating leaves from terminal symbols
Building internal nodes from production rules
Forming syntax trees for the Jzero language
Debugging and testing your syntax tree
Summary
Questions
7
Section 2: Syntax Tree Traversals
Section 2: Syntax Tree Traversals
8
Chapter 6: Symbol Tables
Chapter 6: Symbol Tables
Technical requirements
Establishing the groundwork for symbol tables
Creating and populating symbol tables for each scope
Checking for undeclared variables
Finding redeclared variables
Handling package and class scopes in Unicon
Testing and debugging symbol tables
Summary
Questions
9
Chapter 7: Checking Base Types
Chapter 7: Checking Base Types
Technical requirements
Type representation in the compiler
Assigning type information to declared variables
Determining the type at each syntax tree node
Runtime type checks and type inference in Unicon
Summary
Questions
10
Chapter 8: Checking Types on Arrays, Method Calls, and Structure Accesses
Chapter 8: Checking Types on Arrays, Method Calls, and Structure Accesses
Technical requirements
Checking operations on array types
Checking method calls
Checking structured type accesses
Summary
Questions
11
Chapter 9: Intermediate Code Generation
Chapter 9: Intermediate Code Generation
Technical requirements
Preparing to generate code
An intermediate code instruction set
Annotating syntax trees with labels for control flow
Generating code for expressions
Generating code for control flow
Summary
12
Chapter 10: Syntax Coloring in an IDE
Chapter 10: Syntax Coloring in an IDE
Downloading the example IDEs used in this chapter
Integrating a compiler into a programmer's editor
Avoiding reparsing the entire file on every change
Using lexical information to colorize tokens
Highlighting errors using parse results
Adding Java support
Summary
13
Section 3: Code Generation and Runtime Systems
Section 3: Code Generation and Runtime Systems
14
Chapter 11: Bytecode Interpreters
Chapter 11: Bytecode Interpreters
Technical requirements
Understanding what bytecode is
Comparing bytecode with intermediate code
Building a bytecode instruction set for Jzero
Implementing a bytecode interpreter
Writing a runtime system for Jzero
Running a Jzero program
Examining iconx, the Unicon bytecode interpreter
Summary
Questions
15
Chapter 12: Generating Bytecode
Chapter 12: Generating Bytecode
Technical requirements
Converting intermediate code to Jzero bytecode
Comparing bytecode assembler with binary formats
Linking, loading, and including the runtime system
Unicon example – bytecode generation in icont
Summary
Questions
16
Chapter 13: Native Code Generation
Chapter 13: Native Code Generation
Technical requirements
Deciding whether to generate native code
Introducing the x64 instruction set
Using registers
Converting intermediate code to x64 code
Generating x64 output
Summary
Questions
17
Chapter 14: Implementing Operators and Built-In Functions
Chapter 14: Implementing Operators and Built-In Functions
Implementing operators
Writing built-in functions
Integrating built-ins with control structures
Developing operators and functions for Unicon
Summary
Questions
18
Chapter 15: Domain Control Structures
Chapter 15: Domain Control Structures
Knowing when you need a new control structure
Scanning strings in Icon and Unicon
Rendering regions in Unicon
Summary
Questions
19
Chapter 16: Garbage Collection
Chapter 16: Garbage Collection
Appreciating the importance of garbage collection
Counting references to objects
Marking live data and sweeping the rest
Summary
Questions
20
Chapter 17: Final Thoughts
Chapter 17: Final Thoughts
Reflecting on what was learned from writing this book
Deciding where to go from here
Exploring references for further reading
Summary
21
Section 4: Appendix
Section 4: Appendix
22
Assessments
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 11
Chapter 12
Chapter 13
Chapter 14
Chapter 15
Chapter 16
Why subscribe?
23
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Appendix: Unicon Essentials
Appendix: Unicon Essentials
Running Unicon
Using Unicon's declarations and data types
Evaluating expressions
Debugging and environmental issues
Function mini-reference
Selected keywords

Preface

After 60 years of high-level language development, programming is still too difficult. The demand for software of ever-increasing size and complexity has exploded due to hardware advances, while programming languages have improved far more slowly. Creating new languages for specific purposes is one antidote for this software crisis.

This book is about building new programming languages. The topic of programming language design is introduced, although the primary emphasis is on programming language implementation. Within this heavily studied subject, the novel aspect of this book is its fusing of traditional compiler-compiler tools (Flex and Byacc) with two higher-level implementation languages. A very high-level language (Unicon) plows through a compiler's data structures and algorithms like butter, while a mainstream modern language (Java) shows how to implement the same code in a more typical production environment.

One thing I didn't really understand after my college compiler class is that the compiler is only one part of a programming language implementation. Higher-level languages, including most newer languages, may have a runtime system that dwarfs their compiler. For this reason, the second half of this book spends quality time on a variety of aspects of language runtime systems, ranging from bytecode interpreters to garbage collection.

Previous Section
Next Section