Xtext is an open source Eclipse framework for implementing Domain Specific Languages together with their integration in the Eclipse IDE. Xtext allows you to implement languages quickly by covering all aspects of a complete language infrastructure, starting from the parser, code generator, or interpreter, up to a full Eclipse IDE integration, with all the typical IDE features such as editor with syntax highlighting, code completion, error markers, automatic build infrastructure, and so on.
This book will incrementally guide you through the very basics of DSL implementation with Xtext and Xtend, such as grammar definition, validation, and code generation. The book will then cover advanced concepts such as unit testing, type checking, and scoping. Xtext comes with good and smart default implementations for all these aspects. However, every single aspect can be customized by the programmer.
Although Java can be used for customizing the implementation of a DSL, Xtext fosters the use of Xtend, a Java-like programming language completely interoperable with the Java type system which features a more compact and easier to use syntax and advanced features such as type inference, extension methods, multi-line template strings and lambda expressions. For this reason, we will use Xtend throughout the book.
Most of the chapters have a tutorial nature and will describe the main concepts of Xtext through uncomplicated examples. The book also uses test driven development extensively. The last chapters will describe more advanced topics such as Continuous Integration and Xbase, a reusable Java-like expression language that ships with Xtext which can be used in your DSLs.
This book aims at being complementary to the official documentation, trying to give you enough information to start being productive in implementing a DSL with Xtext. This book will try to teach you some methodologies and best practices when using Xtext, filling some bits of information that are not present in the official documentation.
The chapters are meant to be read in order, since they typically refer to concepts that were introduced in the previous chapters.
All the examples shown in the book are available online, see the section Downloading the example code. We strongly suggest that you first try to develop the examples while reading the chapters and then compare their implementations with the ones provided by the author.
After a small introduction to the features that a DSL implementation should cover, including integration in an IDE, the book will introduce Xtend since it will be used in all the examples. The book proceeds by explaining the main concepts of Xtext. For example, validation, code generation, and customizations of runtime and UI aspects. The book will then show how to test a DSL implemented in Xtext with JUnit in order to follow a Test Driven Development strategy that will help you to quickly implement cleaner and more maintainable code. The test-driven approach is used in the rest of the book when presenting advanced concepts such as type checking and Scoping. The book also shows how to build a DSL with Continuous Integration mechanisms and how to produce a release of your DSL so that others can install it in Eclipse. At the end of the book Xbase is introduced. Finally, the book describes some advanced topics and mechanisms that can be used when implementing an Xtext DSL.
Chapter 1, Implementing a DSL, gives a brief introduction to Domain Specific Languages (DSL) and sketches the main tasks for implementing a DSL and its integration in an IDE. The chapter also shows how to install Xtext and gives a first idea of what you can do with Xtext.
Chapter 2, Creating Your First Xtext Language, shows a first example of a DSL implemented with Xtext and gives an introduction to some features of the Xtext grammar language. The chapter describes the typical development workflow of programming with Xtext and provides a small introduction to EMF (Eclipse Modeling Framework), a framework on which Xtext is based.
Chapter 3, Working with the Xtend Programming Language, describes the main features of the Xtend programming language, a Java-like language interoperable with the Java type system. We will use Xtend in all the other chapters, to implement every aspect of an Xtext DSL.
Chapter 4, Validation, describes validation, in particular, the Xtext mechanism to implement validation, that is, the validator. This chapter is about implementing additional constraint checks that cannot be done at parsing time. It also shows how to implement quickfixes corresponding to the errors generated by the validator.
Chapter 5, Code Generation, shows how to write a code generator for an Xtext DSL using the Xtend programming language. The chapter also shows how a DSL implementation can be exported as a Java standalone command-line compiler.
Chapter 6, Customizing Xtext Components, describes the main mechanism for customizing Xtext components, Google Guice, a Dependency Injection framework. In particular, the chapter shows how to customize both the runtime and the UI aspects of an Xtext DSL.
Chapter 7, Testing, describes how to test a DSL implementation using JUnit and the additional utility classes provided by Xtext. The chapter shows the typical techniques for testing both the runtime and the UI aspects of a DSL implemented in Xtext.
Chapter 8, An Expression Language, covers the implementation of a DSL for expressions, including arithmetic, boolean, and string expressions. The chapter shows how to deal with recursive rules and with typical problems when writing Xtext grammars. The implementation will be described incrementally and in a test-driven way. The chapter also shows how to implement a type system for checking that expressions are correct with respect to types and how to implement an interpreter for these expressions. Some hints for optimizing the performance of a DSL implementation are also presented.
Chapter 9, Type Checking, covers the implementation of a small object-oriented DSL, which can be seen as a smaller version of Java that we call SmallJava. This chapter shows how to implement some type checking techniques that deal with object-oriented features, such as inheritance and subtyping. The chapter also describes some good practices in Xtext DSL implementations.
Chapter 10, Scoping, covers the main mechanism behind visibility and cross-reference resolution in Xtext. Since scoping and typing are often strictly connected and inter-dependent especially for object-oriented languages, the chapter is based on the SmallJava DSL introduced in the previous chapter. The chapter describes both local and global scoping and how to customize them.
Chapter 11, Continuous Integration, describes how you can release your DSL implementation by creating an Eclipse update site so that others can install it in Eclipse. The chapter also describes how to build and test your DSL implementation on a continuous integration server, using Maven/Tycho or Gradle. We will also show how to have a web application with a web editor for your DSL, and how your DSL implementation can be easily ported to IntelliJ.
Chapter 12, Xbase, describes Xbase a reusable expression language interoperable with Java. By using Xbase in your DSL, you will inherit all the Xbase mechanisms for performing type checking according to the Java type system and the automatic Java code generation.
Chapter 13, Advanced Topics, describes a few advanced topics concerning an Xtext DSL implementation, and some advanced techniques. In the first part of the chapter we will show how to manually maintain the Ecore model for the AST of an Xtext DSL. We will show how to create an Xtext DSL starting from an existing Ecore model, how to switch to a manually maintained Ecore model, starting from the one generated by Xtext, and how to use Xcore to maintain the Ecore model for the AST. In the last section we will show how to extend an Xbase DSL with new expressions, customizing the Xbase type system and the Xbase compiler in order to handle the new Xbase expressions.
Chapter 14, Conclusions, concludes the book by providing a few references to some additional material available online.
The book assumes that you have a good knowledge of Java; it also assumes that you are familiar with Eclipse and its main features. Existing basic knowledge of a compiler implementation would be useful, though not strictly required, since the book will explain all the stages of the development of a DSL.
This book is for programmers who want to learn about Xtext and how to use it to implement a DSL or a programming language together with the Eclipse IDE tooling.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "We can include other contexts through the use of the include directive."
A block of code is set as follows:
public static void main(String args[]) { System.out.println("Hello world");
Where keywords of the languages are typeset in bold, and references to static members are typeset in italics (for example, Java static methods).
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.4.0</version> <executions> <execution> <!-- new execution for generating EMF classes --> <id>mwe2GenerateEMFClasses</id>
Any command-line input or output is written as follows:
mvn org.eclipse.tycho:tycho-versions-plugin:set-version -DnewVersion=1.1.0-SNAPSHOT -Dtycho.mode=maven
Bibliographic references are of the form "Author" "year" when there is a single author, or "First author" et al. "year" when there is more than one author. Bibliographic references are used for books, printed articles or articles published on the web. The Bibliography can be found at the end of the book.
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Clicking the Next button moves you to the next screen." When the user is requested to select submenus, we separate each menu with a pipe, like this: "To create a new project, navigate to File | New | Project...".
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
The example code for this book is also available on a Git repository at
https://github.com/LorenzoBettini/packtpub-xtext-book-2nd-examples.
We suggest you to monitor this git repository, since it will always contain the most up-to-date version of the examples.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
The Errata is also available on the Git repository of the examples of the book https://github.com/LorenzoBettini/packtpub-xtext-book-2nd-examples.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.