Book Image

Learn ARCore - Fundamentals of Google ARCore

Book Image

Learn ARCore - Fundamentals of Google ARCore

Overview of this book

Are you a mobile developer or web developer who wants to create immersive and cool Augmented Reality apps with the latest Google ARCore platform? If so, this book will help you jump right into developing with ARCore and will help you create a step by step AR app easily. This book will teach you how to implement the core features of ARCore starting from the fundamentals of 3D rendering to more advanced concepts such as lighting, shaders, Machine Learning, and others. We’ll begin with the basics of building a project on three platforms: web, Android, and Unity. Next, we’ll go through the ARCore concepts of motion tracking, environmental understanding, and light estimation. For each core concept, you’ll work on a practical project to use and extend the ARCore feature, from learning the basics of 3D rendering and lighting to exploring more advanced concepts. You’ll write custom shaders to light virtual objects in AR, then build a neural network to recognize the environment and explore even grander applications by using ARCore in mixed reality. At the end of the book, you’ll see how to implement motion tracking and environment learning, create animations and sounds, generate virtual characters, and simulate them on your screen.
Table of Contents (17 chapters)
Title Page
Packt Upsell


Augmented reality applications have moved from novelty to reality, and with the release of ARKit and now ARCore, have become more accessible to the average developer. Now virtually anyone with a grasp of a programming language can quickly build an AR experience using a variety of platforms. Google, with the release of ARCore, has now made this even easier and also provides support for multiple development platforms. This book will guide you through building AR applications using JavaScript and web in mobile with Java/Android and also in mobile with C# / Unity. Along the way, you will learn the fundamentals of building a quality AR experience for your user.

Who this book is for

This book is for any developer who wants to dive into building an augmented reality app with ARCore, but has no background in game or graphic programming. Although the book only assumes the reader has basic high-school level math, the reader should still have a firm grasp of at least one of the following programming languages: JavaScript, Java, or C#.

What this book covers

Chapter 1Getting Started, covers the fundamental concepts any modern AR app needs to tackle in order to provide a great experience to the user. We will learn the basic concepts of motion tracking, environmental understanding, and light estimation.

Chapter 2, ARCore on Android, is an introduction to Android development with Android Studio, where we show you how to install Android Studio and set up your first ARCore app.

Chapter 3, ARCore on Unity, discusses how to install and build an ARCore app with Unity. This chapter also shows you how to remotely debug an app using the Android development tools.

Chapter 4, ARCore on the Web, jumps into web development with JavaScript and focuses on how to set up your own simple web server with Node.js. Then, this chapter looks through the various sample ARCore templates and discusses how to extend those for further development.

Chapter 5, Real-World Motion Tracking, extends our learnings from the preceding chapter and extend one of the web examples to add a real-world motion tracking. Not only will this showcase several fundamentals of working with 3D concepts, but it will also demonstrate how ARCore tracks a user's motion.

Chapter 6, Understanding the Environment, jumps back to the Android platform and deal with how ARCore understands the user's environment. We will grasp how ARCore identifies planes or surfaces in the environment and meshes them for user interaction and visualization. Here, we will take a look at how to modify a shader in order to measure and colorize the points from the user.

Chapter 7, Light Estimation, explains the role that lighting and shadows play in selling the AR experience to the user. We learn how ARCore provides for the estimation of light and how it is used to light the virtual models placed by the user into the AR world.

Chapter 8, Recognizing the Environment, is where we cover the basics of Machine Learning and how essential is the technology to the success of the AR revolution. We then look to building a simple neural network that learns through supervised training using a technique called back propagation. After learning the basics of NN and deep learning, we look to a more complex example that demonstrates various forms of Machine Learning.

Chapter 9, Blending Light for Architectural Design, covers the building of an AR design app that allows the user to place virtual furniture in the living space or wherever they need to. We also cover how to place and move an object in AR using touch and how to identify when an object is selected. Then, we will extend our lighting and shadows from Chapter 7Light Estimation and provide real-time shadows on the virtual objects.

Chapter 10, Mixing in Mixed Reality, is where we introduce mixed reality through the use of inexpensive MR headsets. ARCore is ideally suited for use in these inexpensive headsets since it already tracks the user and monitors their environment internally. We will oversee how to turn our app from a traditional mapping app using the 3D WRLD API for Unity to an AR mapping app, where we will also provide an option to switch to MR and an MR headset.

Chapter 11, Performance Tips and Troubleshooting, covers techniques for measuring an app's performance on all the development platforms we deal with. We then talk about the importance of performance and the impact it can have to the various systems. After that, we cover general debugging and troubleshooting tips, where we finish off with a table that covers the most common errors a user may encounter in this book.

To get the most out of this book

These are the things to be remembered in order to use this book to the fullest:

  • The reader will need to be proficient in one of the following programming languages: JavaScript, Java, or C#
  • A memory of high-school mathematics
  • An Android device that supports ARCore; the following is the link to check the list:
  • A desktop machine that will run Android Studio and Unity; a dedicated 3D graphics card is not explicitly required

Download the example code files

You can download the example code files for this book from your account at If you purchased this book elsewhere, you can visit and register to have the files emailed directly to you.

You can download the code files by following these steps:

  1. Log in or register at
  2. Select the SUPPORT tab.
  3. Click on Code Downloads & Errata.
  4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR/7-Zip for Windows
  • Zipeg/iZip/UnRarX for Mac
  • 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here:

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Scroll down to the draw method and add the following code beneath the identified line."

A block of code is set as follows:

void main() {
   float t = length(a_Position)/u_FurthestPoint;
   v_Color = vec4(t, 1.0-t,t,1.0);
   gl_Position = u_ModelViewProjection * vec4(, 1.0);
   gl_PointSize = u_PointSize;

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

uniform mat4 u_ModelViewProjection;
uniform vec4 u_Color;
uniform float u_PointSize;
uniform float u_FurthestPoint;

Any command-line input or output is written as follows:

cd c:\Android
npm install http-server -g

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."


Warnings or important notes appear like this.


Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit


Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit