Book Image

Unreal Engine 4 Virtual Reality Projects

By : Kevin Mack, Robert Ruud
Book Image

Unreal Engine 4 Virtual Reality Projects

By: Kevin Mack, Robert Ruud

Overview of this book

Unreal Engine 4 (UE4) is a powerful tool for developing VR games and applications. With its visual scripting language, Blueprint, and built-in support for all major VR headsets, it's a perfect tool for designers, artists, and engineers to realize their visions in VR. This book will guide you step-by-step through a series of projects that teach essential concepts and techniques for VR development in UE4. You will begin by learning how to think about (and design for) VR and then proceed to set up a development environment. A series of practical projects follows, taking you through essential VR concepts. Through these exercises, you'll learn how to set up UE4 projects that run effectively in VR, how to build player locomotion schemes, and how to use hand controllers to interact with the world. You'll then move on to create user interfaces in 3D space, use the editor's VR mode to build environments directly in VR, and profile/optimize worlds you've built. Finally, you'll explore more advanced topics, such as displaying stereo media in VR, networking in Unreal, and using plugins to extend the engine. Throughout, this book focuses on creating a deeper understanding of why the relevant tools and techniques work as they do, so you can use the techniques and concepts learned here as a springboard for further learning and exploration in VR.
Table of Contents (20 chapters)
Title Page
About Packt
Where to Go from Here

Planning your VR project

We've talked quite a lot about VR in the abstract—what we can do with it, and what we think we know so far about how it works and what works well within it. From here on out, this book is going to get pretty practical and hands-on, and our hope is that, as we go through these projects and learn how to build VR experiences in Unreal, these principles we just talked about stay in your mind and guide your choices.

With that in mind, there's one last topic we should explore before we start getting our hands dirty, which is how to turn an idea into a thing you can actually make.

Clarify what you're trying to do

The first thing to do in developing a design is to decide what it's for. This may sound obvious, but it happens all too often that developers jump right into a project and start building without first taking a step back and figuring out what it is that they're really trying to make and who it's for. The result, more often than not, is either an unfocused experience that doesn't really achieve what it was intended to do because the parts aren't all working together to support a common goal, or a project that takes a long time to complete. This wastes a lot of work, as developers discover things that need to change and have to throw out existing work to make the changes. By taking some time to plan before you begin building software, you can save yourself a lot of effort and make it more likely that the project will succeed.



The first thing to remember in design is that the more you build, the more difficult and expensive changes get, so try to make these decisions as early in the process as you can. The cheapest prototype you can make is in your own mind. The second cheapest is on paper. Once you start building software, start with the bare minimum you need to get your project running—a gray box environment, or a simple prototype, and test it to see how it needs to change. You're almost guaranteed to discover a few things you hadn't anticipated, and this is the time to discover these things and change what you need. Once you've gone through this process, discovered what really works and what doesn't, and adjusted your design to respond to what you learned, now you're ready to begin putting expensive art and polish into the work. Too many developers work backward and try to make the final product right out of the gate, and they get locked into decisions that could have been changed more easily if the legwork had been done first.

With that in mind, the first thing to think about is who the project is for and why you're making it for them. Is this a game or an entertainment experience? What do you want the user to feel? What will they be doing while they play or participate in the experience? The same questions apply to cinematic VR—what's this experience about? What story are you trying to tell? Take a moment to write this down.

If you're making a learning experience, what does the user need to learn? What's the best way to teach it?

If you're making an architecture or design visualization application, what's important to your end user? An architect or engineer may want to be able to look inside walls and structures to see electrical and plumbing designs, while a real-estate buyer may care more about the quality of light in the space.

Figure out who your user is, and what's important to them, and clarify what you're trying to create and what's important to you. This should be done on paper. It's very easy for vague design elements to hide in your mental model, only to reveal holes or unexpected questions when you start to write them down.

Is it a good fit for VR? Why?

Pretty much the next thing you should do once you've clarified your design intention for a VR project is to think about how it fits in VR.

Think about your project in terms of what VR allows you to do. Does it rely on immersion and a strong sense of presence to work? Is it about making use of VR's ability to simulate the body or to give context to information? Why does your project work better in VR than it would on a flat screen? What can your user do or experience that they couldn't in traditional media?

Think about the challenges VR imposes as well. As we've seen in the best practices mentioned, VR imposes a different set of challenges than traditional media. Simulator sickness is a major concern—does your project require you to move the camera in ways that are going to be uncomfortable for your users? Does it rely on your users to move in a way that may be difficult or impossible in VR? Are you asking your users to read lots of small text that may not be legible on current headsets? Think about the practices we outlined, and evaluate whether any of them pose challenges to your design. This doesn't necessarily mean your design can't work in VR, but it does mean that you'll have to do some additional design thinking to work through those challenges.

Your choice to put your project into VR should be made deliberately. You should be able to describe why your project works better in VR than in traditional media, and how you plan to handle the challenges VR imposes. This too should happen in writing. You'll probably discover opportunities you hadn't seen, and a few challenges you hadn't realized you'd need to overcome. Writing these down will help you to understand what's important about your project and what needs to happen for it to succeed.

What's important – what has to exist in this project for it to work? (MVP)

Now that you've clarified who your project is for, what you intend it to do, and why it makes sense to do it in VR, you're ready to begin figuring out what it's really going to take to build it. It's helpful to figure this out in terms of a Minimum Viable Product (MVP). This, simply put, is a version of the product that contains only what's needed for it to satisfy its intention. An architectural visualization project, for example, needs to put the viewer into the building at the correct scale, and give the user some way to move around and see what it looks and feels like from different perspectives. What your MVP contains is your choice as a designer, but you should be clear about whether the thing you're talking about is a thing you need or a thing you want. If the project simply isn't worth doing if you can't get a given feature into it, then it's a needed feature and should go into your MVP. If it would improve the experience but users could still get what they needed without it, it's not part of the MVP.


MVP refers to a version of the project that contains only what's needed to satisfy its goal and little or nothing else. Clarifying your MVP can help you to understand what the spine of your project is, which can tell you what to prioritize and gives you a baseline from which to evaluate whether your project succeeds at achieving what it set out to do.

The contents of your MVP will differ greatly between different types of projects—the needs of a cinematic VR experience are substantially different than those of an engineering visualization application, but as a designer, you should know what they are and write them down. You don't need to write a book or an essay here—a list of bullets should be enough, but for each item on the list, ask yourself whether the project could still do what it's intended to do without it, and be clear about your answers. Wants, even strong wants, aren't needs. The point here is to know where your floor is.

Be on the lookout as well for things you missed. Imagine your user using your project—what are they trying to do, from moment to moment, from the moment they start up the application until the moment they shut it down? Use this exercise to discover items you missed, and figure out whether they're wants or needs, and get them on to the list if they belong there.

Break it down

If you've gone through the preceding exercises, you should have a clear idea of what your project is for, why it works in VR, and what needs to be in it for it to work. Now you're ready to figure out how to do it.

For the items in your MVP, what do you need to make them exist? Do you need a UI element to display information to the user? Do you need a way for your user to move around? Does the user need to be able to load or save information, or connect to a server?

For each item in your list, figure out what that item really requires you to build and write it down. You should come out of this exercise with a pretty clear breakdown of the things you need to do to get your project built.


breakdown is a list of things you need to do or build to get your project made. Use it as a tool to ensure that you haven't missed required elements, or underestimated risks, and to see whether the project you're trying to build is realistically achievable with the time and resources you have. It's a tool for spotting problems early, while you still have a chance to fix them, and then later for tracking your progress as you build.

Look through this list—where are the big jobs, and where are the big risks? Can you achieve all of this with the time and resources you have? Do you need to re-evaluate your scope if it's starting to look too big? Bear in mind that it's almost always better to do fewer things well than to try to do everything and do a poor job of it. It's common, at this point, to discover that the project scope exceeds what you can realistically do well, and this is a good thing. The time to discover this is now while it's still on paper and you can reorganize the work, move items off the MVP, or change your schedule or resources. If you discover these problems near the start, you have a fighting chance of solving them, whereas if you discover them only once you're months-deep into software development, you may discover that you've painted yourself into a corner. Set yourself up for success on paper while you still have the flexibility to do so.

Tackle things in the right order

Some items in your breakdown will be easier to do than others, and some will be more fun. Use your judgement when you figure out the order in which you should do things. In general, it's a good idea to tackle the risky things first. If something's important enough to be in your MVP, and there's a risk that it could go long, or might not work at all, it's often smart to get it out of the way early. Doing this gives you time to iterate on a risky item while you work on other things, or in the worst-case scenario, if you discover that a thing you'd counted on just can't be done, you're still early enough in your project that you may be able to fall back to another plan. Don't leave high-risk, high-priority items to the end—you'll be in trouble if something goes wrong.

Look for dependencies between items. If something can't be done before another thing is done—a character, for example, that can't be animated until the character is built and rigged, then make sure those dependencies factor into your plan. It does you no good to plan to do a thing in a certain order and then discover that you can't because something it depends on isn't ready.


As you plan how you're going to get through your breakdown, look for items that involve risks or uncertainties, items that are just going to take a long time, and items that depend on other items. Factor these into your plan. In general, where you can, do your high-importance, high-risk work early in your project so you have time to handle things if something goes wrong.

A word about project management—there's a forest of literature out there about planning and tracking projects, and discussing them in depth falls outside the scope of this book. Broadly, these fall into two major schools of thought: waterfall and agile. Waterfall project management methods lay out tasks in a rigid order that assumes that, once one task is done, the next can begin. This works well when the things you're doing are well-defined and don't entail much risk, such as painting a house, but VR design and development rarely works out like this. You simply may not know whether a feature is done until you see it running alongside other systems, and you may have to loop back at that point and change something or rework it entirely. Agile methods, such as Scrum, take this reality into account, and are intended for design and development projects where things are going to need to be revisited as the project evolves and reveals new information. In general, agile methods work much better for software development than waterfall plans.

Depending on the scope of the project, you may not need to apply a formal project management method, but even if you're planning loosely, there should still be a plan, and you should make sure the plan accommodates the reality that you're going to have to loop back and iterate on features and design, that some items are going to depend on others, and that some things are going to take longer than you thought.

Test early and often

Test your designs as early as you can. VR especially is a very new medium, and people respond to it in very different ways. Test with as diverse a range of subjects as you can, as early as you can, so you can spot things you need to change while they're still relatively easy to change.

Remember as well that VR developers make terrible test subjects. We use VR far, far more than other users and tend to be much more comfortable with VR interfaces and much less prone to simulator sickness than typical users. Test VR with users who are new to it as well as with users who are comfortable with the medium.

Test with as diverse a population as you can. VR embodies the user in ways that previous media don't, and this can matter to your users. Hands that look fine to you may feel alien to a user whose hands look different. Make sure your test population isn't limited to just people like you.

Look for opportunities to test as early in the process as you can make them happen. Even long before you've reached your MVP, test elements such as locomotion systems that may need design iteration. Put users into a gray-box environment and have them navigate through it, and watch what they do and where they get stuck. The more testing you do, the better your project will get, and the earlier you test, the easier it is to act on what you've learned.

Design is iterative

Lots of people assume that finished products somehow spring fully-formed from the minds of genius designers or developers. It doesn't work this way. Anything worth making takes iteration to get there.

Prepare yourself now for the reality that the first iteration of your design isn't going to be everything you wanted it to be, and that's the point. The purpose of the first draft of anything is to show you what's really important about the thing you're building and how it needs to fit together. Plan for this. Design is a process, and time and iteration, more than anything else, are the key elements of that process.

This is why we so strongly advise designing on paper first and testing prototypes in software as early as you can. Each time you give yourself something tangible to respond to, you're going to discover something about it and probably discover a way to make it better.