If you have read through the chapter up to this point then you will already have an understanding of what ESXi is. With the latest version, ESXi 6.0, there are a few scalability and a number of security enhancements.

Each ESXi 6.0 host can now support up to 480 logical CPUs, 12 terabytes of memory, and 1024 virtual machines. Let's compare this with some of the earlier versions of the ESXi hypervisor:

There are a number of security enhancements with the new version, and these include:

In the previous sections of this chapter, we learnt about ESXi and virtual machines. In a large infrastructure, these entities need to be centrally managed. The central management is achieved using VMware vCenter Server. It comes in the form of a Windows installable program and also as a Linux-based virtual appliance. Without the vCenter server, you cannot cluster the ESXi hosts, which is essential for the enablement of the VMware features such as vSphere HA, vSphere DRS, and vSphere DPM. Also, every other management solution that is out there will need to interface with the vCenter Server by means of a plugin.

Currently, there are two types of client available from VMware that can be used to connect and manage your vSphere infrastructure. One of them is a desktop client which can only be installed on a Windows machine. It can be used to connect directly to an ESXi host or a vCenter Server. This form of the client will reach its end of life very soon as VMware will transition every GUI action to be performed through their web client. The desktop client is C# based and it is currently available only for backward compatibility and to support a few plugins which haven't been completely transitioned to the vSphere Web Client. Unlike the desktop client, the vSphere Web Client is a server component installed and configured on a machine and the users willing to connect will rely on their web browsers to connect the web client server to access the vCenter GUI. The most critical difference is that the vSphere Web Client cannot be used to connect to an ESXi host directly. You need to rely on the vSphere C# based desktop client for that, and it is one of many reasons why the desktop client is still around.

VMware vRealize Orchestrator, also known as the vCenter Orchestrator, is a GUI-based process automation tool that is installed along with your vCenter Server. It is primarily used to create workflows to automate repeatable IT processes. It has a plug-in framework which can be used by other solutions to perform actions. The vCenter Server, vRealize Automation, VROPS, VCM, and tools that can leverage the Orchestrator to perform actions.

It becomes necessary to upgrade or patch your vSphere environment to maintain a reliable platform for your virtual machines. Although the ESXi hosts can be patched or upgraded manually it becomes a very tedious process and would require many man-hours to perform the activity in a large environment. This is where VMware vSphere Update Manager (VUM) comes in handy. It provides a mechanism to patch and upgrade the ESXi hosts with reduced manual intervention. It can also be used to upgrade or patch third-party products such as the Cisco Nexus 1,000V.

VMware Power CLI is a set of modules or snap-ins which include cmdlets based on Microsoft Power Shell. It is used as a scripting tool for managing or automating most of the vSphere actions. The latest version, 6.0, has more than 400 cmdlets for both vSphere and vCloud environments.

VMware vRealize Operations Manager (VROPS) is an infrastructure monitoring solution. It does provide greater insights into the performance, capacity, and health characteristics of your vSphere environment. It can present information in the form of dashboards, it can generate smart alerts, and can perform predictive analysis. It comes packaged with a vCenter plugin, but you can install several other third-party plugins to let VROPS gather information from other components as well. For instance, there are adapters for EMC Symmetrix, VNX storage systems, and many more.

vSphere Data Protection (VDP) is an EMC Avamar-based backup and recovery solution from VMware Inc. It is available in the form, a Linux virtual appliance and can support up to 8 terabytes of de-duplicated backup data per appliance and up to 20 such virtual appliances can be associated with a single vCenter Server.

VMware vShield Endpoint is a security framework from VMware which enables hosting the load of performing antivirus or antimalware analysis on virtual machines onto a dedicated appliance. The framework utilizes a thin-agent included with VMware Tools and a heuristics engine running on a separate appliance provided by the security vendor. Every ESXi host will run such an appliance for the virtual machines running on it.

VMware vMotion will let you migrate the live state of a powered-on virtual machine from one ESXi host to another without affecting any of the applications or its services running on it. Whilst Storage vMotion can relocate all the files backing the virtual machine from one data store to another and also migrate its live state from one host to another, or it can migrate only the files backing the virtual machine and leave the live state on the same host.

VMware vSphere High Availability (HA) is a functionality that is used to configure a cluster of ESXi hosts to respond to an unplanned downtime event and ensure the availability of the virtual machines that were running on them, with very minimal downtime possible. It has the ability to monitor the guest operating systems and the applications running inside of a virtual machine and then decide to restart the affected virtual machine in an effort to reduce the downtime of a service due to an affected guest operating system hosting the service or a nonresponsive application corresponding to the service. It is important to understand that even though HA is configured on a cluster of ESXi hosts, it only provides high availability for the virtual machines and not for the hosts. It cannot start up or restart an affected ESXi host.

VMware vSphere Fault Tolerance (FT) is used to enable continuous availability of a virtual machine with zero downtime, maintaining an identical copy of the virtual machine in lock-step mode. We will learn more about this, in Chapter 8, Virtual Machine Concepts and Management. Unlike vSphere HA, FT is enabled on individual virtual machines. Although, FT had imposed a lot of restrictions on the scalability and the actions that can be performed on an FT-enabled virtual machine with the earlier versions of vSphere, with vSphere 6, it has been vastly improved and most of the restrictions don't exist anymore:

VMware vSphere Distributed Resource Scheduler (DRS) is a series of algorithms devised to manage an aggregated pool of computing resources and distribute virtual machines among the ESXi hosts in a cluster in an effort to reduce any resource imbalance in the cluster. It also helps in reducing the power consumption in the data center using DRS's power management feature known as Distributed Power Management (DPM). VMware DPM can help reduce the energy consumption of a data center by vacating VMs from an underutilized host and putting that host in a power-off state.

Unlike DRS, which manages the compute resources, Storage DRS manages the storage resources. It is a mechanism to balance space utilization and the I/O load on data stores in a data store cluster by migrating (using Storage vMotion) the VMs. Storage DRS can only be enabled on a data store cluster. It also influences the initial placement of the VMs on the data stores, by generating placement recommendations. vSphere Storage DRS requires Enterprise Plus licensing.

VMware vSphere Storage I/O Control (SIOC) is used to throttle the VMkernel device queue depth of a LUN, based on the shares set on the virtual machine disks contending for I/O bandwidth. SIOC can only be enabled on data stores (FC/ISCSI/NFS) and not on RDMs. It cannot be enabled on data stores with multiple extents. In this book, you will learn how to enable SIOC on a data store.

VMware vSphere Network I/O Control (NIOC) enables use and creation of Network Resource Pools. Much like with the compute resources of an ESXi cluster, you can use resource pools on a vSphere Distributed Switch (VDS) to configure Shares, Bandwidth Limitation, and Quality of Service (QoS) values. Such resource pools are referred to as Network Resource Pools (NRP). There are both System Defined and User Defined NRPs.

VMware vSphere Standard Switch (vSwitch) is a software switching construct (in other words, a software-based network switch) local to each ESXi host. It provides a network infrastructure for the virtual machines running on that host. Unlike a physical switch, a vSphere Standard Switch is not a managed switch. It doesn't learn MAC addresses to build a MAC table, but it does know the MAC addresses of the virtual machine vNICs connected to it.

Unlike the standard switch, the vSphere Distributed Switch (VDS) spans across multiple ESXi hosts. It is not locally managed at the ESXi host. It requires VMware vCenter Server for configuration and management, though VDS is only available with the vSphere Enterprise Plus license. It has a control plane which resides at the vCenter Server and a data plane which resides on an ESXi host that is connected to the VDS.

VMware vSphere Virtual Symmetric Multiprocessing (SMP) enables a virtual machine to use more than one logical processor simultaneously.

Virtual Machine File System (VMFS) is VMware's proprietary cluster filesystem that can be used to format block storage units presented to an ESXi host. VMFS will let more than one host have simultaneous read/write access to the volume. To make sure that a virtual machine or its files are not simultaneously accessed by more than one ESXi host, VMFS uses an on-disk locking mechanism called distributed locking. The current version of VMFS is 5.

Virtual Volumes (VVols) is a newly introduced concept with vSphere 6.0. It is not intended to replace VMFS, but to take advantage of the hardware capabilities of the storage system. It requires a supported vSphere API for Storage Awareness (VASA) provider for its functioning. It is not a filesystem by any means. It is only a method to encapsulate files, backing a virtual machine into virtual volumes, and these are created automatically when you create or modify a virtual machine. ESXi does not have direct control over the VVols created, instead it interacts with a Protocol Endpoint, which again is provided by the storage vendor.

VMware vSphere Storage API is an application programming interface framework from VMware that enables the storage and backup software vendors to enable or enhance integration with vSphere. The vSphere Storage APIs-Data Protection (VADP) is a framework that enables backup vendors to create backup and recovery solutions that integrate with vSphere. The vSphere Storage APIs-Storage Awareness (VASA) enables storage vendors to create storage providers which become an interface for vCenter to gather storage characteristics for the LUNs presented to the ESXi hosts. The vSphere Storage APIs-Array Integration (VAAI) enables ESXi to offload certain storage operations to a supported storage array. For instance, the process of zeroing the blocks of an eager-zeroed thick VMDK during its creation can be offloaded to the array to speed up the process. The availability of these APIs is dependent of the type of license in use. So, when you are designing an environment for performance it is important to understand what APIs are available with which VMware license editions.

VMware Virtual SAN is a hyper-converged storage architecture that enables creating a shared storage platform using the local storage on the participating ESXi hosts. Since this ability is built into the hypervisor, there is no requirement to deploy appliances. All the management is from the vCenter Server. VSAN supports two types of configuration, an all-flash architecture and a hybrid architecture. In a hybrid-architecture, SSD and magnetic HDDs are mixed together to form the storage layer. The SSDs will be used for caching purposes to increase performance. In an all-flash architecture, both caching and storage are done on SDDs, hence delivering a very high performance storage platform.

vSphere Storage Thin Provisioning enables the creation of Virtual Machine Disks (VMDKs) that consume the space required for the data in it and not the actual size of the VMDK. Meaning, if the VMDK is of the size 50GB, but the data in it is only 15 GB, then only 15 GB worth of storage space is consumed from the data store. It is beneficial because not every disk created is fully consumed leading to wastage of storage space. Thin provisioning helps in over-allocation, but requires better reporting to manage the consumption of the storage resources.

Flash storage (Solid State Disks-SSDs) disks offer higher I/O performance when comparted to the magnetic disks. Unfortunately, SSDs are far more expensive than the regular hard disks. With the vSphere Flash Read Cache mechanism, you can configure the available local SDD storage to act as a cache for virtual machines to use. VMkernel handles the assignment and allocation of the cache.

With vSphere 6, VMware introduced a new feature called the Content Library. It is used to store templates and other files that can be shared across infrastructures, and it is backed by a data store. They can be local to a vCenter, published to be subscribed, or subscribed from a published library.

vSphere Auto Deploy is a web server component, which once configured can be used to quickly provision a large number of the ESXi hosts without the need to use the ESXi installation image to perform an installation on the physical machine. It can also be used to perform the upgrade or patching of the ESXi hosts without the need for VUM.

A VMware vSphere Host Profile is a configuration template that is created from existing ESXi hosts. It could only be created using the vCenter GUI. Host Profiles can be attached to other ESXi hosts managed by the vCenter and can be used to track configuration changes by monitoring compliance of the attached hosts, or it can even be used to apply configuration changes to a large number of hosts, greatly reducing the amount of manual work which would otherwise be required.

vSphere Replication is a replication engine that can be leveraged to configure replication on individual virtual machines. It can replicate a virtual machine and its disks from one location to another without the need to incorporate an expensive array-based replication. What it really does is provide a mechanism to replicate a virtual machine using the existing Ethernet infrastructure and recover them when there is a need. It directly integrates with the vSphere platform and is available with Standard, Enterprise, and Enterprise Plus editions. It is storage agnostic, which means that a virtual machine or its disk files can be replicated to a data store, regardless of it being a VMFS volume or an NFS mount. You can learn more about vSphere Replication from the book Disaster Recovery using VMware vSphere Replication and vCenter Site Recovery Manager, Abhilash GB, ISBN 9781782176442, Packt Publishing.