In the last few years, many IT environments have been facing profound changes in how hardware infrastructure is designed and how software is being managed. This is thanks to what is popularly called The Cloud.
A cloud provides users with computation power, storage, and software services. It does not require knowledge of the physical location or the specific hardware configuration of where the services are running. It runs on the the same logic that applies to a power grid providing energy to different houses, where a consumer does not need to know how the power is produced or how it is delivered to the house. Services provided by a cloud can be subdivided into three main service layers, as follows:
The Application layer is used when the user has access to a service without a need to know the physical infrastructure on which the application relies. The installation, configuration, and maintenance are completely managed by the provider itself, such as the popular Gmail service. The main advantages over legacy software are minor maintenance costs and no hardware costs at all.
The Platform layer is used when the user has access to a particular instance of an operating system or a software stack, with the desired amount of CPU power, memory, and storage available. Each instance is completely isolated from others running on the same hardware, and it is easy to scale out resources as they are needed. Take for example Google App Engine, where developers can deploy their own Python applications without the need to know how to configure a high-performance scalable stack. Also, they do not need to make long-term hardware provisioning plans to be prepared for future resource needs.
The Infrastructure layer is the most complex one. It involves different components that need to be orchestrated to be effective (for example, CPU power, memory allocation, storage, network, Virtual Memory (VM) instance management, and end-user access). It also allows complete outsourcing of IT infrastructure—the users pay for their effective usage, and even complex architectures can be managed without the need to have a bunch of physical systems. A typical example is Amazon EC2.
As there are situations where you cannot simply rely on external cloud providers (because of strict business policies), and you cannot afford proprietary virtualization solutions, cloud frameworks such as OpenNebula will be of great help. It will act as an effective open source toolkit, which can easily be adapted to work flawlessly even in heterogeneous environments, to build private, public, and hybrid IaaS.
OpenNebula started as a research project by Distributed Systems Architecture (DSA) Research Group (http://dsa-research.org) with a few of the current features. Based in Madrid in 2005, and later released in 2008 as completely open source, it is now actively developed by the community that has grown around it. Today, OpenNebula has achieved a very neat feature set thanks also to its great modularity, which eases the integration with other solutions.
The main difference between OpenNebula and other commercial cloud solutions is that its true open source blood guarantees users complete interoperability with every existing infrastructure component already available. Thus, it avoids the vendor lock-in using common open industrial standards, such as EC2 API and Open Cloud Computing Interface (OCCI).
Unlike other open source alternatives, OpenNebula does not embrace a particular hypervisor. It also does not have any specific infrastructure requirements, fitting well into any pre-existing environment, storage, network, or user-management policies.
The plugin model on which OpenNebula is implemented, gave system integrators the ability to customize every aspect including virtualization, storage, information, authentication, authorization, and remote cloud services. Every action is managed by a bash script that can easily be modified or plugged with some other custom script or software written in any language and supported by your operating system.
The following diagram will help you to understand which components are involved in OpenNebula and also the level at which they operate (lower ones interact directly with the host's resources, higher ones interact with user interfaces).
On the lowest level, there are drivers that directly talk to the underlying software OS components. They are as follows:
Transfer drivers: These are used to manage the disk images on the current storage system—a shared one, such as Network File System (NFS) or Internet Small Computer System Interface (iSCSI), or on a non-shared one such as a simple copy over Secure Shell (SSH).
Virtual Machine drivers: These are hypervisor-specific and they are used for managing the virtual machine instances on the current hosts.
Information drivers: These are used to retrieve the current status of virtual machine instances and hosts. They are hypervisor-specific, too—they are copied and remotely executed in every physical host through SSH.
All the monitoring information collected from physical hosts and VM instances, along with the configurations of every VM, the available disk images (images catalog) and the virtual networks, is stored in a simple SQLite database or a replicated MySQL database. It can easily be retrieved or altered by custom scripts or software, depending on your infrastructure needs.
The user can take advantage of the native OpenNebula cloud API, available as Java, Ruby, and XML-RCP API. It gives access to all the functions available and permits an easy integration of custom procedures before or after the standard one.
For performance reasons, the core of OpenNebula is written in highly optimized C++ code, giving good scalability. A good example of its robustness is the CERN infrastructure prototype, initially managing 480 server hosts; in spring 2010, they reached an impressive figure of 16,000 virtual instances.
A hook system is implemented to give users the ability to execute custom scripts after a predefined set of events, providing an easy way to send e-mail notifications upon changes, or to provide a simple failover mechanism.
Security is also taken into serious consideration. Host communication takes place exclusively through secured connections protected with the SSH RSA keypairs and Secure Socket Layer (SSL). Each virtual network is isolated with a firewall, ebtables (http://ebtables.sourceforge.net/). It works at the Address Resolution Protocol (ARP) level, also known as the data link layer in the OSI stack.
OpenNebula has a dedicated Quality Assurance (QA) team that deals with both unit and functional tests with a wide range of scenarios. This greatly reduces the number of bugs, or at least enables a greater user awareness of such bugs. There is also in use a continuous integration system, which carries on automatically building and testing every change-set pushed by OpenNebula developers, available at (http://hudson.opennebula.org/).
The OpenNebula project aims high, as its objectives are to develop the most advanced, scalable, and adaptable software toolkit with quality and stability checks for every release. It also actively searches for community support and contributes to the open source ecosystem—every bug request is analyzed by the team, and every bug found in the underlying software components is forwarded back to the respective project owner.
These kinds of operations are essential for maintaining a high profile and for attracting new people, to use and participate in the development of the project. Many third-party projects related to OpenNebula are directly hosted on the main site, emphasizing the work of every participating user, even for small contributions. The initial contributions arrived from the DSA research group (http://dsa-research.org) at the Universidad Complutense de Madrid, which has received funding from the European Union's Seventh Framework Programme, for the adoption of cloud solutions for scientific grid computing
Since mid-2010, OpenNebula has had official commercial support from C12G Labs, after numerous requests for support; this gave a boost in vitality to the project.