When designing a new network, some of the following factors might be important to you:
- Simple, focused, yet non-blocking IP fabric
- Multistage parallel fabrics based on the Clos network concept
- Simple merchant silicon
- Distributed control plane with some centralized controls
- Wide multi-path (ECMP)
- Uniform chipset, bandwidth, and buffering
- 1:1 oversubscribed (non-blocking fabric)
- Minimizing the hardware necessary to carry east—west traffic
- Ability to support a large number of bare metal servers without adding an additional layer
- Limiting fabric to a five-stage Clos within the data center to minimize lookups and switching latency
- Support host attachment at 10 G, 25 G, 50 G, and 100G Ethernet
- Traffic management
In a modern network, one of the first decisions is whether you will use a centralized controller or not. If you use a centralized controller, you will be able to see and control the entire network from one location. If you do not use a centralized controller, you will need to either manage each system directly or via automation. There is a middle space where you can use some software-defined network pieces to manage parts of the network, such as an OpenFlow controller for the WAN or VMware NSX for your virtualized workloads.
Once you know what the general management goal is, the next decision is whether to use open, proprietary, or a combination of both open and proprietary networking equipment. Open networking equipment is a concept that has been around less than a decade and started when very large network operators decided that they wanted better control of the cost and features of the equipment in their networks. Google is a good example. In the following figure, you can see how Facebook used both their own hardware, 6-pack/Backpack, and legacy vendor hardware for their interoperability and performance testing:
Google wanted to build a high-speed backbone, but were not looking to pay the prices that the incumbent proprietary vendors such as Cisco and Juniper wanted. Google set a price per port (1 G/10 G/40 G) that they wanted to hit and designed equipment around that. Later, companies such as Facebook decided to go in the same direction and contracted with commodity manufacturers to build network switches that met their needs.
Proprietary vendors can offer the same level of performance or better using their massive teams of engineers to design and optimize hardware. This distinction even applies on the software side, where companies such as VMware and Cisco have created SDN tools such as NSX and ACI.
With the large amount of networking gear available, designing and building a modern network can appear to be a complex concept. Designing a modern network requires research and a good understanding of networking equipment. While complex, the task is not hard if you follow the guidelines listed in this section.
These are a few of the stages of planning that need to be followed before the modern network design is started:
- The first step is to understand the scope of the project (single site, multisite, multicontinent, and multiplanet).
- The second step is to determine whether the project is a green field (new) or brown field deployment (how many of the sites already exist and will/will not be upgraded?).
- The third step is to determine whether there will be any SDN, NGN, or open networking pieces.
- Finally, it is key that the equipment to be used is assembled and tested to determine whether the equipment meets the needs of the network.
The project scope is one of the most important pieces of information needed. The project scope can go from a single device in one location to hundreds or even thousands of devices across multiple continents. Understanding the project scope provides a guideline on which to base the network design and hardware/software needs.
If the network is being designed for internal use, then looking at other locations should give information about the best practices of the company the network is being designed for. If the network is being designed for an external company, then it is useful to ask for documentation, hardware lists, and even a tour of a current site so that the concept can be understood.
This is not meant to be an exhaustive list, but there are a few things that need to be understood when designing the network:
- Is the network all internal?
- Does the network have a DMZ?
- Does the network have multiple internet connections?
- Does the network have storage and compute separate or together?
- Does the network need to support iSCSI or other SAN protocols?
- Does the network use MPLS, SD-WAN, or other tunneling technologies?
- Does the network have multiple Points of Presence (POP), and how large is a POP?
- Does the network use containers? If so, does it have a container-specific network?
At the end of the book, you will find a generic check sheet to scope the project.
A greenfield network is a site where there is no networking equipment currently. For a greenfield deployment, there are a lot of options, but the needs of the network musts be clearly understood. In a perfect situation, the site would be completely malleable, allowing for power, cooling, and infrastructure to be built out to meet the needs of the design. Since a perfect situation is not always possible, taking inventory of the infrastructure is necessary before a design can be committed.
At the end of the book, you will find a generic check sheet that provides an overview of what should be understood about the site.
If you plan to use any next generation hardware, you will need to do some research and show due diligence. Since next generation hardware generally means equipment that has not been out for very long (or may not be out in the public), there will be little to no public information or testing of the equipment. You will want to schedule a Proof of Concept (PoC) to be done with the hardware you expect to use.
If you are using open next generation hardware, you may be able to use reference customers to understand what designs are being used and what features.
While PC-based network devices have been available since the 80s, they were generally used by small companies and networking enthusiasts who didn't or couldn't afford to buy a commercial-based solution. In the last few years, many drivers have brought PC-based networking devices back into the limelight, including: Ethernet as the last mile, better network interface cards, and Intel's focus on networking processing in its last few generation of chips.
Today, many vendors are producing PC-based network devices with advancements in packet handling within Intel's processors, allowing processor cores to be re-programmed into network processors, and allowing PC-based network devices to push tens or even hundreds of Gbps.
Some of the values of the NFV concept are speed, agility, and cost reduction. By centralizing designs around commodity server hardware, network operators can do the following:
- Do a single PoP/site design based on commodity compute hardware:
- Avoiding designs involving one-off installations of appliances that have different power, cooling, and space needs simplifies planning
- Utilize resources more effectively:
- Virtualization allows providers to allocate only the necessary resources needed by each feature/function
- Deploy network functions without having to send engineers to each site:
- Truck rolls are costly both from a time and money standpoint
- Achieve reductions in OpEX and CapEX
- Achieve reduction of system complexity
Traffic engineering and traffic shaping is the concept of detecting and prioritizing different types of network traffic. Once prioritized, different bandwidth allocations can be provided to the traffic. Prioritization can be strict or loose and as a set amount or a variable amount (percentage).
Traffic engineering can be done in a few different ways, including MPLS TE tunnels, Virtual Circuits (VCs), and Quality of Service (QoS).
There are many tools that we will discuss and utilize in this book, tools to monitor networks, tools to configure networks, and everything in between.
Monitoring the network is highly important and has been the basis for quite a few great open-source tools such as Nagios, Monit, Sensu, and Zabbix.
Nagios is one of the older and most mature open source monitoring tools, providing a core infrastructure and a set of plugins for different devices:
The generic Nagios display shows alerts and details about the network status.
The following tools will help you in network configuration.
While we are working with the network, it will be useful to keep logs of changes in configurations. Really Awesome New Cisco confIg Differ (RANCID) is a free tool that can log in to many different types of systems, not just Cisco (though it started out that way).
Note
You can find more about RANCID at http://www.shrubbery.net/rancid/.
Postman is a program that works with APIs. It is available at http://www.getpostman.com. We will utilize Postman when dealing with REST-based APIs:
