So we’ve established the meaning of hyperscale, examined the various types of data center and identified the different tiers of hyperscaler, but what is it that makes hyperscale, hyperscale? What are the specific characteristics that set hyperscale data centers apart?
Scalability
To meet rapidly growing demand, hyperscale data centers are designed with extreme scalability, apparent in the versatility of the physical connectivity itself. Enterprise data centers will traditionally fill a hall with 350 empty cabinets that they then fill with servers over time. Servers are installed one at a time. In hyperscale data centers, pre-configured server and switch cabinets are bought as a whole unit as opposed to individual servers – the unit of measure is a full rack. Scalability also applies to the ability of a hyperscale network to grow globally with new data centers being built at an ever-increasing rate to expand availability zones.
Technology
Hyperscalers drive technological advancement. The technology behind hyperscale data centers is a step above that of others. Advances made in network bandwidth, server chip speed and power efficiency mean the typical equipment technology lifetime is just 42 months. Hyperscale data centers will represent 53% of all installed data center servers by 2021. Enterprise data center halls must have plenty of cooling to keep servers running at the correct temperature. Servers in these halls are designed for general use by many customers in different situations. By contrast, hyperscalers control the complete thermal environment including hall layout, direction of airflow, server internal design and chip selection. They can go directly to chip manufacturers and ask for a chip that works at a higher temperature or buy a chip designer outright! More and more frequently, hyperscalers are taking control and disaggregating supply chains.
Maintenance
For small and medium business enterprises, if a server fails, the impact to the business is often immediate. In the event of a faulty server, or any other hardware, within a hyperscale data center, overlaid management software automatically transfers workloads to other available equipment, ensuring continuity and uptime. It is only when enough servers in one area need to be repaired or replaced that crash carts are brought in. Ideally, hyperscale networks should require as little maintenance as possible once operational, only being physically touched to repair or replace. Despite the large size of a hyperscale data center, on-site staffing tends to be very low. The average repair time of a server used to be an hour - it is now 2 minutes or less with an average ratio of 25,000 servers per support technician per shift.
Power Consumption
Power is the largest operational expense for most data centers and reducing power costs is often high on the agenda. With an average of 20-50 megawatts (MW) per data center and 10-25 kilowatts (KW) per rack, power consumption in hyperscale data centers is on another level. Globally, data centers consume around 3% of the total energy generated worldwide with hyperscale data centers accounting for 20% of the world's data center electricity usage, growing to 50% by 2020. One of the world’s largest data centers, Microsoft’s 700,000 sq. ft. data center in Chicago, Illinois has the capacity to consume 198MW of power. To put that into context, in the US, 1MW of electricity would be enough to power an average of 750 homes, theoretically, this data center could pull an amount of power similar to that of 150,000 homes. In some cases, hyperscale operators are purchasing every single kilowatt of power generated from new renewable energy sources to support the power needs of their data centers. Hyperscale data centers, while using lots of power, are far more efficient than if businesses scaled their own networks and servers on-site. The Lawrence Berkeley National Laboratory estimates that moving 80 percent of servers in the U.S. over to optimized hyperscale facilities would cut energy usage by roughly 25%. While many would think that large hyperscale data centers are an excessive drain on electricity, the opposite is in fact true.
Power Consumption
Traditional enterprise networks are based on a three-tier architecture that is built on a hierarchy of a core, aggregation, and access. As data flows escalate and applications become increasingly time-sensitive, this three-tier architecture is becoming more and more problematic as data flows vary depending on where the source and destination devices reside in a network. In some cases, data may only have to flow up one level to the distribution layer to reach a destination. In others, the flow may have to reach all the way up to the core, then back down the stack, making the data flow inefficient. Advancements in connectivity combined with software automation are causing network architects to reimagine how efficiency in a network can be improved end-to-end.
Architecture
Traditional enterprise networks are based on a three-tier architecture that is built on a hierarchy of a core, aggregation, and access. As data flows escalate and applications become increasingly time-sensitive, this three-tier architecture is becoming more and more problematic as data flows vary depending on where the source and destination devices reside in a network. In some cases, data may only have to flow up one level to the distribution layer to reach a destination. In others, the flow may have to reach all the way up to the core, then back down the stack, making the data flow inefficient. Advancements in connectivity combined with software automation are causing network architects to reimagine how efficiency in a network can be improved end-to-end.
If we cast our minds back a couple of decades, the strength of a network relied more on computers than the network itself. Conducting a simple search on Yahoo! in the 90s would take what would now be considered an unthinkably long time due to limited bandwidth. Nowadays, tens of thousands of searches are conducted every second across all manner of text, images, videos, shopping, and real-time navigational maps whilst promoted search items and adverts are simultaneously displayed in line with your search request, all within a fraction of a second.
All the above is made possible by leaf-spine architecture. The primary benefit of a leaf-spine architecture is that it allows connection across very large numbers of servers. Data flows within a leaf-spine fabric take the same number of hops on the network regardless of the source and destination meaning that all devices are equally well-connected with the same time delay or latency for all possible paths.
90% of all hyperscale traffic happens within the data center.
This is known as East-West traffic because it travels across the data center as opposed to in and out (North–South traffic). North-South traffic is commonly a user request coming into the data center, whereas East-West traffic is the amalgamation of information and internal processing behind the request. Think about the processes involved in building a Facebook page.
Leaf-spine architecture works particularly well in hyperscale facilities when using distributed server architectures where resources for a specific application or service are scattered across multiple servers. This topology minimizes latency, jitter, and workload bottlenecks and does not block any traffic, even when all servers are communicating that full bandwidth.
Read the full What is Hyperscale? e-book for more! Learn about the evolution of hyperscale data centers, the challenges hyperscale networks face, and what the future looks like as demand for data and connectivity continues to grow.
