Ethernet Networking: The Proliferation of Data Rates
While telecom networks progress beyond 100G, local area and data center networks have more data rate choices than ever before.
The coming of age of Ethernet has brought with it many different networking applications. Carrier Ethernet is embarking on wider adoption along with Ethernet over optical transport networks (OTNs). In the mean time, local area and data center networks now have many choices for data rate progression in the Ethernet paradigm.
Ethernet Data Rates
The following table shows a summary of current open projects that the IEEE is working on that may impact technologies used in both data centers and the overall LAN.
The IEEE P802.3bz and “cb” Task Forces are working on 2.5G and 5G variants to enable higher bandwidth to wireless access points (WAPs). With WiFi connection data rates increasing beyond Gigabit data rates, WAP connections back to the wired network need to be more than just the two Gigabits (two 1G ports) that are typical of these types of products; hence, the 2.5G and 5G standards development effort that is in process in the IEEE. Even though this started as a WAP access business case, these data rates are applicable to smaller data centers as well – a much simpler and cost-effective upgrade from 1G than 10G or 40G would be. Publication of the “bz” standard is currently slated for September 2016.
The IEEE P802.3by and IEEE P802.3bq Task Forces are working on 25G variants that are targeted for server connections specific to mega data centers. Eventually, 25G promises to be the next mass-market server upgrade from 10G, even for smaller data centers. Twinax copper and short-reach MMF variants are included in the “by” standard, while 25GBASE-T (twisted pair) was added to the existing 40GBASE-T “bq” project. The “bq” standard was originally slated for completion by March 2016, but with the late addition of 25G, it has slipped to June 2016.
The IEEE P802.3bs is responsible for the 400G standards development. Publication of this standard is scheduled for December 2018.
Three study groups, which may turn into task forces in the near future, include 25G SMF, 50G single-lane, and next-generation 100G/200G.
Data Centers
Mega data centers, like Amazon, Facebook, Google, and Microsoft, are already starting to adopt 25G at the server even before the standards are complete. While Microsoft started the conversation about 25G, there are many proponents for it now. It just makes sense in the grand scheme of data-center networking and perhaps, eventually, for the entire local area network.
There are three data center use cases for 25G connections that were outlined in the original IEEE call-for-interest document as shown in these three configurations:
- Direct-connect – This is the same topology as was used for 10G: 48 servers per ToR switch with 3:1 oversubscription and 100G uplinks, non-blocking with 400G. It would use an SFP28 direct-attach copper (DAC) cable for 25G connections.
- 4x25G breakout – This is the same topology as 4x10G: permits non-blocking 25G mesh; 100G ports used as 4x25G with QSFP28-to-four SFP28 breakout cables; uses the same network interface card in the server as 25G direct connect.
- High-density 25G – This allows for increased switch port density with 64 servers in non-blocking architecture and 96 servers in a 3:1 oversubscription or 24-port 400G ToR switch with 192 servers in non-blocking architecture.
In addition to these, with 2.5G and 5G data rates being developed, there could be many more use cases for smaller data centers. Envision an upgrade path from a 1G server to 2.5G with either 5G or 25G uplinks – or 5G server connections with 25G uplinks and 100G in the core. These may be much more cost-effective solutions for smaller data centers.
In the end, because each data center has its own set of networking requirements, we expect to see many more variations in the marketplace.
