Challenges in Developing I/O Systems for Today’s Telecom and Datacom Needs
By Jim David, FCI

Today we crave real-time, information-rich news and data within very short periods of time. Whether it is transmission of video on sites like YouTube, communication through social networks like Facebook and MySpace, downloading digital MP3 files or video files for instant entertainment, or receiving real-time Twitter updates, these Internet activities are now part of everyday life, even though they weren’t around just five years ago.

This seemingly insatiable need for more data and commensurate network bandwidth will continue unabated for the foreseeable future, driven by emerging video-rich applications like IPTV, peer-to-peer, video-on-demand, and Internet video to TVs and PCs. The demand for video content is expected to grow at a CAGR of 52 percent from 2008 to 2011.

Given this projected growth in demand, leading companies, industry organizations, and trade associations have been diligently working to ensure specifications and products are ready to address these anticipated capacity needs. A number of industry specifications have been developed to assure commonality, compatibility, and networking functionality of hardware connections, signaling, and software communications. These industry standards include those for data center interconnect technologies such as Infiniband, Fibre Channel, Ethernet, Serial Attach SCSI (SAS), and Serial ATA (SATA). Meanwhile, organizations like the Infiniband Trade Association and various IEEE 802.3 sub-committees are in the process of finalizing specifications that address the industry’s desire for 40 Gb/s and 100 Gb/s bandwidth-capable systems and I/O links. Further reinforcing these expected trends are published developmental roadmaps (below) that point to link bandwidths that extend well beyond100 Gb/s, even heading into the 400 Gb/s bandwidth range.

Satisfying these bandwidth needs isn’t always easy, as service providers must balance customer demands for timely, reliable, and cost-effective delivery of services against equipment and energy costs, equipment utilization, and overall data center productivity and efficiency. It is clear that these challenges will continue to affect virtually all data center and communication equipment platforms, including switches, routers, servers, and storage systems.

The growth and proliferation in the number of 10 Gb/s server connections over the next five to six years will be followed by a similar growth cycle for the 40 Gb/s connections beginning around 2015. To connect these servers to the network, industry analysts expect 10G Ethernet switch ports to experience 143 percent CAGR from 2008 to 2012, and high-end router demand for 10G ports to see a CAGR of 31 percent during the same time period.

Enabling this industry progression are new and evolving cable link and I/O interface specifications, such as SFP+, QSFP+, CXP, mini-SAS HD, and CFP, which will provide the high-speed external and internal cable links needed to handle this explosive growth.

So where does this product evolution and technical advancement leave suppliers of today’s copper-based I/O link solutions? The short answer is that making a cable assembly for these systems isn’t as simple as it used to be. There are a number of challenges that any viable cable assembly supplier must address in order to assure a high quality, compliant interconnect link is supplied to their customers.

Equipment and System Design
Equipment and system designers are challenged in numerous ways as they attempt to adapt to the rapidly growing bandwidth demands. Technologies such as multi-core processors, virtualization, consolidation, rising host bus speeds, and memory performance have certainly helped expand the available capability a designer can integrate into a system design, but these technologies strain bandwidth capacity, power consumption, and power and thermal management. The migration to increased signal speeds while preserving adequate signal integrity makes the continued use of commonly used, cost-effective printed circuit board materials, like FR-4 and cables with commonly employed insulations and manufacturing processes, a difficult task.

As an example of the power management challenge, let’s consider a Google search. Given today’s chip technologies and capabilities, it has been estimated that a single Google search requires three watts of power to complete the inquiry. But for proper cooling and dissipation of the heat generated by the search, an additional three watts of power is required. These power needs are driving designers to employ “green” techniques, such as port power management functionality that directs the port to automatically go into a “sleep” mode when not being utilized. The intent is to reduce the power consumption with better power management. This is just one example of the multiple, and sometimes conflicting, considerations that system designers and users must balance in next-generation equipment designs.

Signal integrity at increased signal speeds and power consumption isn’t the only thing designers and users must consider. Other factors, like proper system heat dissipation and management, sufficient air flow, cable routing, and EMC/EMI shielding, port density, and cable assembly installation, removal, and attachment, also require careful design consideration.

Equipment manufacturers and users are looking for flexible, future-proofed interconnect systems that are easy to install, easy to maintain, and provide performance headroom to support future system upgrades. While the minimum requirement is to maintain existing system port density, the preference is to achieve an increase in the I/O port bandwidth density along the edge of a line card to provide increased capacity. The capability to freely designate or configure any available system port with either copper or fiber-based cabling, as dictated by the specific installation environment with minimal issues and cost implications, is desirable.

All of these needs facillitated a closer working relationship between system designers and high-speed I/O system suppliers. In the past, there wasn’t a lot of collaboration between these two disciplines, but with the advent of higher signaling speeds, it became apparent that a higher level working relationship between system designers and the I/O system designer was necessary in order to meet all of the goals outlined above. It also requires that both parties have a deeper appreciation and understanding of the specific functional and design capabilities each party can bring to the overall system design, without adding excessive costs and overhead. This new dynamic is best illustrated by the collaboration within and among industry standard organizations, committees, and sub-committees, as well as industry ad hoc groups, such as the Small Form Factor (SFF) committee, where a great deal of discussion takes place. This interaction has become an absolute must for equipment suppliers to ultimately give their customers what they are asking for. The I/O system supplier must give the equipment designer as much flexibility and functionality as possible.

I/O System Solutions
The good news is that I/O systems have been developed to address many of the requirements. XFP and SFP (right) copper- and fiber optic-based I/O systems have been on the market for some time now. They have been instrumental in bringing I/O port bandwidths to the 5 to 6+ Gb/s per channel capability level. These systems are also more compact to minimize the linear board “shoreline” required. The SFP system significantly reduced the module outlines and shoreline required from earlier GBIC and XENPAK systems.

One need that the SFP system failed to address was 10 Gb/s channel capability, which is being demanded today. This need led to the development of the SFP+ system, which can support 10 Gb/s channel capability. While the SFP and SFP+ systems share the same board space, connector, and cages, only the SFP+ systems support 10 Gb/s channel bandwidth.

The I/O product development progression is continuing with the recent developments of industry standard interfaces such as QSFP+, mini-SAS/SATA, mini-SAS HD, CXP, and CFP.

The QSFP+ system (right) has been developed to address the need for an I/O system capable of supporting a 40 Gb/s total bandwidth in each port. Similarly the CXP system is being developed to support systems that are looking for 100 to 120 Gb/s total bandwidth per port. Both of these systems are being developed, offered and aligned with a number of interconnect technologies such as Infiniband and Ethernet, and are being adopted in multiple industry specifications. Both systems offer connector and cage products that support either a passive copper-based cable solution generally used for relatively short length cables (five to seven meters or longer depending on the acceptance criteria); an actively equalized copper-based cable solution for longer lengths (up to 15 meters or longer depending on the acceptance criteria); a plug-in optical transceiver module with an optical-based I/O connector on the back side of the module; or an active optical cable assembly (AOC) with the optical fiber terminated inside the cable backshell. This architectural approach gives the system installer and system user the flexibility to define and change the port configuration and capabilities as required.

The CFP system, which was announced in early 2009 as a multi-source agreement (MSA), takes a similar approach to the CXP interface in that it has capability to support 100 Gb/s bandwidth. The CFP system, as it’s currently configured, has the transceiver embedded in the module and utilizes a standardized two-piece connector interface between the module and equipment port. The I/O side of the module allows for multiple port configuration options (SFP+, QSFP+, CXP, or optical simplex or multi-fiber interface combinations) that can be customized depending on the customer-desired I/O interface and data distribution. In contrast to the more compact CXP module, the larger CFP transceiver module is optimized for longer reach, single-mode fiber applications.

Signal transmission speeds and bandwidth demand are being driven by the video-rich and social networking applications with significant growth forecast for the future. There is a significantly higher level of collaboration between equipment designers, raw cable suppliers, component suppliers, and high-speed I/O system suppliers that is necessary to properly address these market demands, and it will be reflected in products that offer higher port density and flexibility in port configuration. In light of the added functionality and increased signal speeds, manufacturing these cable assemblies is more challenging than ever before. The quality considerations for raw cable, PCB design, wire management, wire stripping, wire termination, and wire strain relief all must be carefully addressed and properly controlled as development of these systems and components evolves.

Jim David is global product manager for high-speed cable assemblies and connectors for FCI Electronics. He has been with FCI Electronics/Berg Electronics/DuPont Electronics in product engineering, field application engineering, field sales, engineering management, and business management for 29 years. Jim has a BSME from the University of Massachusetts-Lowell and an MBA from Penn State University.