Open Compute Delivers Flexibility and Empowerment
to Fidelity Investments
Fidelity Investments reconfigured its data centers to better fit its business needs, engaging its engineering team in redesigning a revolutionary new rack, and reducing energy consumption by 20%.
At a glance
Industry: Financial Services
Data centers: 4
Physical servers: 11,000
Virtual servers: 22,000
IT staff: 10,000+
Founded in 1946, Fidelity Investments is a multinational financial services corporation headquartered in Boston, Massachusetts. It is one of the largest mutual fund and financial services groups in the world.
Fidelity’s Enterprise Infrastructure team is responsible for the company’s data centers, networks, technology platforms and enterprise software, providing the underlying technology that enables the company’s IT developers to deliver timely, innovative solutions and services to its customers.
Challenge: Finding new efficiencies
In the wake of the 2008 financial crisis, Fidelity Investments was looking for opportunities to streamline its operations. The company’s Enterprise Infrastructure team found room for improvement in the way it consumed technology in its servers and data centers.
“Many enterprises have been sold on proprietary gear for years,” says Eric Wells, vice president of data center services for Fidelity Investments. “At Fidelity, we’d gotten used to our suppliers telling us what was available, what the next innovations were. That’s fine, but it drives a very high-cost operating model. We needed to do it differently going forward.”
Fidelity’s Enterprise Infrastructure team also wanted to transform that way its members worked. Instead of maintaining a closed shop, the team was looking to open up and engage with an external community of engineers as a way to keep up with the latest developments and innovate in-house.
The Enterprise Infrastructure team is focused on three key things: time-to-market, cost efficiency and system security/stability. It was looking to build an underlying architecture that supported all of these things, and would also help reduce operational cost and complexity.
Solution: Reinventing the data center
Open Compute is an open-source community of engineers working to redesign servers, storage and data center hardware designs for maximum efficiency and scalability. Open Compute was born inside Facebook, when the company realized that it had to re-invent the data center to maintain its exponential growth rate. Facebook designed and built a new kind of data center from the ground up that is 38% more efficient and 24% less expensive to run than a traditional data center. The company then shared these technologies as the Open Compute Project in 2011, hoping to create an open-source movement in the data center space that would bring about the same kind of rapid innovation typical of open source software.
Today, OCP has 150 member companies and thousands of participants working on eight different projects.
For Fidelity, Open Compute came along at just the right time. Since joining Open Compute in 2011, the company’s hardware engineers have participated in a number of initiatives, held a variety of positions and contributed several key specs back to the community. “We’re excited and proud to be part of a community that complements our own innovation in data center design, cloud services, social media and mobile devices,” says Keith Shinn, senior vice president, enterprise infrastructure. The company is currently migrating much of its distributed IT footprint to a private cloud platform based on Open Compute—a project that wasn’t possible before Fidelity designed the Open Bridge Rack.
Open Bridge Rack
The Open Bridge Rack is a convertible datacenter rack that can physically adjust to hold any size server up to 23 inches across. It’s a key link in helping enterprises make the switch from proprietary gear to Open Compute.
Traditionally, servers and racks are sold together. Standard servers are 19 inches wide while OCP servers (21 inches) and Telco devices (23 inches) are wider, so they require a larger rack. But even within 19-inch servers, there are slight size differences between different vendors, making their servers and racks incompatible. This is one way suppliers ensure future business.
The Open Bridge Rack changes all that, because it can be adjusted to accommodate a 19-inch server today, and a 21 or 23-inch server tomorrow, allowing Fidelity—or any enterprise—to future-proof its investment in racks.
The rack also improves on the standard power supply configuration. Known to be a big mark-up item for vendors, standard racks have 80 power supplies for 40 servers—many more than strictly necessary. What’s more, they can’t be replaced if they fail. The result is energy inefficiency, additional cost and wasted material. The Open Bridge Racks, like their predecessor the Open Compute Open Rack, have just 12 power supplies for 40 servers. They’re easy to replace if they fail, and they can be reused when an enterprise refreshes its compute.
Not only does Fidelity save money by not having to buy new racks, but Open Bridge Racks are also easier to maintain and more efficient. “This rack bridges the gap between today’s EIA racks/hardware and the next generation Open Compute racks/hardware,” says Brian Obernesser, vice president data center architecture. “The rack itself is convertible between EIA and OCP standard configurations. Its flexibility enables consumers to convert EIA racks either free or ganged in place to the OCP configuration as they begin to adopt OCP hardware. It also allows integrators or colocation facilities to standardize on a single rack that can respond and adapt to ever-changing customer requirements.”
Fidelity designed (patent pending) and donated it to the Open Compute Foundation, making it available to different manufacturers. The company has standardized on it, using it for its OCP deployments and everything else.
Before designing the Open Bridge Rack, Fidelity partnered with Goldman Sachs and a number of other financial firms to create an Open Compute server that would fit into a standard 19-inch rack. That work resulted in two new specs: the AMD Open 3.0 modular server and Decathlete, for Xeon/Intel.
Members of the Enterprise Infrastructure team have also partnered with other Open Compute participants to start the Compliance and Interoperability Project, which is focused on certification of Open Compute hardware and software. Today Fidelity engineers are working with Open InfraShare, a group focused on how to incorporate open software and hardware into existing architectures. The group meets to share designs and discuss requirements specific to enterprises. The aim is to draw attention and lend credibility to open source solutions.
Throughout the experience, the community itself has been one of the biggest benefits of participating in Open Compute for Fidelity. Bob Thurston, director of integrated engineering has found the community to be a remarkable asset. “The relationships we’ve built through our OCP involvement, especially with peers from other enterprise and hyper-scale data centers, have been invaluable in helping us look at things differently.”
Today, Fidelity is using Open Compute gear to transform its infrastructure. The company is staring to target go-forward platforms, like its private cloud. That’s an environment that allows developers to rapidly build and test critical new apps and features for Fidelity customers—and it’s all based on Open Compute.
Open hardware works for Fidelity because it’s more cost-effective, and also allows the engineers to design for different types of workloads, whether that’s a private cloud or big data analytics. Michael Poulter, senior vice president architecture says, “Open Compute gives us more flexibility to use non-proprietary components in our systems, so that we’re not locked in to any single vendor, and we can more quickly evolve what we’re working on. We want more flexibility, and we think this is a desire we share with a lot of other companies.”
Results: Influence, serviceability, cost savings and sustainability
Despite successive years of double-digit IT growth, Fidelity has achieved a 20% energy reduction across its North American data centers and is ahead of projected optimization targets in data center infrastructure costs. “We have totally changed the way we build data centers in the spirit of Open Compute,” says Joe Higgins, corporate sustainability officer and vice president of engineering. “Open communities are critical in achieving the highest levels of optimization for enterprise data centers.”
Costs have gone down due to less power consumption and also because now Fidelity can use gear from several different vendors who are competing on price. Maintenance is easier as well, which also contributes cost savings. Fidelity’s participation in the Open Compute Project also helps influence the evolution of data center components, shaping the industry as a whole in a way that supports Fidelity’s business.
“Open Compute has helped transform our organization into an ‘enlightened consumer,’” says Alok Kapoor, executive vice president, enterprise infrastructure, “one that’s better educated, aware and self-sufficient. It has empowered our people and firm to take a much more engaged and active part in the technology with a real ROI that goes well beyond cost.”
All the improvements haven’t gone without notice. Fidelity’s leadership understands the benefits as well, embracing open infrastructure as a key element of the firm’s executive-sponsored “compute strategy” initiative. In addition Fidelity has established an open source center of excellence to forward the firm’s broader adoption of open source technologies.