Summary Bullets:
• What can we learn from global server powerhouse Inspur about customizing servers in order to optimize demanding AI workloads?
• Using Inspur’s joint design and manufacturing (JDM) business model as a guide, it seems current server hardware and software architectures are leaning heavily in the direction of extreme customization, scale and agility.
A long time ago (shortly before 475 BC), the Greek philosopher Heraclitus commented on the ephemeral nature of human perception and experience, noting that “No man ever steps in the same river twice.” To be sure, this is a fun notion to entertain while standing in line for coffee for the Nth time on the Nth commute to the same old office. But for hyperscale hardware manufacturer Inspur, it’s much more than a philosophical distraction.
For Inspur, a company with aspirations to become the world’s largest provider of servers by 2021, this idea permeates and drives its entire approach to the market and is made manifest in the joint JDM business model. Basically, JDM allows Inspur to treat every customer engagement as a unique, collaborative endeavor, building bespoke solutions tailored to specific requirements.
Obviously, this idea is not new. Peterbilt Trucks was talking about rebuilding its assembly lines back in the early nineties to support customer-defined, one-off configurations. That never truly panned out for Peterbilt (and many “at scale” manufacturers), explaining why the most common customization available today to the masses is a choice of several colors and custom engraving. The problem revolves around one to one vs. one to many manufacturing. If every Apple customer requests a truly unique, one-off iPhone with a unique circuit board, memory configuration and let’s say chip type, Apple would have to charge even more than it does now, unbelievably.
But if Apple were to receive a request for 2,000 custom iPhones, well, that’s an entirely different story. And that’s the story of Inspur’s meteoric rise in the server market. Its AI server shipments in particular have grown four-fold over the past year because it can accommodate custom requests at scale. The company, after all, isn’t building custom white box servers for each “PC enthusiast.” It builds thousands upon thousands of servers for hyperscale, public cloud providers Facebook, Alibaba, and Tencent. These vendors work in concert with Inspur to design some highly specific hardware tailored to specific use cases, like image classification and content tagging, standing these up by the hundreds at a go.
The way this works is via a heavy investment in physical facilities — eight R&D data centers, six manufacturing centers, and 113 delivery facilities worldwide, to be exact. With this supporting infrastructure and JDM (also referred to as “Joint Destiny Model” by Inspur), the company has plied knowledge jointly gained to build out a wide array of customizable, purpose-built AI server solutions. It’s important to note that these aren’t simply custom configurations, but rather custom designs that make use of and preserve open source AI frameworks like TensorFlow and Caffe while accelerating performance via some decidedly proprietary hardware.
Case in point is Inspur’s TF2 computing acceleration, which compress TensorFlow models up to 1/8th the size of the original, transforming them to run on FPGA processors. The result is the preservation of the model’s original accuracy combined with the reduction of power consumption and improvement in hardware efficiency. That’s pretty compelling for big players like Facebook, which look to optimize data center operational efficiency and AI workload performance.
Certainly that’s all well and good for customers adding hundreds of servers per day, but what about the long tail opportunity? Is Inspur’s JDM philosophy applicable more broadly across the enterprise marketplace? Absolutely. The company intends to take what it’s learned from hyperscale service providers about breaking away from traditional server architectures and making that available to a wider audience. Whether that ends up being a selection of pre-packaged options “with engraving” or something more individualized remains to be seen.
But as I mentioned a few weeks back in looking at Comcast’s unique VNF endeavors, it is clear that current server hardware and software architectures are leaning heavily in the direction of extreme customization, scale, and agility. What that actually looks like to individual enterprise IT buyers will of course depend on when they dip their toes into this ever-evolving stream of opportunities.
IT Connection 