Microsoft splashes compression algorithm and hardware specs for Azure into open source

Microsoft kicked off the Open Compute Project Summit on Thursday with the news that it has put its cloud-compression algorithm into open source.

Microsoft is also contributing its hardware design cloud specifications, and Verilog source code for register transfer language (RTL) into the Open Compute Project (OCP).

RELATED: Facebook, Microsoft join forces to form Co-Packaged Optics Collaboration

Microsoft's compression algorithm, which is called Project Zipline, and the hardware implementations were developed to more efficiently address the management and storage of data at scale in a cloud. According to a blog post by Kushagra Vaid, general manager, Azure Hardware Infrastructure, Zipline's compression algorithm yields up to two-times the compression ratios versus a commonly used model.

"Enhancements like this can lead to direct customer benefits in the potential for cost savings, for instance, and indirectly, access to petabytes or exabytes of capacity in a cost-effective way could enable new scenarios for our customers," Vaid said in his blog. "Project Zipline is a cutting-edge compression technology optimized for a large variety of data sets, and our release of RTL allows hardware vendors to use the reference design to produce hardware chips to allow the highest compression, lowest cost, and lowest power out of the algorithm."

Project Zipline is now available in the OCP ecosystem. The OCP community will be able to take Zipline for trial runs, and, hopefully, contribute to the further development of the algorithm and hardware specifications. Microsoft also said the addition of Zipline could "open the doors for hardware innovation at the silicon level."

"Over time, we anticipate Project Zipline compression technology will make its way into several market segments and usage models such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT, and edge devices," according to Vaid.

According to Vaid's blog, Intel, AMD, Ampere, Arm, Marvell, and SiFive are currently developing CPUs that use Project Zipline. Networking vendors include Broadcom, Fungible and Mellanox Technologies while the storage vendor ecosystem is comprised of Eideticom, NGD Systems and Pure Storage.  

At last year's OCP Summit, Microsoft announced Project Denali, which is a specification for standardizing SSD firmware interfaces. Microsoft, which joined OCP in 2014 as a founding member, has also contributed Project Cerberus—for hardware security— and its Project Olympus data center design to the OCP over the years.