High Performance DEFLATE Compression Intel® Architecture: Paper
There is a critical need for lossless data compression in enterprise storage and applications such as databases and web servers, which process huge amounts of data. DEFLATE is a widely used standard to perform lossless compression and forms the basis of utilities such as gzip and libraries such as Zlib. In these applications, compression imposes a large computational burden on the servers, and they could benefit from a highly optimized implementation. This paper describes the performance characteristics of fast prototype implementations of DEFLATE compression on Intel® processors based on the 32nm microarchitecture. As the performance of compression is data dependent, we report the performance on various industry standard corpora data sets.
This paper describes the performance characteristics of fast prototype implementations of DEFLATE compression. In terms of throughput, we are able to perform DEFLATE compression at the aggregate rate of ~2.7 Gigabits/sec on the Calgary Corpus data-set, on a single core of an Intel® Core™ i5 650 processor.
Our fastest DEFLATE compression implementation is ~4.5 times as fast as the fastest mode of the best open source version of Zlib compression on the Intel Core i5 650 processor. To achieve such large performance gains, we sacrifice a small amount of compressibility compared to Zlib-1.
Read the full High Performance DEFLATE Compression Intel® Architecture White Paper.