This bug was reported by Danilo Ramos of Eideticom, Inc. It has
lain in wait 13 years before being found! The bug was introduced
in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
option forces the use of fixed Huffman codes. For rare inputs with
a large number of distant matches, the pending buffer into which
the compressed data is written can overwrite the distance symbol
table which it overlays. That results in corrupted output due to
invalid distances, and can result in out-of-bound accesses,
crashing the application.
The fix here combines the distance buffer and literal/length
buffers into a single symbol buffer. Now three bytes of pending
buffer space are opened up for each literal or length/distance
pair consumed, instead of the previous two bytes. This assures
that the pending buffer cannot overwrite the symbol table, since
the maximum fixed code compressed length/distance is 31 bits, and
since there are four bytes of pending space for every three bytes
of symbol space.
This allows deflate to generate the same output when continuing after
a Z_SYNC_FLUSH vs. using deflateSetDictionary() after a Z_FULL_FLUSH
or a deflateReset(). It also slightly improves compression when
flushing by providing two more strings to possibly match at the start
of the new block.
Previously, the bit buffer would hold 1 to 16 bits after "all" of the
output is provided after a Z_BLOCK deflate() call. Now at most seven
bits remain in the output buffer after Z_BLOCK. flush_pending() now
flushes the bit buffer before copying out the byte buffer, in order
for it to really flush as much as possible.
Z_PARTIAL_FLUSH would sometimes emit two empty static blocks instead
of one in order to provide enough lookahead for inflate to be able
to decode what was last compressed. inflate no longer needs that
much lookahead, so this removes the possibility of emitting the
second empty static block. Z_PARTIAL_FLUSH will now emit only one
empty static block.
