This bug was reported by Danilo Ramos of Eideticom, Inc. It has
lain in wait 13 years before being found! The bug was introduced
in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
option forces the use of fixed Huffman codes. For rare inputs with
a large number of distant matches, the pending buffer into which
the compressed data is written can overwrite the distance symbol
table which it overlays. That results in corrupted output due to
invalid distances, and can result in out-of-bound accesses,
crashing the application.
The fix here combines the distance buffer and literal/length
buffers into a single symbol buffer. Now three bytes of pending
buffer space are opened up for each literal or length/distance
pair consumed, instead of the previous two bytes. This assures
that the pending buffer cannot overwrite the symbol table, since
the maximum fixed code compressed length/distance is 31 bits, and
since there are four bytes of pending space for every three bytes
of symbol space.
When debugging the Huffman coding would warn about resulting codes
greater than 15 bits in length. This is handled properly, and is
not uncommon. This increases the verbosity of the warning by one,
so that it is not displayed by default.
This speeds up level 0 by about a factor of three, as compared to
the previous byte-at-a-time loop. We can do much better though. A
later commit avoids this copy for level 0 with large buffers,
instead copying directly from the input to the output. This commit
still speeds up storing incompressible data found when compressing
normally.
This patch allows zlib to compile cleanly with the -Wcast-qual gcc
warning enabled, but only if ZLIB_CONST is defined, which adds
const to next_in and msg in z_stream and in the in_func prototype.
A --const option is added to ./configure which adds -DZLIB_CONST
to the compile flags, and adds -Wcast-qual to the compile flags
when ZLIBGCCWARN is set in the environment.
Previously, the bit buffer would hold 1 to 16 bits after "all" of the
output is provided after a Z_BLOCK deflate() call. Now at most seven
bits remain in the output buffer after Z_BLOCK. flush_pending() now
flushes the bit buffer before copying out the byte buffer, in order
for it to really flush as much as possible.
Z_PARTIAL_FLUSH would sometimes emit two empty static blocks instead
of one in order to provide enough lookahead for inflate to be able
to decode what was last compressed. inflate no longer needs that
much lookahead, so this removes the possibility of emitting the
second empty static block. Z_PARTIAL_FLUSH will now emit only one
empty static block.
