The previous releases of zlib were not sensitive to incorrect CRC
inputs with bits set above the low 32. This commit restores that
behavior, so that applications with such bugs will continue to
operate as before.
The ARM crc32 instructions will be used only if an architecture is
explicitly specified at compile time that has those instructions.
For example, -march=armv8.1-a or -march=armv8-a+crc, or if the
machine being compiled on has the instructions, -march=native.
Define the macro Z_ARM_CRC32 at compile time to use the ARMv8
(aarch64) crc32x and crc32b instructions. This code does not check
for the presence of the crc32 instructions. Those instructions are
optional for ARMv8.0, though mandatory for ARMv8.1 and later. The
use of the crc32 instructions is about ten times as fast as the
software braided calculation of the CRC-32. This can noticeably
speed up the decompression of gzip streams.
Use the interleaved method of Kadatch and Jenkins in order to make
use of pipelined instructions through multiple ALUs in a single
core. This also speeds up and simplifies the combination of CRCs,
and updates the functions to pre-calculate and use an operator for
CRC combination.
When the same len2 is used repeatedly, it is faster to use
crc32_combine_gen() to generate an operator, that is then used to
combine CRCs with crc32_combine_op().
This isn't the right type anyway to assure that it contains a
pointer. That type would be intptr_t or uintptr_t. However the C99
standard says that those types are optional, so their use would not
be portable. This commit simply uses size_t or whatever configure
decided to use for size_t. That would be the same length as
ptrdiff_t, and so will work just as well. The code checks to see if
the length of the type used is the same as the length of a void
pointer, so there is already protection against the use of the
wrong type. The use of size_t (or ptrdiff_t) will almost always
work, as all modern architectures have an array size that is the
same as the pointer size. Only old segmented architectures would
have to fall back to the slower CRC-32 calculation, where the
amount of memory that can be accessed is larger than the maximum
array size.
See the comment for more details. This is in response to an issue
raised as a result of a security audit of the zlib code by Trail
of Bits and TrustInSoft, in support of the Mozilla Foundation.
There was a small optimization for PowerPCs to pre-increment a
pointer when accessing a word, instead of post-incrementing. This
required prefacing the loop with a decrement of the pointer,
possibly pointing before the object passed. This is not compliant
with the C standard, for which decrementing a pointer before its
allocated memory is undefined. When tested on a modern PowerPC
with a modern compiler, the optimization no longer has any effect.
Due to all that, and per the recommendation of a security audit of
the zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, this "optimization" was removed, in order to
avoid the possibility of undefined behavior.
crc_table is made using a four-byte integer (when that can be
determined). However get_crc_table() returned a pointer to an
unsigned long, which could be eight bytes. This fixes that by
creating a new z_crc_t type for the crc_table.
This type is also used for the BYFOUR crc calculations that depend
on a four-byte type. The four-byte type can now be determined by
./configure, which also solves a problem where ./configure --solo
would never use BYFOUR. No the Z_U4 #define indicates that four-
byte integer was found either by ./configure or by zconf.h.
crc32.c was #including limits.h in order to find a four-byte integer
type. It was doing this even if Z_SOLO were defined, violating the
intent of Z_SOLO, which is to include no library headers and require
no library functions. Now crc32.c obeys the intent of Z_SOLO, but
with the downside that crc32() will be slower than when not compiled
with Z_SOLO. This can be remedied manually by typedefing u4 to a
known four-byte unsigned integer type, and #defining BYFOUR in
crc32.c.
