From: Date: March 12 2005 8:42pm Subject: bk commit into 4.0 tree (ingo:1.2064) BUG#8321 List-Archive: http://lists.mysql.com/internals/22966 X-Bug: 8321 Message-Id: Below is the list of changes that have just been committed into a local 4.0 repository of mydev. When mydev does a push these changes will be propagated to the main repository and, within 24 hours after the push, to the public repository. For information on how to access the public repository see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html ChangeSet 1.2064 05/03/12 20:42:39 ingo@stripped +1 -0 Bug#8321 - myisampack bug in compression algorithm This is the second of three changesets. It contains the pure bug fix. The problem was that with gcc on x86, shifts are done modulo word size. 'value' is 32 bits wide and shifting it by 32 bits is a noop. This was triggered by an evil distribution of character incidences. A distribution of 2917027827 characters made of 202 distinct values led to 34 occurrences of 32-bit Huffman codes. This might have been the first time ever that write_bits() had to write 32-bit values. Since it can be expected that one day even 32 bits might be insufficient, the third changeset will suggest to enlarge some variables to 64 bits. myisam/myisampack.c 1.36 05/03/12 20:42:37 ingo@stripped +12 -4 Bug#8321 - myisampack bug in compression algorithm This is the second of three changesets. It contains the pure bug fix. The problem was that with gcc on x86, shifts are done modulo word size. 'value' is 32 bits wide and shifting it by 32 bits is a noop. This was triggered by an evil distribution of character incidences. A distribution of 2917027827 characters made of 202 distinct values led to 34 occurrences of 32-bit Huffman codes. This might have been the first time ever that write_bits() had to write 32-bit values. Since it can be expected that one day even 32 bits might be insufficient, the third changeset will suggest to enlarge some variables to 64 bits. # This is a BitKeeper patch. What follows are the unified diffs for the # set of deltas contained in the patch. The rest of the patch, the part # that BitKeeper cares about, is below these diffs. # User: ingo # Host: chilla.local # Root: /home/mydev/mysql-4.0-bug8321 --- 1.35/myisam/myisampack.c Fri Mar 11 21:16:57 2005 +++ 1.36/myisam/myisampack.c Sat Mar 12 20:42:37 2005 @@ -2750,9 +2750,12 @@ bits= (uint) -file_buffer.bits; /* Copy the buffer word plus the value - reduced by that number of bits - - to the temp word. + to the temp word. If the whole word is left for later accumulation, + do not try to shift the value. Some platforms shift modulo word width. + This would mess the buffer contents about here. (Bug #8321) */ - byte_buff=file_buffer.byte | (uint) (value >> bits); + byte_buff= (file_buffer.byte | + ((bits != 8 * sizeof(value)) ? (uint) (value >> bits) : 0)); /* Copy the temp word - byte by byte, high first - to the file buffer. */ #if BITS_SAVED == 32 DBG5(("bit buffer I/O: 0x%02x bits: 8 bin: %s\n", @@ -2769,8 +2772,13 @@ *file_buffer.pos++= (byte) (byte_buff >> 8) ; *file_buffer.pos++= (byte) byte_buff; - /* Clear the copied bits from the value. */ - value&=(1 << bits)-1; + /* + Clear the copied bits from the value. Again, do not try to shift + a whole word width. Some platforms shift modulo word width. + This would destroy the value here. (Bug #8321) + */ + if (bits != 8 * sizeof(value)) + value&= (1 << bits) - 1; DBG5(("prelim value: 0x%08lx bits: %2u bin: %s\n", value, bits, bindigits(value, bits)));