bk commit into 4.0 tree (ingo:1.2064) BUG#8321

[ingo]

Below is the list of changes that have just been committed into a local
4.0 repository of mydev. When mydev does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet
  1.2064 05/03/12 20:42:39 ingo@stripped +1 -0
  Bug#8321 - myisampack bug in compression algorithm
  This is the second of three changesets. It contains the pure bug fix.
  The problem was that with gcc on x86, shifts are done modulo word size. 
  'value' is 32 bits wide and shifting it by 32 bits is a noop.
  This was triggered by an evil distribution of character incidences. 
  A distribution of 2917027827 characters made of 202 distinct values led to
  34 occurrences of 32-bit Huffman codes.
  This might have been the first time ever that write_bits() had to write
  32-bit values. Since it can be expected that one day even 32 bits might
  be insufficient, the third changeset will suggest to enlarge some variables
  to 64 bits.

  myisam/myisampack.c
    1.36 05/03/12 20:42:37 ingo@stripped +12 -4
    Bug#8321 - myisampack bug in compression algorithm
    This is the second of three changesets. It contains the pure bug fix.
    The problem was that with gcc on x86, shifts are done modulo word size. 
    'value' is 32 bits wide and shifting it by 32 bits is a noop.
    This was triggered by an evil distribution of character incidences. 
    A distribution of 2917027827 characters made of 202 distinct values led to
    34 occurrences of 32-bit Huffman codes.
    This might have been the first time ever that write_bits() had to write
    32-bit values. Since it can be expected that one day even 32 bits might
    be insufficient, the third changeset will suggest to enlarge some variables
    to 64 bits.

# This is a BitKeeper patch.  What follows are the unified diffs for the
# set of deltas contained in the patch.  The rest of the patch, the part
# that BitKeeper cares about, is below these diffs.
# User:	ingo
# Host:	chilla.local
# Root:	/home/mydev/mysql-4.0-bug8321

--- 1.35/myisam/myisampack.c	Fri Mar 11 21:16:57 2005
+++ 1.36/myisam/myisampack.c	Sat Mar 12 20:42:37 2005
@@ -2750,9 +2750,12 @@
     bits= (uint) -file_buffer.bits;
     /*
       Copy the buffer word plus the value - reduced by that number of bits -
-      to the temp word. 
+      to the temp word. If the whole word is left for later accumulation,
+      do not try to shift the value. Some platforms shift modulo word width.
+      This would mess the buffer contents about here. (Bug #8321)
     */
-    byte_buff=file_buffer.byte | (uint) (value >> bits);
+    byte_buff= (file_buffer.byte |
+                ((bits != 8 * sizeof(value)) ? (uint) (value >> bits) : 0));
     /* Copy the temp word - byte by byte, high first - to the file buffer. */
 #if BITS_SAVED == 32
     DBG5(("bit buffer I/O:     0x%02x  bits:  8  bin: %s\n",
@@ -2769,8 +2772,13 @@
     *file_buffer.pos++= (byte) (byte_buff >> 8) ;
     *file_buffer.pos++= (byte) byte_buff;
 
-    /* Clear the copied bits from the value. */
-    value&=(1 << bits)-1;
+    /*
+      Clear the copied bits from the value. Again, do not try to shift
+      a whole word width. Some platforms shift modulo word width.
+      This would destroy the value here. (Bug #8321)
+    */
+    if (bits != 8 * sizeof(value))
+      value&= (1 << bits) - 1;
     DBG5(("prelim value: 0x%08lx  bits: %2u  bin: %s\n",
             value, bits, bindigits(value, bits)));