List:Commits« Previous MessageNext Message »
From:John David Duncan Date:April 9 2011 7:40am
Subject:bzr push into mysql-5.1-telco-7.2 branch (john.duncan:4157 to 4158)
View as plain text  
 4158 John David Duncan	2011-04-09
      New "sandbox.sh" script to quickly demonstate cluster & memcached; fixed metadata.sql; revised README.
      Fixed bugs so that memcapable tests all pass, for both "db-only" and "cache-only" policies.

    added:
      storage/ndb/memcache/sandbox.sh.in
    modified:
      storage/ndb/memcache/Makefile.am
      storage/ndb/memcache/README
      storage/ndb/memcache/include/Configuration.h
      storage/ndb/memcache/scripts/metadata.sql
      storage/ndb/memcache/src/Configuration.cc
      storage/ndb/memcache/src/ndb_engine.c
      storage/ndb/memcache/src/ndb_worker.cc
      storage/ndb/memcache/src/schedulers/Flex.cc
      storage/ndb/memcache/src/schedulers/Flex_cluster.cc
      storage/ndb/memcache/src/schedulers/Stockholm.cc
      storage/ndb/ndb_configure.m4
 4157 John David Duncan	2011-04-08
      Implement append & prepend; run-tests.sh now passes all tests.

    modified:
      storage/ndb/memcache/include/NdbInstance.h
      storage/ndb/memcache/include/Scheduler.h
      storage/ndb/memcache/run-tests.sh
      storage/ndb/memcache/src/NdbInstance.cc
      storage/ndb/memcache/src/ndb_engine.c
      storage/ndb/memcache/src/ndb_worker.cc
      storage/ndb/memcache/src/schedulers/Flex_cluster.cc
=== modified file 'storage/ndb/memcache/Makefile.am'
--- a/storage/ndb/memcache/Makefile.am	2011-04-07 11:20:56 +0000
+++ b/storage/ndb/memcache/Makefile.am	2011-04-09 07:36:24 +0000
@@ -53,8 +53,8 @@ ndb_engine_la_SOURCES= \
                     src/thread_identifier.c \
                     src/timing.c \
                     src/workitem.c  \
-		    cache-src/default_engine.c \
-		    cache-src/assoc.c \
+                    cache-src/default_engine.c \
+            		    cache-src/assoc.c \
                     cache-src/items.c \
                     cache-src/slabs.c
 

=== modified file 'storage/ndb/memcache/README'
--- a/storage/ndb/memcache/README	2011-04-06 05:45:04 +0000
+++ b/storage/ndb/memcache/README	2011-04-09 07:36:24 +0000
@@ -1,40 +1,57 @@
-NDB Memcache -- Prototype release
+-------------------------------------------------------------------------
+NDB Engine for Memcached -- April 2011 Preview Release
+-------------------------------------------------------------------------
 
 Memcached is a high-performance data cache.  Memcached servers are often 
 expected to serve 100,000 operations per second or more.  
 
 The NDB Engine for memcached is a storage engine for Memcached 1.6 
 (which is not yet released).  It is designed to do everything memcached does, 
-with the same  performance, and automatically persist data to MySQL Cluster.  
+with similar performance, and automatically persist data to MySQL Cluster.  
 It is intended to be highly configurable, so that multiple memcached servers can
 connect to multiple clusters and access data from many different tables. 
 It will support a large number of cache policies, based on the prefix of a 
 memcache key.  Some keys can be stored in local cache only, while other keys are 
-written though synchronously or asynchronously to the database, and still other 
-keys go directly to the database without using a local cache at all.
+written though synchronously to the database, and still other keys go directly 
+to the database without using a local cache at all.
+
+In the default configuration, all values will be persistently stored in 
+MySQL Cluster, except for keys which begin with "mc:" -- those will only be 
+cached locally in memcached.
+
 
 -------------------------------------------------------------------------
 QUICK START
+-------------------------------------------------------------------------
+
+# PREREQUISITES
+# --------------------------
+# You will be obtaining a development repository of memcached 1.6 and 
+# building it.  This will require a recent version of git. 
+# Git can be obtained from http://git-scm.com/ 
+# It also requires GNU autotools, including autoconf 2.60 or newer.
+#
+# To test memcached, you will at least need "telnet", but you might also
+# want a memcached client library such as libmemcached, from libmemcached.org
 
 
 # OBTAIN MEMCACHED
 # --------------------------
-# Obtain memcached 1.6 source from github, and then checkout the 
+# Use git to obtain memcached 1.6 source from github, and then checkout the 
 # "engine-pu" branch which contains the "storage engine API" code
 #
 git clone git://github.com/memcached/memcached
 cd memcached
-MEMCACHED_SRC_DIR=`pwd`
 git checkout -t origin/engine-pu 
 
 
 # DECIDE WHERE MEMCACHED WILL BE INSTALLED
 # --------------------------
 # You will have to configure memcached and the NDB engine separately, but
-# it is important to install them in the same place
+# we recommend that you install them in the same place
 #
 INSTALL_PREFIX=/usr/local/mysql-memcache
-
+export INSTALL_PREFIX
 
 # BUILD AND INSTALL MEMCACHED
 # --------------------------
@@ -51,50 +68,36 @@ sudo make install
 # cd into the top mysql source directory, and build and install mysql cluster 7.2
 # 
 sh BUILD/autorun.sh
-./configure --with-plugins=ndbcluster --prefix=$INSTALL_PREFIX
+./configure --with-plugins=ndbcluster \
+   --with-memcache=$INSTALL_PREFIX --prefix=$INSTALL_PREFIX
 make 
 sudo make install 
 
 
-# BUILD AND INSTALL THE NDB ENGINE FOR MEMCACHED
-# --------------------------
-# Use the same install location as before.
+# START MYSQL CLUSTER AND MEMCACHED IN THE SANDBOX
+# ------------------------------------------------
+# The sandbox.sh script will start a cluster, load the NDB Memcache 
+# configuration metadata, and start a memcached server.  For details on how 
+# this is done, refer to the script itself.  
+# You can use "sandbox.sh stop" to stop this cluster, and after stopping can
+# completely remove the sandbox by just deleting the "sandbox" directory.
+# 
 cd storage/ndb/memcache
-sh autorun.sh
-./configure \
-  --prefix=$INSTALL_PREFIX \
-  --with-memcached-src=$MEMCACHED_SRC_DIR \
-  --enable-debug
-make
-sudo make install
-
-
-# START MYSQL CLUSTER
-# --------------------------
-# (Unfortunately this part does not work yet:) 
-make start-sandbox
+sh sandbox.sh 
 
 
-# CREATE THE DATABASE OBJECTS
-# --------------------------
-mysql -u root < scripts/metadata.sql 
+# TEST MEMCACHED
+# --------------
+# If you have installed libmemcached, then you can run the test script,
+# which is based on "memcapable".
+# 
+sh run-tests.sh
 
 
-# START THE SERVER
+# RUN SOME CLIENT COMMANDS 
 # --------------------------
-# The "-E" option names a storage engine, and "-e" is for engine-specific  
-# options.  You could also start a standard memcached server, like this: 
-# bin/memcached -E lib/memcached/default_engine.so 
-
-cd $INSTALL_PREFIX
-bin/memcached -E lib/ndb_engine.so -e "connectstring=localhost:1186;role=db-only"
-
-
-
-RUN SOME CLIENT COMMANDS 
---------------------------
-From another session, you can run any sort of memcached client using either the 
-text protocol or the binary protocol -- including plain "telnet".  
+# From another session, you can run any sort of memcached client using either the 
+# text protocol or the binary protocol -- including plain "telnet".  
 
 $ telnet localhost 11211
 Escape character is '^]'.

=== modified file 'storage/ndb/memcache/include/Configuration.h'
--- a/storage/ndb/memcache/include/Configuration.h	2011-04-06 23:52:36 +0000
+++ b/storage/ndb/memcache/include/Configuration.h	2011-04-09 07:36:24 +0000
@@ -84,6 +84,7 @@ class Configuration {
   const KeyPrefix * getPrefixForKey(const char *key, int nkey) const;
   const KeyPrefix * getPrefixByInfo(const prefix_info_t info) const;
   const KeyPrefix * getPrefix(int id) const;                         // inlined
+  const KeyPrefix * getNextPrefixForCluster(int cluster_id, KeyPrefix *k) const;
   void setPrimaryConnectString(const char *);                        // inlined
   void setServerRole(const char *);                                  // inlined
   bool prefetchDictionary();

=== added file 'storage/ndb/memcache/sandbox.sh.in'
--- a/storage/ndb/memcache/sandbox.sh.in	1970-01-01 00:00:00 +0000
+++ b/storage/ndb/memcache/sandbox.sh.in	2011-04-09 07:36:24 +0000
@@ -0,0 +1,166 @@
+#!/bin/sh
+
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+bindir=@bindir@
+libexecdir=@libexecdir@
+
+MYSQL_BIN=$bindir
+MYSQL_LIBEXEC=$libexecdir
+MEMCACHE_BASE=@MEMCACHE_PATH@
+
+HOME_BASE=@abs_top_srcdir@/storage/ndb/memcache
+
+build_dirs() {
+  echo Creating sandbox
+  mkdir $HOME_BASE/sandbox
+  mkdir $HOME_BASE/sandbox/data
+  mkdir $HOME_BASE/sandbox/ndb 
+  mkdir $HOME_BASE/sandbox/ndb-config
+}
+
+write_my_cnf() {
+  ( echo "[mysqld]"
+    echo "ndbcluster"
+    echo "datadir=$HOME_BASE/sandbox/data"
+    echo "pid-file=$HOME_BASE/sandbox/mysqld.pid"
+    echo "user="`whoami`
+    echo
+  ) > $HOME_BASE/sandbox/my.cnf
+ }
+
+write_cluster_ini() {
+  ( echo "[MGM]"
+    echo "NodeId=1"
+    echo "datadir=$HOME_BASE/sandbox/ndb"
+    echo "hostname=localhost"
+    echo 
+    echo "[DB DEFAULT]"
+    echo "NoOfReplicas=1"
+    echo "TimeBetweenGlobalCheckpoints=12000"
+    echo "DataMemory=180M"
+    echo "IndexMemory=60M"
+    echo "HeartbeatIntervalDbApi=15000"
+    echo
+    echo "[NDBD]"
+    echo "NodeId=2"
+    echo "datadir=$HOME_BASE/sandbox/ndb"
+    echo 
+    echo "[API]"
+    echo "[API]"
+    echo "[API]"
+    echo "[API]"
+    echo "[API]"
+    echo "[API]"
+    echo "[API]"
+  ) > $HOME_BASE/sandbox/cluster.ini
+}
+
+do_install_db() {
+  $MYSQL_BIN/mysql_install_db \
+    --basedir=@prefix@ --datadir=$HOME_BASE/sandbox/data \
+     > /dev/null  && echo Created MySQL System Tables
+}
+
+start_mgm_server() {
+  INITIAL="--initial"
+  test -f sandbox/ndb-config/ndb_1_config.bin.1 && INITIAL=""
+
+  $MYSQL_LIBEXEC/ndb_mgmd -f $HOME_BASE/sandbox/cluster.ini \
+    --config-dir=$HOME_BASE/sandbox/ndb-config $INITIAL
+    
+  sleep 5
+}
+
+start_ndbd() {
+  $MYSQL_LIBEXEC/ndbd 
+  sleep 5
+}
+
+start_mysql_server() {
+  $MYSQL_BIN/mysqld_safe --defaults-file=$HOME_BASE/sandbox/my.cnf &
+  sleep 5
+}
+
+mysqld_is_running() {  
+   test_pid mysqld.pid
+}
+
+mgm_server_is_running() {  
+   test_pid ndb/ndb_1.pid
+}
+
+ndbd_is_running()  {  
+   test_pid ndb/ndb_2.pid
+}
+
+memcached_is_running() {
+   test_pid memcached.pid
+}
+
+test_pid() {
+  test -f sandbox/$1  || return -1
+  SERVERPID=`cat $HOME_BASE/sandbox/$1`
+  ps -p "$SERVERPID" > /dev/null || return -1
+  return 0
+}
+
+load_metadata() {
+  echo Loading NDB Memcache configuration data
+  $MYSQL_BIN/mysql -u root < $HOME_BASE/scripts/metadata.sql || exit
+}
+
+start_memcached() {
+  $MEMCACHE_BASE/bin/memcached -d -v \
+    -P $HOME_BASE/sandbox/memcached.pid \
+    -E $MEMCACHE_BASE/lib/ndb_engine.so
+}
+
+stop_memcached() {
+  kill `cat $HOME_BASE/sandbox/memcached.pid`
+}
+
+stop_mysqld() {
+  $MYSQL_BIN/mysqladmin -u root shutdown
+}
+
+stop_mgm_server() {
+  $MYSQL_BIN/ndb_mgm -e shutdown
+}
+
+final_message() {
+  sleep 2
+  echo
+  echo "Use \"sh sandbox.sh stop\" to stop memcached & MySQL Cluster"
+  echo 
+}
+
+
+# ------------------------------ main script --------------------------
+
+( cd $HOME_BASE 
+
+if test "$1" = "stop" 
+ then 
+   mysqld_is_running        &&   stop_mysqld
+   memcached_is_running     &&   stop_memcached
+   mgm_server_is_running    &&   stop_mgm_server
+
+ else 
+  
+  test -d sandbox                     ||  build_dirs
+  test -f sandbox/my.cnf              ||  write_my_cnf
+  test -f sandbox/cluster.ini         ||  write_cluster_ini
+  test -d sandbox/data/mysql          ||  do_install_db
+
+  mgm_server_is_running               ||  start_mgm_server
+  ndbd_is_running                     ||  start_ndbd 
+  mysqld_is_running                   ||  start_mysql_server
+  
+  test -d sandbox/data/ndbmemcache    ||  load_metadata
+  
+  memcached_is_running                ||  start_memcached  
+
+  final_message
+fi
+)

=== modified file 'storage/ndb/memcache/scripts/metadata.sql'
--- a/storage/ndb/memcache/scripts/metadata.sql	2011-04-06 04:31:28 +0000
+++ b/storage/ndb/memcache/scripts/metadata.sql	2011-04-09 07:36:24 +0000
@@ -217,14 +217,17 @@ INSERT INTO ndb_clusters values (0, @@nd
 
 -- cache_policies table
 -- Create some sample policies.
-INSERT INTO cache_policies
+INSERT INTO cache_policies (policy_name, get_policy, set_policy, delete_policy)
   VALUES("memcache-only", "cache_only", "cache_only", "cache_only"),
-        ("ndb-only", "ndb_only", "ndb_only", "ndb_only"),
-        ("ndb-test", "ndb_only", "ndb_only", "ndb_only", "true"),
+        ("ndb-only", "ndb_only", "ndb_only", "ndb_only"),  /* no flush */
+        ("ndb-test", "ndb_only", "ndb_only", "ndb_only"),  /* flush allowed */
         ("caching", "caching", "caching", "caching"),
         ("caching-with-local-deletes", "caching", "caching", "cache_only"),
         ("ndb-read-only", "ndb_only", "disabled", "disabled");
 
+-- FLUSH_ALL is enabled on the ndb-test policy
+UPDATE cache_policies set flush_from_db = "true" where policy_name = "ndb-test";
+
 -- containers table 
 INSERT INTO containers 
   SET name = "demo_table", db_schema = "ndbmemcache", db_table = "demo_table",
@@ -237,14 +240,16 @@ INSERT INTO containers 
 -- key_prefixes table
 INSERT INTO key_prefixes (server_role_id, key_prefix, cluster_id, 
                           policy, container)
-  VALUES (0, "mc:", 0, "memcache-only", NULL),
-         (0, "db:", 0, "ndb-only", "demo_table"),
-         (0, "",    0, "caching", "demo_table"),
+  VALUES (0, "",    0, "ndb-test", "demo_table"),
+         (0, "mc:", 0, "memcache-only", NULL),
+         (0, "t:",  0, "ndb-test", "demo_tabs"),    /* default role */
+
          (1, "",    0, "ndb-only", "demo_table"),
-         (1, "t:",  0, "ndb-only", "demo_tabs"),
-         (2, "",    0, "memcache-only", NULL),
-         (3, "",    0, "caching", "demo_table"),
-         (4, "",    0, "ndb-test", "demo_table")
+         (1, "t:",  0, "ndb-only", "demo_tabs"),    /* db-only role */
+
+         (2, "",    0, "memcache-only", NULL),      /* mc-only role */
+
+         (3, "",    0, "caching", "demo_table")     /* ndb-caching role */
 ;
 
 

=== modified file 'storage/ndb/memcache/src/Configuration.cc'
--- a/storage/ndb/memcache/src/Configuration.cc	2011-04-06 23:52:36 +0000
+++ b/storage/ndb/memcache/src/Configuration.cc	2011-04-09 07:36:24 +0000
@@ -207,6 +207,18 @@ const KeyPrefix * Configuration::getPref
 }
 
 
+const KeyPrefix * Configuration::getNextPrefixForCluster(int cluster_id, 
+                                                         KeyPrefix *k) const {
+  int i = 0;
+
+  if(k) while(prefixes[i] != k && i < nprefixes) i++;
+  while(i < nprefixes && prefixes[i]->info.cluster_id != cluster_id) i++;
+
+  if(i == nprefixes) return 0;
+  else return prefixes[i];
+}
+
+
 void Configuration::disconnectAll() {
   DEBUG_ENTER();
   for(int i = 0; i < nclusters; i++) delete connections[i];
@@ -801,7 +813,7 @@ bool config_v1::store_prefix(const char 
     if(table->exp_column)   prefix.info.has_expire_col = 1;
     if(table->flags_column) prefix.info.has_flags_col  = 1;
   }
-  else {
+  else {    
     /* If the prefix does not use NDB, you cannot specify a container. */
     if(table != 0) {
       logger->log(LOG_WARNING, 0, "Error at key prefix \"%s\": "
@@ -811,18 +823,21 @@ bool config_v1::store_prefix(const char 
     }
   }
 
-  /* The cluster_id must refer to a known cluster: */
   int internal_cluster_idx = -1;
-  for(int i = 0 ; i < nclusters ; i++) 
-    if(cluster_ids[i] == cluster_id)
-      internal_cluster_idx = i;
-
-  if(internal_cluster_idx == -1) {
-    logger->log(LOG_WARNING, 0, "Error at key prefix \"%s\": cluster_id %d "
-                                "does not exist in ndb_clusters table.\n", 
-                                name, cluster_id);
-     return false;
-  }  
+
+  if(prefix.info.use_ndb) {
+    /* The cluster_id must refer to a known cluster: */
+    for(int i = 0 ; i < nclusters ; i++) 
+      if(cluster_ids[i] == cluster_id)
+        internal_cluster_idx = i;
+
+    if(internal_cluster_idx == -1) {
+      logger->log(LOG_WARNING, 0, "Error at key prefix \"%s\": cluster_id %d "
+                                  "does not exist in ndb_clusters table.\n", 
+                                  name, cluster_id);
+       return false;
+    }
+  }
   
   /* Tie it all together */
   prefix.info.cluster_id = internal_cluster_idx;

=== modified file 'storage/ndb/memcache/src/ndb_engine.c'
--- a/storage/ndb/memcache/src/ndb_engine.c	2011-04-08 08:14:11 +0000
+++ b/storage/ndb/memcache/src/ndb_engine.c	2011-04-09 07:36:24 +0000
@@ -270,7 +270,6 @@ static ENGINE_ERROR_CODE ndb_allocate(EN
   DEBUG_ENTER();
   struct ndb_engine* ndb_eng = ndb_handle(handle);
   struct default_engine *def_eng = default_handle(ndb_eng);
-
   
   return def_eng->engine.allocate(ndb_eng->m_default_engine, cookie, item, 
                                   key, nkey, nbytes, flags, exptime);
@@ -355,6 +354,7 @@ static void ndb_release(ENGINE_HANDLE* h
   */
   workitem *wqitem = ndb_eng->server.cookie->get_engine_specific(cookie);  
   if(wqitem) {
+    /* pipeline_io_completed()?  */
     DEBUG_PRINT("Releasing workitem %d.%d.", wqitem->pipeline->id, wqitem->id);
     ndb_eng->server.cookie->store_engine_specific(cookie, wqitem->previous);
     workitem_free(wqitem);
@@ -384,7 +384,7 @@ static ENGINE_ERROR_CODE ndb_get(ENGINE_
   /* Is this a callback after completed I/O? */
   if(wqitem && ! wqitem->base.complete) {  
     DEBUG_PRINT("Got read callback on workitem %d.%d: %s", 
-                       pipeline->id, wqitem->id, wqitem->status->comment);
+                pipeline->id, wqitem->id, wqitem->status->comment);
     *item = wqitem->cache_item;
     wqitem->base.complete = 1;
     pipeline_io_completed(pipeline, wqitem);
@@ -400,9 +400,13 @@ static ENGINE_ERROR_CODE ndb_get(ENGINE_
   prefix_info_t prefix = get_prefix_info_for_key(nkey, key);
   
   /* Cache read */
-  if(prefix.do_mc_read) {   
+  if(prefix.do_mc_read) {
     *item = item_get(default_handle(ndb_eng), key, nkey);
-    if (*item != NULL) return ENGINE_SUCCESS;  /* cache hit */
+    if (*item != NULL) {
+      DEBUG_PRINT(" cache hit");
+      return ENGINE_SUCCESS;
+    }
+    DEBUG_PRINT(" cache miss");
   }
 
   /* Build and send the NDB transaction */
@@ -489,6 +493,7 @@ static ENGINE_ERROR_CODE ndb_store(ENGIN
   
   /* write to cache */
   if(prefix.do_mc_write) {
+    DEBUG_PRINT(" cache-only store.");
     return store_item(default_handle(ndb_eng), item, cas, op, cookie);
   }
   
@@ -520,15 +525,15 @@ static ENGINE_ERROR_CODE ndb_arithmetic(
   wqitem = ndb_eng->server.cookie->get_engine_specific(cookie);
   if(wqitem && ! wqitem->base.complete) {
     DEBUG_PRINT("Got arithmetic callback: %s", wqitem->status->comment);
+    ENGINE_ERROR_CODE status = wqitem->status->status;  
     wqitem->base.complete = 1;
     *result = wqitem->math_value;
+    /* There will be no call to release(), so pop and free now. */
     pipeline_io_completed(pipeline, wqitem);
-    if(wqitem->status->status != ENGINE_SUCCESS) {
-      DEBUG_PRINT("pop and free the workitem.");
-      ndb_eng->server.cookie->store_engine_specific(cookie, wqitem->previous);
-      workitem_free(wqitem);
-    }    
-    return wqitem->status->status;  
+    ndb_eng->server.cookie->store_engine_specific(cookie, wqitem->previous);
+    workitem_free(wqitem);
+
+    return status;
   }
   
   prefix_info_t prefix = get_prefix_info_for_key(nkey, key);
@@ -603,14 +608,21 @@ static bool ndb_get_item_info(ENGINE_HAN
                               item_info *item_info)
 {
   struct ndb_engine* ndb_eng = ndb_handle(handle);
+  struct default_engine *def_eng = default_handle(ndb_eng);
+
   workitem *wqitem = ndb_eng->server.cookie->get_engine_specific(cookie);
 
+  if(wqitem == NULL) {
+    DEBUG_PRINT(" cache-only");
+    return def_eng->engine.get_item_info(handle, cookie, item, item_info);
+  }
+
   if (item_info->nvalue < 1) {
     DEBUG_PRINT("nvalue too small.");
     return false;
   }
-
-  if(wqitem && wqitem->base.has_value) {
+  
+  if(wqitem->base.has_value) {
     /* Use the workitem. */
     item_info->cas = wqitem->cas ? *(wqitem->cas) : 0;
     item_info->exptime = 0;  

=== modified file 'storage/ndb/memcache/src/ndb_worker.cc'
--- a/storage/ndb/memcache/src/ndb_worker.cc	2011-04-08 08:14:11 +0000
+++ b/storage/ndb/memcache/src/ndb_worker.cc	2011-04-09 07:36:24 +0000
@@ -206,7 +206,7 @@ bool worker_do_write(workitem *wqitem, b
   op.clearKeyNullBits();
   op.setKeyPart(COL_STORE_KEY, dbkey, wqitem->base.nsuffix);
 
-  /* Allocate and encode the buVALUffer for the row */ 
+  /* Allocate and encode the buffer for the row */ 
   workitem_allocate_rowbuffer_1(wqitem, op.requiredBuffer());
   op.buffer = wqitem->row_buffer_1;
 
@@ -909,7 +909,8 @@ ENGINE_ERROR_CODE ndb_flush_all(ndb_pipe
                        p->table->table_name);
     }
     else DEBUG_PRINT(" Not scanning table %s -- use_ndb:%d flush:%d",
-                     p->table->table_name, p->info.use_ndb, p->info.do_db_flush);    
+                     p->table ? p->table->table_name : "",
+                     p->info.use_ndb, p->info.do_db_flush);
   }
   
   return ENGINE_SUCCESS;

=== modified file 'storage/ndb/memcache/src/schedulers/Flex.cc'
--- a/storage/ndb/memcache/src/schedulers/Flex.cc	2011-04-06 04:32:39 +0000
+++ b/storage/ndb/memcache/src/schedulers/Flex.cc	2011-04-09 07:36:24 +0000
@@ -53,12 +53,12 @@ void Scheduler_flex::init(int t, int nth
   /* Set some baseline default values for the configuration */
   config.n_engine_threads = nthreads;
   config.n_broker_threads = 0;
-  config.n_commit_threads = 2;
+  config.n_commit_threads = 1;
   config.n_db_pending = 0;
   config.n_connections = 1;
   
   /* The default config parameters */
-  static const char * default_config = "b0,c2,p1";
+  static const char * default_config = "b0,c1,p1";
  
   /* Choose which string to parse */
   if(user_config && *user_config) active_cf = user_config;

=== modified file 'storage/ndb/memcache/src/schedulers/Flex_cluster.cc'
--- a/storage/ndb/memcache/src/schedulers/Flex_cluster.cc	2011-04-08 08:14:11 +0000
+++ b/storage/ndb/memcache/src/schedulers/Flex_cluster.cc	2011-04-09 07:36:24 +0000
@@ -88,31 +88,36 @@ Scheduler_flex::Cluster::Cluster(Schedul
   /* Hoard a transaction (an API connect record) for each Ndb object.  This
    first call to startTransaction() will send TC_SEIZEREQ and wait for a 
    reply, but later at runtime startTransaction() should return immediately.
-   Also, for each NDB instance, ore-build the QueryPlan for the default key prefix.
+   Also, pre-build a QueryPlan for each NDB instance.
    TODO? Start one tx *per data node*.
    */
   QueryPlan *plan;
-  const KeyPrefix *default_prefix = conf.getDefaultPrefix();
-  NdbTransaction ** txlist;
-  txlist = ( NdbTransaction **) calloc(nInst, sizeof(NdbTransaction *));
-  
-  // Open them all.
-  for(int i = 0 ; i < nInst ; i++) {
-    NdbTransaction *tx;
-    plan = instances[i]->getPlanForPrefix(default_prefix);
-    tx = instances[i]->db->startTransaction();
-    if(! tx) logger->log(LOG_WARNING, 0, instances[i]->db->getNdbError().message);
-    txlist[i] = tx;
+  const KeyPrefix *prefix = conf.getNextPrefixForCluster(id, NULL);
+  if(prefix) {
+    NdbTransaction ** txlist;
+    txlist = ( NdbTransaction **) calloc(nInst, sizeof(NdbTransaction *));
+  
+    // Open them all.
+    for(int i = 0 ; i < nInst ; i++) {
+      NdbTransaction *tx;
+      plan = instances[i]->getPlanForPrefix(prefix);
+      tx = instances[i]->db->startTransaction();
+      if(! tx) logger->log(LOG_WARNING, 0, instances[i]->db->getNdbError().message);
+      txlist[i] = tx;
+    }
+      
+    // Close them all.
+    for(int i = 0 ; i < nInst ; i++) {
+      txlist[i]->close();
+    }    
+    
+    // Free the list.
+    free(txlist);
+
+    // Also get the plan for the master NDB instance
+    pool->getNdbInstance(sched->thread_id)->getPlanForPrefix(prefix);
   }
-  
-  // Close them all.
-  for(int i = 0 ; i < nInst ; i++) {
-    txlist[i]->close();
-  }    
-  
-  // Free the list.
-  free(txlist);
-  
+
   /* Allocate thread ids */
   commit_thread_ids = (pthread_t *) calloc(sched->config.n_commit_threads, 
                                            sizeof(pthread_t));

=== modified file 'storage/ndb/memcache/src/schedulers/Stockholm.cc'
--- a/storage/ndb/memcache/src/schedulers/Stockholm.cc	2011-04-06 04:31:28 +0000
+++ b/storage/ndb/memcache/src/schedulers/Stockholm.cc	2011-04-09 07:36:24 +0000
@@ -99,19 +99,22 @@ void Scheduler_stockholm::init(int my_th
   QueryPlan *plan;
   const KeyPrefix *default_prefix = conf.getDefaultPrefix();
   for(int c = 0 ; c < conf.nclusters ; c++) {
-    NdbTransaction ** txlist;
-    txlist = ( NdbTransaction **) calloc(cluster[c].nInst, sizeof(NdbTransaction *));
-    // Open them all.
-    for(int i = 0 ; i < cluster[c].nInst ; i++) {
-      plan = cluster[c].instances[i]->getPlanForPrefix(default_prefix);
-      txlist[i] = cluster[c].instances[i]->db->startTransaction();
+    const KeyPrefix *prefix = conf.getNextPrefixForCluster(c, NULL); 
+    if(prefix) {
+      NdbTransaction ** txlist;
+      txlist = ( NdbTransaction **) calloc(cluster[c].nInst, sizeof(NdbTransaction *));
+      // Open them all.
+      for(int i = 0 ; i < cluster[c].nInst ; i++) {
+        plan = cluster[c].instances[i]->getPlanForPrefix(prefix);
+        txlist[i] = cluster[c].instances[i]->db->startTransaction();
+      }
+      // Close them all.
+      for(int i = 0 ; i < cluster[c].nInst ; i++) {
+        txlist[i]->close();
+      }    
+      // Free the list.
+      free(txlist);
     }
-    // Close them all.
-    for(int i = 0 ; i < cluster[c].nInst ; i++) {
-      txlist[i]->close();
-    }    
-    // Free the list.
-    free(txlist);
   }  
 
   /* Allocate and initialize a workqueue for each cluster.

=== modified file 'storage/ndb/ndb_configure.m4'
--- a/storage/ndb/ndb_configure.m4	2011-04-07 11:20:56 +0000
+++ b/storage/ndb/ndb_configure.m4	2011-04-09 07:36:24 +0000
@@ -51,6 +51,8 @@ AC_SUBST(DOXYGEN)
 AC_SUBST(PDFLATEX)
 AC_SUBST(MAKEINDEX)
 
+AC_CHECK_HEADERS_ONCE([mach/mach_time.h])
+
 dnl ---------------------------------------------------------------------------
 dnl Check if ndbmtd should/can be built
 dnl - skipped if with --without-ndbmtd specified
@@ -698,7 +700,7 @@ AC_DEFUN([MYSQL_SETUP_NDBCLUSTER], [
   # checking various functions
   AC_CHECK_FUNCS(pthread_self \
     sched_get_priority_min sched_get_priority_max sched_setaffinity \
-    sched_setscheduler processor_bind epoll_create \
+    sched_setscheduler processor_bind epoll_create srandomdev \
     posix_memalign memalign sysconf directio atomic_swap_32 mlock \
     ffs pthread_mutexattr_init pthread_mutexattr_settype)
 
@@ -935,6 +937,7 @@ AC_DEFUN([MYSQL_SETUP_NDBCLUSTER], [
   AC_SUBST([NDB_SIZEOF_LONG_LONG])
 
   AC_CONFIG_FILES([storage/ndb/include/ndb_types.h])
+  AC_CONFIG_FILES([storage/ndb/memcache/sandbox.sh])
 
 ])
 

No bundle (reason: useless for push emails).
Thread
bzr push into mysql-5.1-telco-7.2 branch (john.duncan:4157 to 4158) John David Duncan9 Apr