List:Cluster« Previous MessageNext Message »
From:Steiner Helmut Date:March 21 2011 2:10pm
Subject:Wasting space with NDBCLUSTER and STORAGE DISK
View as plain text  
Hi,
we have major troubles with our configuration of  MySQL Cluster 7.1.3.

We try to use to store large data volumn tables on DISK using the
following configuration.


Unfortunatly we end up with the following message from MySql which we
don't understand as there is enough disk space available.

2011-03-17 16:24:11,193 WARN  [
JDBCExceptionReporter ] SQL Error: 1114, SQLState: HY000
2011-03-17 16:24:11,193 ERROR [
JDBCExceptionReporter ] The table 'TM' is full
2011-03-17 16:24:11,193 ERROR [
AbstractFlushingEventListener ] Could not synchronize database state
with session
org.hibernate.exception.GenericJDBCException: could not insert
collection: [TMP.ms#412859]
        at
org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQ
LStateConverter.java:126)
        at
org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java
:114)
        at
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.
java:66)
        at
org.hibernate.persister.collection.AbstractCollectionPersister.recreate(
AbstractCollectionPersister.java:1205)
        at
org.hibernate.action.CollectionRecreateAction.execute(CollectionRecreate
Action.java:58)
        at
org.hibernate.engine.ActionQueue.execute(ActionQueue.java:279)
        at
org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
        at
org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:171)
        at
org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(
AbstractFlushingEventListener.java:321)
        at
org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEv
entListener.java:50)
        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1027)

After inserting a number of data (approx 1Million Rows) all extents are
eaten up and no space is available although only minimal data of around
150MB has been inserted.

mysql> SELECT FILE_NAME, (FREE_EXTENTS*EXTENT_SIZE) AS FREE_SPACE,
(TOTAL_EXTENTS*EXTENT_SIZE) AS TOTAL_SPACE, EXTRA FROM
information_schema.FILES WHERE FILE_TYPE = 'DATAFILE';
+---------------------+-------------+-------------+----------------+
| FILE_NAME           | FREE_SPACE  | TOTAL_SPACE | EXTRA          |
+---------------------+-------------+-------------+----------------+
| data_cmn_free_2.dat |           0 | 34359738368 | CLUSTER_NODE=4 |
| data_cmn_free_2.dat |           0 | 34359738368 | CLUSTER_NODE=5 |
| data_cmn_free_2.dat |           0 | 34359738368 | CLUSTER_NODE=6 |
| data_cmn_free_1.dat |           0 | 34359738368 | CLUSTER_NODE=4 |
| data_cmn_free_1.dat |           0 | 34359738368 | CLUSTER_NODE=5 |
| data_cmn_free_1.dat |           0 | 34359738368 | CLUSTER_NODE=6 |
+---------------------+-------------+-------------+----------------+
6 rows in set (0.02 sec)

Any advice is appreciated.
Cheers Helmut


The configuration is as follows:

config.ini

[NDBD DEFAULT]
NoOfReplicas=3
MaxNoOfOrderedIndexes=16384
MaxNoOfAttributes=16384
MaxNoOfConcurrentOperations=524288

DataMemory=1600M
SharedGlobalMemory=256M
DiskPageBufferMemory=128M
IndexMemory=512M
LockPagesInMainMemory=0

TransactionDeadlockDetectionTimeout=600000
TransactionInactiveTimeout=0
TimeBetweenInactiveTransactionAbortCheck=1000
DataDir=/var/lib/mysql-cluster

HeartbeatIntervalDbDb=15000
HeartbeatIntervalDbApi=15000

TimeBetweenLocalCheckpoints=20
TimeBetweenGlobalCheckpoints=1000
TimeBetweenEpochs=100

TimeBetweenEpochsTimeout=32000

FragmentLogFileSize=512M

MaxNoOfExecutionThreads=4

[TCP DEFAULT]
SendBufferMemory=256M
ReceiveBufferMemory=256M

# Managment Server (is managing the cluster system)
[NDB_MGMD]
HostName=charlie
Id=1
[NDB_MGMD]
HostName=alan
Id=2
[NDB_MGMD]
HostName=jake
Id=3

# Storage Engines  (where the data come from / goes to)
[NDBD]
HostName=charlie
DataDir= /var/db/mysql-data/db.cmn
Id=4
[NDBD]
HostName=alan
DataDir= /var/db/mysql-data/db.cmn
Id=5
[NDBD]
HostName=jake
DataDir= /var/db/mysql-data/db.cmn
Id=6


# 3 MySQL Clients  (SQL APIs - destination of the real connection from
outside)
[MYSQLD]
HostName=charlie
[MYSQLD]      
HostName=alan
[MYSQLD]
HostName=jake

We used the following script to create a table space as described in
http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data-objects.h
tml


CREATE LOGFILE GROUP LG_CMN_FREE 
ADD UNDOFILE 'undo_cmn_free_1.dat' 
INITIAL_SIZE 256M 
UNDO_BUFFER_SIZE 100M 
ENGINE NDB; 

ALTER LOGFILE GROUP LG_CMN_FREE 
ADD UNDOFILE 'undo_cmn_free_2.dat' 
INITIAL_SIZE 256M 
ENGINE NDB;

CREATE TABLESPACE ts_cmn_free
ADD DATAFILE 'data_cmn_free_1.dat' 
USE LOGFILE GROUP LG_CMN_FREE 
EXTENT_SIZE = 32M
INITIAL_SIZE = 32G
ENGINE NDB; 

ALTER TABLESPACE ts_cmn_free 
ADD DATAFILE 'data_cmn_free_2.dat' 
INITIAL_SIZE = 32G
ENGINE NDB; 

ALTER TABLESPACE ts_cmn_free 
ADD DATAFILE 'data_cmn_free_3.dat' 
INITIAL_SIZE = 32G
ENGINE NDB;

ALTER TABLE `TM` TABLESPACE ts_cmn_free STORAGE DISK ENGINE NDBCLUSTER;









Thread
Wasting space with NDBCLUSTER and STORAGE DISKSteiner Helmut21 Mar
  • Re: Wasting space with NDBCLUSTER and STORAGE DISKpetya21 Mar
    • RE: Wasting space with NDBCLUSTER and STORAGE DISKJeff Sturm21 Mar