List:Cluster« Previous MessageNext Message »
From:Mikael Ronström Date:July 30 2004 2:08pm
Subject:Re: table full error?
View as plain text  
Hi Luke,

There is an outstanding bug report I filed on this problem. The 
workaround is to
increase Watch Dog Timeout.
TimeBetweenWatchDogCheck: 30000
as an example.

2004-07-30 kl. 15.41 skrev Crouch, Luke H.:

> since we have 4 machines with 2GB memory in each, I went ahead and 
> set...
>
> DataMemory: 1600M
> IndexMemory: 48M
>

Would recommend more balance between DataMemory and IndexMemory, also 
providing some
memory for other processes running on the machine (if I remember 
correctly you also run a
mysqld on the machine.

So something like:
DataMemory: 800M
IndexMemory: 250M

> in the DB DEFAULT section. but when I try to start up now, I get...
>
>
> [root@rh-mysql-1 2-db2]# ndbd -i
> 2004-07-30 03:21:33 [NDB] INFO     -- Angel pid: 2904 ndb pid: 2906
> 2004-07-30 03:21:33 [NDB] INFO     -- NDB Cluster -- DB node 2
> 2004-07-30 03:21:33 [NDB] INFO     -- Version 3.5.0 (beta) --
> 2004-07-30 03:21:33 [NDB] INFO     -- Start initiated (version 3.5.0)
> Ndb kernel is stuck in: Job Handling
> Error handler shutting down system
> Error handler shutdown completed - exiting
>
> another question regarding the memory requirement here...if we have 4 
> nodes, and 2 replicas, and only 1 node on each machine, then each 
> machine will only need .5*DBSIZE of memory, correct? and for this 
> particular table, if it is, say, 400MB, then each machine would 
> actually only be using around 200MB for this particular table, right?
>

Right.

Rgrds Mikael

> thanks,
> -L
>
>> -----Original Message-----
>> From: Mikael Ronström [mailto:mikael@stripped]
>> Sent: Thursday, July 29, 2004 5:29 PM
>> To: Crouch, Luke H.
>> Cc: cluster@stripped
>> Subject: Re: table full error?
>>
>>
>> Hi Luke,
>>
>> 2004-07-30 kl. 00.07 skrev Crouch, Luke H.:
>>
>>> Mikael,
>>>
>>> the table has 4 million records, each about 100 bytes, and
>> no primary
>>> key (doh!)
>>>
>>
>> If no primary key exists then an 8 byte primary key will be
>> generated.
>> 100 bytes and default DataMemory setting should not
>> provide space for more than around 800k records so 996k
>> records is even
>> more than that. IndexMemory default setting with
>> 8 byte PK should also provide space for around 800k records. If you
>> have a 4 node cluster then it can handle twice the amount
>> of data.
>>
>>> could that be part of our problem? this doesn't seem like
>> an unusually
>>> large table...do you think the default values would be able
>> to handle
>>> a table this size? could it be something else other than
>> these memory
>>> settings?
>>>
>>
>> You need to increase default settings. Default settings is mostly for
>> easy start on any machine, also a small machine. Sounds like you
>> need to set DataMemory to something like 500M and IndexMemory to
>> something like 150M. Check the manual at
>> http://dev.mysql.com/doc/mysql/en/MySQL_Cluster_DB_Definition.html
>> and the entries for DataMemory and IndexMemory for more info
>> on how to
>> calculate the proper parameters.
>>
>> Also I am not exactly sure what your error code was. I can't think of
>> any other table full variant than what here has been discussed but
>> providing the error message will clear the doubts away.
>>
>> Rgrds Mikael
>>
>>> thanks,
>>> -L
>>>
>>>> -----Original Message-----
>>>> From: Mikael Ronström [mailto:mikael@stripped]
>>>> Sent: Thursday, July 29, 2004 4:47 PM
>>>> To: Crouch, Luke H.
>>>> Cc: Vinay Joosery; cluster@stripped
>>>> Subject: Re: table full error?
>>>>
>>>>
>>>> Hi Luke,
>>>> I am referring to the MySQL Manual available on
>>>> http://dev.mysql.com/doc/mysql/en/index.html
>>>> where chapter 17 is dedicated to MySQL Cluster. This page
>> is updated
>>>> immediately as changes are done in the manual.
>>>>
>>>> To set a parameter to 300 MByte you write 300M as an example.
>>>>
>>>> The cluster administration guide is a useful resource
>> still but it is
>>>> not updated so the
>>>> manual contains the "latest and the greatest".
>>>>
>>>> Rgrds Mikael
>>>>
>>>> 2004-07-29 kl. 23.30 skrev Crouch, Luke H.:
>>>>
>>>>> Mikael,
>>>>>
>>>>> what manual are you referencing? the mysql cluster administration
>>>>> guide that I have has the parameters NoOfIndexPages and
>>>> NoOfDataPages
>>>>> for DB definitions, with a unit of 8KB pages. what unit is the
>>>>> DataMemory and IndexMemory parameters expressed in? kB?
>>>>>
>>>>> is there a kind of 'live' manual available online? where
>> can we get
>>>>> the most recent manual each day?
>>>>>
>>>>> thanks,
>>>>> -L
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mikael Ronström [mailto:mikael@stripped]
>>>>>> Sent: Thursday, July 29, 2004 3:58 PM
>>>>>> To: Crouch, Luke H.
>>>>>> Cc: Vinay Joosery; cluster@stripped
>>>>>> Subject: Re: table full error?
>>>>>>
>>>>>>
>>>>>> Hi Luke,
>>>>>> Most likely DataMemory and IndexMemory from your short
>> description.
>>>>>> Check the manual for more details on how
>>>>>> to set it. Default is DataMemory size of 80000 kBytes sounds
>>>>>> reasonable
>>>>>> if your records are about 60-70 bytes.
>>>>>> IndexMemory is by default set to 24000 kBytes so it is
>>>>>> reasonable that
>>>>>> this parameter is your problem if you
>>>>>> have a 4 byte primary key.
>>>>>>
>>>>>> Rgrds Mikael
>>>>>>
>>>>>> 2004-07-29 kl. 22.36 skrev Crouch, Luke H.:
>>>>>>
>>>>>>> we are trying to load up an NDB table thru one of the API
>>>> nodes. we
>>>>>>> have a file (generated by a mysqldump) that inserts 4
>>>>>> million records,
>>>>>>> but when we run it, it stops at about 996k and says
>> table is full!
>>>>>>> where is the setting that controls this?
>>>>>>>
>>>>>>> thanks,
>>>>>>> -L
>>>>>>>
>>>>>
>>>>> -- 
>>>>> MySQL Cluster Mailing List
>>>>> For list archives: http://lists.mysql.com/cluster
>>>>> To unsubscribe:
>>>>> http://lists.mysql.com/cluster?unsub=1
>>>>>
>>>>>
>>>>>
>>>> Mikael Ronström, Senior Software Architect
>>>> MySQL AB, www.mysql.com
>>>>
>>>> Clustering:
>>>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>>>
>>>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> MySQL Cluster Mailing List
>>> For list archives: http://lists.mysql.com/cluster
>>> To unsubscribe:
>>> http://lists.mysql.com/cluster?unsub=1
>>>
>>>
>>>
>> Mikael Ronström, Senior Software Architect
>> MySQL AB, www.mysql.com
>>
>> Clustering:
>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>
>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>
>>
>>
>>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com

Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp


Thread
table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?pekka31 Jul
RE: table full error?Luke H. Crouch31 Jul
  • Re: table full error?pekka1 Aug
    • NDB documentationSundeep Narravula1 Aug
      • Re: NDB documentationpekka1 Aug
RE: table full error?Luke H. Crouch2 Aug
  • Re: table full error?pekka2 Aug