List:Cluster« Previous MessageNext Message »
From:Mikael Ronström Date:July 30 2004 3:23pm
Subject:Re: table full error?
View as plain text  
Hi Luke,
No, I am pretty clueless on this one. Only possibility I can think of 
is that the OS has fragmented its memory.
Second possibility is lack of swap memory?? It doesn't sound as a 
problem with MySQL Cluster.

Rgrds Mikael

2004-07-30 kl. 16.25 skrev Crouch, Luke H.:

> I don't get the Ndb kernel stuck message, but I do still get an error 
> trying to start up...
>
> 2004-07-30 04:19:52 [NDB] INFO     -- Angel pid: 2097 ndb pid: 2098
> 2004-07-30 04:19:52 [NDB] INFO     -- NDB Cluster -- DB node 5
> 2004-07-30 04:19:52 [NDB] INFO     -- Version 3.5.0 (beta) --
> 2004-07-30 04:20:01 [NDB] INFO     -- Start initiated (version 3.5.0)
> Error handler shutting down system
> Error handler shutdown completed - exiting
>
> error.log:
>
> Date/Time: Friday 30 July 2004 - 04:20:16
> Type of error: error
> Message: Memory allocation failure
> Fault ID: 2327
> Problem data: DBTUP could not allocate memory for Page
> Object of reference: Requested: 32768x16000 = 524288000 bytes
> ProgramName: NDB Kernel
> ProcessID: 2098
> TraceFile: NDB_TraceFile_13.trace
>
> that's after I changed the DataMemory to be 500M. and when I check the 
> memory on the boxes, I see...
>
> Mem:  2061516k av,   55052k used, 2006464k free,       0k shrd,    
> 7324k buff
>                      21728k actv,    1936k in_d,       8k in_c
>
> that's 2 GB free memory! and it's having problems allocating just 500M 
> ?
>
> is there a portion of the trace file you'd like to see for this, or do 
> you have an idea of what is happening now?
>
> thanks again for all the help,
> -L
>
>> -----Original Message-----
>> From: Mikael Ronström [mailto:mikael@stripped]
>> Sent: Friday, July 30, 2004 9:09 AM
>> To: Crouch, Luke H.
>> Cc: cluster@stripped
>> Subject: Re: table full error?
>>
>>
>> Hi Luke,
>>
>> There is an outstanding bug report I filed on this problem. The
>> workaround is to
>> increase Watch Dog Timeout.
>> TimeBetweenWatchDogCheck: 30000
>> as an example.
>>
>> 2004-07-30 kl. 15.41 skrev Crouch, Luke H.:
>>
>>> since we have 4 machines with 2GB memory in each, I went ahead and
>>> set...
>>>
>>> DataMemory: 1600M
>>> IndexMemory: 48M
>>>
>>
>> Would recommend more balance between DataMemory and IndexMemory, also
>> providing some
>> memory for other processes running on the machine (if I remember
>> correctly you also run a
>> mysqld on the machine.
>>
>> So something like:
>> DataMemory: 800M
>> IndexMemory: 250M
>>
>>> in the DB DEFAULT section. but when I try to start up now, I get...
>>>
>>>
>>> [root@rh-mysql-1 2-db2]# ndbd -i
>>> 2004-07-30 03:21:33 [NDB] INFO     -- Angel pid: 2904 ndb pid: 2906
>>> 2004-07-30 03:21:33 [NDB] INFO     -- NDB Cluster -- DB node 2
>>> 2004-07-30 03:21:33 [NDB] INFO     -- Version 3.5.0 (beta) --
>>> 2004-07-30 03:21:33 [NDB] INFO     -- Start initiated
>> (version 3.5.0)
>>> Ndb kernel is stuck in: Job Handling
>>> Error handler shutting down system
>>> Error handler shutdown completed - exiting
>>>
>>> another question regarding the memory requirement here...if
>> we have 4
>>> nodes, and 2 replicas, and only 1 node on each machine, then each
>>> machine will only need .5*DBSIZE of memory, correct? and for this
>>> particular table, if it is, say, 400MB, then each machine would
>>> actually only be using around 200MB for this particular
>> table, right?
>>>
>>
>> Right.
>>
>> Rgrds Mikael
>>
>>> thanks,
>>> -L
>>>
>>>> -----Original Message-----
>>>> From: Mikael Ronström [mailto:mikael@stripped]
>>>> Sent: Thursday, July 29, 2004 5:29 PM
>>>> To: Crouch, Luke H.
>>>> Cc: cluster@stripped
>>>> Subject: Re: table full error?
>>>>
>>>>
>>>> Hi Luke,
>>>>
>>>> 2004-07-30 kl. 00.07 skrev Crouch, Luke H.:
>>>>
>>>>> Mikael,
>>>>>
>>>>> the table has 4 million records, each about 100 bytes, and
>>>> no primary
>>>>> key (doh!)
>>>>>
>>>>
>>>> If no primary key exists then an 8 byte primary key will be
>>>> generated.
>>>> 100 bytes and default DataMemory setting should not
>>>> provide space for more than around 800k records so 996k
>>>> records is even
>>>> more than that. IndexMemory default setting with
>>>> 8 byte PK should also provide space for around 800k records. If you
>>>> have a 4 node cluster then it can handle twice the amount
>>>> of data.
>>>>
>>>>> could that be part of our problem? this doesn't seem like
>>>> an unusually
>>>>> large table...do you think the default values would be able
>>>> to handle
>>>>> a table this size? could it be something else other than
>>>> these memory
>>>>> settings?
>>>>>
>>>>
>>>> You need to increase default settings. Default settings is
>> mostly for
>>>> easy start on any machine, also a small machine. Sounds like you
>>>> need to set DataMemory to something like 500M and IndexMemory to
>>>> something like 150M. Check the manual at
>>>> http://dev.mysql.com/doc/mysql/en/MySQL_Cluster_DB_Definition.html
>>>> and the entries for DataMemory and IndexMemory for more info
>>>> on how to
>>>> calculate the proper parameters.
>>>>
>>>> Also I am not exactly sure what your error code was. I
>> can't think of
>>>> any other table full variant than what here has been discussed but
>>>> providing the error message will clear the doubts away.
>>>>
>>>> Rgrds Mikael
>>>>
>>>>> thanks,
>>>>> -L
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mikael Ronström [mailto:mikael@stripped]
>>>>>> Sent: Thursday, July 29, 2004 4:47 PM
>>>>>> To: Crouch, Luke H.
>>>>>> Cc: Vinay Joosery; cluster@stripped
>>>>>> Subject: Re: table full error?
>>>>>>
>>>>>>
>>>>>> Hi Luke,
>>>>>> I am referring to the MySQL Manual available on
>>>>>> http://dev.mysql.com/doc/mysql/en/index.html
>>>>>> where chapter 17 is dedicated to MySQL Cluster. This page
>>>> is updated
>>>>>> immediately as changes are done in the manual.
>>>>>>
>>>>>> To set a parameter to 300 MByte you write 300M as an example.
>>>>>>
>>>>>> The cluster administration guide is a useful resource
>>>> still but it is
>>>>>> not updated so the
>>>>>> manual contains the "latest and the greatest".
>>>>>>
>>>>>> Rgrds Mikael
>>>>>>
>>>>>> 2004-07-29 kl. 23.30 skrev Crouch, Luke H.:
>>>>>>
>>>>>>> Mikael,
>>>>>>>
>>>>>>> what manual are you referencing? the mysql cluster
>> administration
>>>>>>> guide that I have has the parameters NoOfIndexPages and
>>>>>> NoOfDataPages
>>>>>>> for DB definitions, with a unit of 8KB pages. what unit is
> the
>>>>>>> DataMemory and IndexMemory parameters expressed in? kB?
>>>>>>>
>>>>>>> is there a kind of 'live' manual available online? where
>>>> can we get
>>>>>>> the most recent manual each day?
>>>>>>>
>>>>>>> thanks,
>>>>>>> -L
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Mikael Ronström [mailto:mikael@stripped]
>>>>>>>> Sent: Thursday, July 29, 2004 3:58 PM
>>>>>>>> To: Crouch, Luke H.
>>>>>>>> Cc: Vinay Joosery; cluster@stripped
>>>>>>>> Subject: Re: table full error?
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Luke,
>>>>>>>> Most likely DataMemory and IndexMemory from your short
>>>> description.
>>>>>>>> Check the manual for more details on how
>>>>>>>> to set it. Default is DataMemory size of 80000 kBytes
> sounds
>>>>>>>> reasonable
>>>>>>>> if your records are about 60-70 bytes.
>>>>>>>> IndexMemory is by default set to 24000 kBytes so it is
>>>>>>>> reasonable that
>>>>>>>> this parameter is your problem if you
>>>>>>>> have a 4 byte primary key.
>>>>>>>>
>>>>>>>> Rgrds Mikael
>>>>>>>>
>>>>>>>> 2004-07-29 kl. 22.36 skrev Crouch, Luke H.:
>>>>>>>>
>>>>>>>>> we are trying to load up an NDB table thru one of the
> API
>>>>>> nodes. we
>>>>>>>>> have a file (generated by a mysqldump) that inserts
> 4
>>>>>>>> million records,
>>>>>>>>> but when we run it, it stops at about 996k and says
>>>> table is full!
>>>>>>>>> where is the setting that controls this?
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> -L
>>>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> MySQL Cluster Mailing List
>>>>>>> For list archives: http://lists.mysql.com/cluster
>>>>>>> To unsubscribe:
>>>>>>> http://lists.mysql.com/cluster?unsub=1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Mikael Ronström, Senior Software Architect
>>>>>> MySQL AB, www.mysql.com
>>>>>>
>>>>>> Clustering:
>>>>>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>>>>>
>>>>>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> MySQL Cluster Mailing List
>>>>> For list archives: http://lists.mysql.com/cluster
>>>>> To unsubscribe:
>>>>> http://lists.mysql.com/cluster?unsub=1
>>>>>
>>>>>
>>>>>
>>>> Mikael Ronström, Senior Software Architect
>>>> MySQL AB, www.mysql.com
>>>>
>>>> Clustering:
>>>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>>>
>>>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>>>
>>>>
>>>>
>>>>
>> Mikael Ronström, Senior Software Architect
>> MySQL AB, www.mysql.com
>>
>> Clustering:
>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>
>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>
>>
>>
>>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com

Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp


Thread
table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?pekka31 Jul
RE: table full error?Luke H. Crouch31 Jul
  • Re: table full error?pekka1 Aug
    • NDB documentationSundeep Narravula1 Aug
      • Re: NDB documentationpekka1 Aug
RE: table full error?Luke H. Crouch2 Aug
  • Re: table full error?pekka2 Aug