List:Cluster« Previous MessageNext Message »
From:Luke H. Crouch Date:July 30 2004 2:25pm
Subject:RE: table full error?
View as plain text  
I don't get the Ndb kernel stuck message, but I do still get an error trying to start
up...

2004-07-30 04:19:52 [NDB] INFO     -- Angel pid: 2097 ndb pid: 2098
2004-07-30 04:19:52 [NDB] INFO     -- NDB Cluster -- DB node 5
2004-07-30 04:19:52 [NDB] INFO     -- Version 3.5.0 (beta) --
2004-07-30 04:20:01 [NDB] INFO     -- Start initiated (version 3.5.0)
Error handler shutting down system
Error handler shutdown completed - exiting

error.log:

Date/Time: Friday 30 July 2004 - 04:20:16
Type of error: error
Message: Memory allocation failure
Fault ID: 2327
Problem data: DBTUP could not allocate memory for Page
Object of reference: Requested: 32768x16000 = 524288000 bytes
ProgramName: NDB Kernel
ProcessID: 2098
TraceFile: NDB_TraceFile_13.trace

that's after I changed the DataMemory to be 500M. and when I check the memory on the
boxes, I see...

Mem:  2061516k av,   55052k used, 2006464k free,       0k shrd,    7324k buff
                     21728k actv,    1936k in_d,       8k in_c

that's 2 GB free memory! and it's having problems allocating just 500M ?

is there a portion of the trace file you'd like to see for this, or do you have an idea of
what is happening now?

thanks again for all the help,
-L

> -----Original Message-----
> From: Mikael Ronström [mailto:mikael@stripped]
> Sent: Friday, July 30, 2004 9:09 AM
> To: Crouch, Luke H.
> Cc: cluster@stripped
> Subject: Re: table full error?
> 
> 
> Hi Luke,
> 
> There is an outstanding bug report I filed on this problem. The 
> workaround is to
> increase Watch Dog Timeout.
> TimeBetweenWatchDogCheck: 30000
> as an example.
> 
> 2004-07-30 kl. 15.41 skrev Crouch, Luke H.:
> 
> > since we have 4 machines with 2GB memory in each, I went ahead and 
> > set...
> >
> > DataMemory: 1600M
> > IndexMemory: 48M
> >
> 
> Would recommend more balance between DataMemory and IndexMemory, also 
> providing some
> memory for other processes running on the machine (if I remember 
> correctly you also run a
> mysqld on the machine.
> 
> So something like:
> DataMemory: 800M
> IndexMemory: 250M
> 
> > in the DB DEFAULT section. but when I try to start up now, I get...
> >
> >
> > [root@rh-mysql-1 2-db2]# ndbd -i
> > 2004-07-30 03:21:33 [NDB] INFO     -- Angel pid: 2904 ndb pid: 2906
> > 2004-07-30 03:21:33 [NDB] INFO     -- NDB Cluster -- DB node 2
> > 2004-07-30 03:21:33 [NDB] INFO     -- Version 3.5.0 (beta) --
> > 2004-07-30 03:21:33 [NDB] INFO     -- Start initiated 
> (version 3.5.0)
> > Ndb kernel is stuck in: Job Handling
> > Error handler shutting down system
> > Error handler shutdown completed - exiting
> >
> > another question regarding the memory requirement here...if 
> we have 4 
> > nodes, and 2 replicas, and only 1 node on each machine, then each 
> > machine will only need .5*DBSIZE of memory, correct? and for this 
> > particular table, if it is, say, 400MB, then each machine would 
> > actually only be using around 200MB for this particular 
> table, right?
> >
> 
> Right.
> 
> Rgrds Mikael
> 
> > thanks,
> > -L
> >
> >> -----Original Message-----
> >> From: Mikael Ronström [mailto:mikael@stripped]
> >> Sent: Thursday, July 29, 2004 5:29 PM
> >> To: Crouch, Luke H.
> >> Cc: cluster@stripped
> >> Subject: Re: table full error?
> >>
> >>
> >> Hi Luke,
> >>
> >> 2004-07-30 kl. 00.07 skrev Crouch, Luke H.:
> >>
> >>> Mikael,
> >>>
> >>> the table has 4 million records, each about 100 bytes, and
> >> no primary
> >>> key (doh!)
> >>>
> >>
> >> If no primary key exists then an 8 byte primary key will be
> >> generated.
> >> 100 bytes and default DataMemory setting should not
> >> provide space for more than around 800k records so 996k
> >> records is even
> >> more than that. IndexMemory default setting with
> >> 8 byte PK should also provide space for around 800k records. If you
> >> have a 4 node cluster then it can handle twice the amount
> >> of data.
> >>
> >>> could that be part of our problem? this doesn't seem like
> >> an unusually
> >>> large table...do you think the default values would be able
> >> to handle
> >>> a table this size? could it be something else other than
> >> these memory
> >>> settings?
> >>>
> >>
> >> You need to increase default settings. Default settings is 
> mostly for
> >> easy start on any machine, also a small machine. Sounds like you
> >> need to set DataMemory to something like 500M and IndexMemory to
> >> something like 150M. Check the manual at
> >> http://dev.mysql.com/doc/mysql/en/MySQL_Cluster_DB_Definition.html
> >> and the entries for DataMemory and IndexMemory for more info
> >> on how to
> >> calculate the proper parameters.
> >>
> >> Also I am not exactly sure what your error code was. I 
> can't think of
> >> any other table full variant than what here has been discussed but
> >> providing the error message will clear the doubts away.
> >>
> >> Rgrds Mikael
> >>
> >>> thanks,
> >>> -L
> >>>
> >>>> -----Original Message-----
> >>>> From: Mikael Ronström [mailto:mikael@stripped]
> >>>> Sent: Thursday, July 29, 2004 4:47 PM
> >>>> To: Crouch, Luke H.
> >>>> Cc: Vinay Joosery; cluster@stripped
> >>>> Subject: Re: table full error?
> >>>>
> >>>>
> >>>> Hi Luke,
> >>>> I am referring to the MySQL Manual available on
> >>>> http://dev.mysql.com/doc/mysql/en/index.html
> >>>> where chapter 17 is dedicated to MySQL Cluster. This page
> >> is updated
> >>>> immediately as changes are done in the manual.
> >>>>
> >>>> To set a parameter to 300 MByte you write 300M as an example.
> >>>>
> >>>> The cluster administration guide is a useful resource
> >> still but it is
> >>>> not updated so the
> >>>> manual contains the "latest and the greatest".
> >>>>
> >>>> Rgrds Mikael
> >>>>
> >>>> 2004-07-29 kl. 23.30 skrev Crouch, Luke H.:
> >>>>
> >>>>> Mikael,
> >>>>>
> >>>>> what manual are you referencing? the mysql cluster 
> administration
> >>>>> guide that I have has the parameters NoOfIndexPages and
> >>>> NoOfDataPages
> >>>>> for DB definitions, with a unit of 8KB pages. what unit is the
> >>>>> DataMemory and IndexMemory parameters expressed in? kB?
> >>>>>
> >>>>> is there a kind of 'live' manual available online? where
> >> can we get
> >>>>> the most recent manual each day?
> >>>>>
> >>>>> thanks,
> >>>>> -L
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Mikael Ronström [mailto:mikael@stripped]
> >>>>>> Sent: Thursday, July 29, 2004 3:58 PM
> >>>>>> To: Crouch, Luke H.
> >>>>>> Cc: Vinay Joosery; cluster@stripped
> >>>>>> Subject: Re: table full error?
> >>>>>>
> >>>>>>
> >>>>>> Hi Luke,
> >>>>>> Most likely DataMemory and IndexMemory from your short
> >> description.
> >>>>>> Check the manual for more details on how
> >>>>>> to set it. Default is DataMemory size of 80000 kBytes
> sounds
> >>>>>> reasonable
> >>>>>> if your records are about 60-70 bytes.
> >>>>>> IndexMemory is by default set to 24000 kBytes so it is
> >>>>>> reasonable that
> >>>>>> this parameter is your problem if you
> >>>>>> have a 4 byte primary key.
> >>>>>>
> >>>>>> Rgrds Mikael
> >>>>>>
> >>>>>> 2004-07-29 kl. 22.36 skrev Crouch, Luke H.:
> >>>>>>
> >>>>>>> we are trying to load up an NDB table thru one of the
> API
> >>>> nodes. we
> >>>>>>> have a file (generated by a mysqldump) that inserts 4
> >>>>>> million records,
> >>>>>>> but when we run it, it stops at about 996k and says
> >> table is full!
> >>>>>>> where is the setting that controls this?
> >>>>>>>
> >>>>>>> thanks,
> >>>>>>> -L
> >>>>>>>
> >>>>>
> >>>>> -- 
> >>>>> MySQL Cluster Mailing List
> >>>>> For list archives: http://lists.mysql.com/cluster
> >>>>> To unsubscribe:
> >>>>> http://lists.mysql.com/cluster?unsub=1
> >>>>>
> >>>>>
> >>>>>
> >>>> Mikael Ronström, Senior Software Architect
> >>>> MySQL AB, www.mysql.com
> >>>>
> >>>> Clustering:
> >>>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> >>>>
> >>>> http://www.eweek.com/article2/0,1759,1567546,00.asp
> >>>>
> >>>>
> >>>>
> >>>
> >>> -- 
> >>> MySQL Cluster Mailing List
> >>> For list archives: http://lists.mysql.com/cluster
> >>> To unsubscribe:
> >>> http://lists.mysql.com/cluster?unsub=1
> >>>
> >>>
> >>>
> >> Mikael Ronström, Senior Software Architect
> >> MySQL AB, www.mysql.com
> >>
> >> Clustering:
> >> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> >>
> >> http://www.eweek.com/article2/0,1759,1567546,00.asp
> >>
> >>
> >>
> >>
> Mikael Ronström, Senior Software Architect
> MySQL AB, www.mysql.com
> 
> Clustering:
> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> 
> http://www.eweek.com/article2/0,1759,1567546,00.asp
> 
> 
> 
Thread
table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?pekka31 Jul
RE: table full error?Luke H. Crouch31 Jul
  • Re: table full error?pekka1 Aug
    • NDB documentationSundeep Narravula1 Aug
      • Re: NDB documentationpekka1 Aug
RE: table full error?Luke H. Crouch2 Aug
  • Re: table full error?pekka2 Aug