List:Cluster« Previous MessageNext Message »
From:Luke H. Crouch Date:July 30 2004 1:41pm
Subject:RE: table full error?
View as plain text  
since we have 4 machines with 2GB memory in each, I went ahead and set...

DataMemory: 1600M
IndexMemory: 48M

in the DB DEFAULT section. but when I try to start up now, I get...


[root@rh-mysql-1 2-db2]# ndbd -i
2004-07-30 03:21:33 [NDB] INFO     -- Angel pid: 2904 ndb pid: 2906
2004-07-30 03:21:33 [NDB] INFO     -- NDB Cluster -- DB node 2
2004-07-30 03:21:33 [NDB] INFO     -- Version 3.5.0 (beta) --
2004-07-30 03:21:33 [NDB] INFO     -- Start initiated (version 3.5.0)
Ndb kernel is stuck in: Job Handling
Error handler shutting down system
Error handler shutdown completed - exiting

another question regarding the memory requirement here...if we have 4 nodes, and 2
replicas, and only 1 node on each machine, then each machine will only need .5*DBSIZE of
memory, correct? and for this particular table, if it is, say, 400MB, then each machine
would actually only be using around 200MB for this particular table, right?

thanks,
-L

> -----Original Message-----
> From: Mikael Ronström [mailto:mikael@stripped]
> Sent: Thursday, July 29, 2004 5:29 PM
> To: Crouch, Luke H.
> Cc: cluster@stripped
> Subject: Re: table full error?
> 
> 
> Hi Luke,
> 
> 2004-07-30 kl. 00.07 skrev Crouch, Luke H.:
> 
> > Mikael,
> >
> > the table has 4 million records, each about 100 bytes, and 
> no primary 
> > key (doh!)
> >
> 
> If no primary key exists then an 8 byte primary key will be 
> generated. 
> 100 bytes and default DataMemory setting should not
> provide space for more than around 800k records so 996k 
> records is even 
> more than that. IndexMemory default setting with
> 8 byte PK should also provide space for around 800k records. If you 
> have a 4 node cluster then it can handle twice the amount
> of data.
> 
> > could that be part of our problem? this doesn't seem like 
> an unusually 
> > large table...do you think the default values would be able 
> to handle 
> > a table this size? could it be something else other than 
> these memory 
> > settings?
> >
> 
> You need to increase default settings. Default settings is mostly for 
> easy start on any machine, also a small machine. Sounds like you
> need to set DataMemory to something like 500M and IndexMemory to 
> something like 150M. Check the manual at
> http://dev.mysql.com/doc/mysql/en/MySQL_Cluster_DB_Definition.html
> and the entries for DataMemory and IndexMemory for more info 
> on how to 
> calculate the proper parameters.
> 
> Also I am not exactly sure what your error code was. I can't think of 
> any other table full variant than what here has been discussed but
> providing the error message will clear the doubts away.
> 
> Rgrds Mikael
> 
> > thanks,
> > -L
> >
> >> -----Original Message-----
> >> From: Mikael Ronström [mailto:mikael@stripped]
> >> Sent: Thursday, July 29, 2004 4:47 PM
> >> To: Crouch, Luke H.
> >> Cc: Vinay Joosery; cluster@stripped
> >> Subject: Re: table full error?
> >>
> >>
> >> Hi Luke,
> >> I am referring to the MySQL Manual available on
> >> http://dev.mysql.com/doc/mysql/en/index.html
> >> where chapter 17 is dedicated to MySQL Cluster. This page 
> is updated
> >> immediately as changes are done in the manual.
> >>
> >> To set a parameter to 300 MByte you write 300M as an example.
> >>
> >> The cluster administration guide is a useful resource 
> still but it is
> >> not updated so the
> >> manual contains the "latest and the greatest".
> >>
> >> Rgrds Mikael
> >>
> >> 2004-07-29 kl. 23.30 skrev Crouch, Luke H.:
> >>
> >>> Mikael,
> >>>
> >>> what manual are you referencing? the mysql cluster administration
> >>> guide that I have has the parameters NoOfIndexPages and
> >> NoOfDataPages
> >>> for DB definitions, with a unit of 8KB pages. what unit is the
> >>> DataMemory and IndexMemory parameters expressed in? kB?
> >>>
> >>> is there a kind of 'live' manual available online? where 
> can we get
> >>> the most recent manual each day?
> >>>
> >>> thanks,
> >>> -L
> >>>
> >>>> -----Original Message-----
> >>>> From: Mikael Ronström [mailto:mikael@stripped]
> >>>> Sent: Thursday, July 29, 2004 3:58 PM
> >>>> To: Crouch, Luke H.
> >>>> Cc: Vinay Joosery; cluster@stripped
> >>>> Subject: Re: table full error?
> >>>>
> >>>>
> >>>> Hi Luke,
> >>>> Most likely DataMemory and IndexMemory from your short 
> description.
> >>>> Check the manual for more details on how
> >>>> to set it. Default is DataMemory size of 80000 kBytes sounds
> >>>> reasonable
> >>>> if your records are about 60-70 bytes.
> >>>> IndexMemory is by default set to 24000 kBytes so it is
> >>>> reasonable that
> >>>> this parameter is your problem if you
> >>>> have a 4 byte primary key.
> >>>>
> >>>> Rgrds Mikael
> >>>>
> >>>> 2004-07-29 kl. 22.36 skrev Crouch, Luke H.:
> >>>>
> >>>>> we are trying to load up an NDB table thru one of the API
> >> nodes. we
> >>>>> have a file (generated by a mysqldump) that inserts 4
> >>>> million records,
> >>>>> but when we run it, it stops at about 996k and says 
> table is full!
> >>>>> where is the setting that controls this?
> >>>>>
> >>>>> thanks,
> >>>>> -L
> >>>>>
> >>>
> >>> -- 
> >>> MySQL Cluster Mailing List
> >>> For list archives: http://lists.mysql.com/cluster
> >>> To unsubscribe:
> >>> http://lists.mysql.com/cluster?unsub=1
> >>>
> >>>
> >>>
> >> Mikael Ronström, Senior Software Architect
> >> MySQL AB, www.mysql.com
> >>
> >> Clustering:
> >> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> >>
> >> http://www.eweek.com/article2/0,1759,1567546,00.asp
> >>
> >>
> >>
> >
> > -- 
> > MySQL Cluster Mailing List
> > For list archives: http://lists.mysql.com/cluster
> > To unsubscribe:    
> > http://lists.mysql.com/cluster?unsub=1
> >
> >
> >
> Mikael Ronström, Senior Software Architect
> MySQL AB, www.mysql.com
> 
> Clustering:
> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> 
> http://www.eweek.com/article2/0,1759,1567546,00.asp
> 
> 
> 
Thread
table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch29 Jul
  • Re: table full error?Mikael Ronström29 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?Mikael Ronström30 Jul
RE: table full error?Luke H. Crouch30 Jul
  • Re: table full error?pekka31 Jul
RE: table full error?Luke H. Crouch31 Jul
  • Re: table full error?pekka1 Aug
    • NDB documentationSundeep Narravula1 Aug
      • Re: NDB documentationpekka1 Aug
RE: table full error?Luke H. Crouch2 Aug
  • Re: table full error?pekka2 Aug