From: Mikael Ronström Date: August 4 2004 1:34pm Subject: Re: memory overhead question List-Archive: http://lists.mysql.com/cluster/266 Message-Id: <09C65E65-E61B-11D8-83A6-000A959312A2@mysql.com> MIME-Version: 1.0 (Apple Message framework v618) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Hi Luke, Due to a bug that I am currently fixing you need to have twice the=20 number of operation records as the maximum records involved in a=20 transaction. I am not exactly sure of how big your insert transactions are. I have=20 seen 50.000 records per transaction in another mysqldump usage which=20 would then require around 100.000. =46rom what I have seen, each insert is=20 about 1 MByte in size when using extended_insert, dependent on your=20 record size this should at least not be bigger than 50.000. Rgrds Mikael 2004-08-04 kl. 15.16 skrev Crouch, Luke H.: > thanks Mikael...that's very helpful. we had a very high number for=20 > MaxNoOfConcurrentOperations, which was probably another thing using=20 > lots of our memory. what would be the suggested number of concurrent=20= > operations? right now, we're just trying to load our large table (4=20 > million records, dumped with --extended-inserts)... > > thanks again, > -L > >> -----Original Message----- >> From: Mikael Ronstr=F6m [mailto:mikael@stripped] >> Sent: Wednesday, August 04, 2004 5:39 AM >> To: Crouch, Luke H. >> Cc: cluster@stripped >> Subject: Re: memory overhead question >> >> >> Hi Luke, >> Here are some experiments I performed on my machine. >> >> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE >> VSIZE >> 465 ndbd 0.0% 0:02.35 22 99 155 230M >> 3.25M 232M >> 301M >> 464 ndbd 0.0% 0:00.00 1 9 27 720K >> 3.24M 188K >> 48.7M >> 462 ndb_mgmd 0.0% 0:00.19 11 46 49 6.10M >> 1.30M 1.20M >> 41.0M >> >> This is a printout from a quick-start of a 1-node MySQL Cluster on my >> PowerBook running >> Mac OS X 10.3. So about 300M of virtual address space is used for a >> standard configured >> ndbd process. >> >> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE >> VSIZE >> 486 ndbd 0.0% 0:03.11 22 99 155 367M >> 3.25M 368M >> 437M >> >> This printout was achieved when IndexMemory was set to 50M and >> DataMemory to 190M. >> Memory increased as much as IndexMemory and DataMemory increased. >> >> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE >> VSIZE >> 496 ndbd 0.8% 0:03.15 20 95 149 405M+ >> 3.25M 406M+ >> 476M >> >> And changing MaxNoOfConcurrentOperations to 65536 adds another 39M of >> memory. >> >> Optimisations of MySQL Cluster both in terms of memory usage and >> improved performance >> of mysql clients using the storage engine is on the TODO >> list. At first >> however we focus on >> ensuring that MySQL Cluster is fully integrated with MySQL. >> >> Rgrds Mikael >> >> >> 2004-08-03 kl. 17.23 skrev Crouch, Luke H.: >> >>> when we start our cluster up with default settings (DataMemory: >>> 80000k, IndexMemory: 24000k), and check the memory usage on our >>> different nodes, it shows ndbd size as 2395M! and the >> machine is using >>> 2G memory, and 1G of swap! >>> >>> are these numbers overhead with only 80M of memory >> allocated to data? >>> how much memory is required other than the memory dedicated to data >>> via DataMemory setting? >>> >>> thanks, >>> -L >>> >>> --=20 >>> MySQL Cluster Mailing List >>> For list archives: http://lists.mysql.com/cluster >>> To unsubscribe: >>> http://lists.mysql.com/cluster?unsub=3Dmikael@stripped >>> >>> >>> >> Mikael Ronstr=F6m, Senior Software Architect >> MySQL AB, www.mysql.com >> >> Clustering: >> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html >> >> http://www.eweek.com/article2/0,1759,1567546,00.asp >> >> >> > > Mikael Ronstr=F6m, Senior Software Architect MySQL AB, www.mysql.com Clustering: http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html http://www.eweek.com/article2/0,1759,1567546,00.asp