Hi Luke,
Due to a bug that I am currently fixing you need to have twice the
number of operation records as the maximum records involved in a
transaction.
I am not exactly sure of how big your insert transactions are. I have
seen 50.000 records per transaction in another mysqldump usage which
would
then require around 100.000. From what I have seen, each insert is
about 1 MByte in size when using extended_insert, dependent on your
record
size this should at least not be bigger than 50.000.
Rgrds Mikael
2004-08-04 kl. 15.16 skrev Crouch, Luke H.:
> thanks Mikael...that's very helpful. we had a very high number for
> MaxNoOfConcurrentOperations, which was probably another thing using
> lots of our memory. what would be the suggested number of concurrent
> operations? right now, we're just trying to load our large table (4
> million records, dumped with --extended-inserts)...
>
> thanks again,
> -L
>
>> -----Original Message-----
>> From: Mikael Ronström [mailto:mikael@stripped]
>> Sent: Wednesday, August 04, 2004 5:39 AM
>> To: Crouch, Luke H.
>> Cc: cluster@stripped
>> Subject: Re: memory overhead question
>>
>>
>> Hi Luke,
>> Here are some experiments I performed on my machine.
>>
>> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE
>> VSIZE
>> 465 ndbd 0.0% 0:02.35 22 99 155 230M
>> 3.25M 232M
>> 301M
>> 464 ndbd 0.0% 0:00.00 1 9 27 720K
>> 3.24M 188K
>> 48.7M
>> 462 ndb_mgmd 0.0% 0:00.19 11 46 49 6.10M
>> 1.30M 1.20M
>> 41.0M
>>
>> This is a printout from a quick-start of a 1-node MySQL Cluster on my
>> PowerBook running
>> Mac OS X 10.3. So about 300M of virtual address space is used for a
>> standard configured
>> ndbd process.
>>
>> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE
>> VSIZE
>> 486 ndbd 0.0% 0:03.11 22 99 155 367M
>> 3.25M 368M
>> 437M
>>
>> This printout was achieved when IndexMemory was set to 50M and
>> DataMemory to 190M.
>> Memory increased as much as IndexMemory and DataMemory increased.
>>
>> PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE
>> VSIZE
>> 496 ndbd 0.8% 0:03.15 20 95 149 405M+
>> 3.25M 406M+
>> 476M
>>
>> And changing MaxNoOfConcurrentOperations to 65536 adds another 39M of
>> memory.
>>
>> Optimisations of MySQL Cluster both in terms of memory usage and
>> improved performance
>> of mysql clients using the storage engine is on the TODO
>> list. At first
>> however we focus on
>> ensuring that MySQL Cluster is fully integrated with MySQL.
>>
>> Rgrds Mikael
>>
>>
>> 2004-08-03 kl. 17.23 skrev Crouch, Luke H.:
>>
>>> when we start our cluster up with default settings (DataMemory:
>>> 80000k, IndexMemory: 24000k), and check the memory usage on our
>>> different nodes, it shows ndbd size as 2395M! and the
>> machine is using
>>> 2G memory, and 1G of swap!
>>>
>>> are these numbers overhead with only 80M of memory
>> allocated to data?
>>> how much memory is required other than the memory dedicated to data
>>> via DataMemory setting?
>>>
>>> thanks,
>>> -L
>>>
>>> --
>>> MySQL Cluster Mailing List
>>> For list archives: http://lists.mysql.com/cluster
>>> To unsubscribe:
>>> http://lists.mysql.com/cluster?unsub=1
>>>
>>>
>>>
>> Mikael Ronström, Senior Software Architect
>> MySQL AB, www.mysql.com
>>
>> Clustering:
>> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
>>
>> http://www.eweek.com/article2/0,1759,1567546,00.asp
>>
>>
>>
>
>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com
Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
http://www.eweek.com/article2/0,1759,1567546,00.asp
