[Fwd: Re: How to start Cluster using SHM for data nodes]

[Jon Stephens]

Forwarding to list since I'm in serious danger of being mistaken for 
someone who actually knows all about this stuff...

-------- Original Message --------
Subject: Re: How to start Cluster using SHM for data nodes
Date: Thu, 22 Oct 2009 08:38:01 -0400
From: Marc Zampetti <Marc.Zampetti@stripped>
To: Jon Stephens <jon.stephens@stripped>
References: <4ADE7905.8080003@stripped> <4AE028E1.6050003@stripped>

Jon,

Yes, I knew that the feature was somewhat experimental, so I'm not
terribly surprised with your answer. Thank you for getting back to be on
that.

Let me describe the problem I am having, maybe you can suggest a
solution. I'm trying to run two data nodes on the same box in order to
take advantage of all of the cores and memory the box has. My database
is very high volume, but not high storage. Lots of writes, but not a
huge amount of data. The issue is that I'm running into issues with the
number of packets the box can support. Since the size of the packets are
small, the overall network bandwidth is not an issue, but the number of
packets that the data nodes are generating is more than the box can
handle, and thus is limiting scalability. I have a couple of other apps,
in house C-based apps, were we so the same issues. The solution in those
cases was to utilize the TCP_CORK capabilities in Linux. This brought
the packet rates down, and while it did affect per -transaction
throughput a bit, the overall utilization and total throughput was
increased. It also reduced CPU utilization due to the reduced network
packet handling overhead.

Is there similar a way to enable similar functionality with NDB? I've
turned on the real-time extensions, which did help by the way, but now
I'm running into this packet issue. All the docs I've seen around TCP
tuning haven't rung a bell yet, so I haven't tried anything like that.
Anything you can point to that might help?

Marc Zampetti



On 10/22/09 5:41 AM, Jon Stephens wrote:
> Marc Zampetti wrote:
>> I have a cluster on two hosts with 4 data nodes. I want to test using 
>> shared memory links for the data nodes that are on the same hosts. 
>> What is the correct way to enable shm links for this setup, including 
>> any OS-level configurations. All of the nodes are running as the 
>> "mysql" user, and this is on CentOS 5 64-bit. I'm starting the 
>> cluster from scratch, with no data.
>
> Hi Marc,
>
> You're probably not going to like this answer very much, but right now 
> we don't really support SHM configuration and can make no guarantees 
> if it even works in any given release.
>
> Also, as it says here
>
> http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-shm-definition.html
>
> our own testing suggests that there's really no performance or any 
> other advantage to using the SHM transporter over TCP/IP.
>
> I spoke about this with one of our developers, and he says (more or 
> less), that currently SHM is really only of interest if you're 
> prepared to go digging into the code and possibly fixing it yourself.
>
> Sorry.
>
> Thanks!
>
> jon.
>
>>
>> Setup
>>
>> host 1 - data node id 2, data node id 4.
>> host 2 - data node id 3, data node id 5.
>>
>> config.ini
>>
>> [NDBD DEFAULT]
>> NoOfReplicas=2
>> Datadir=/db/data/cluster
>> FileSystemPathDD=/db/data/cluster
>> #FileSystemPathUndoFiles=/db/data/cluster
>> #FileSystemPathDataFiles=/db/data/cluster
>> DataMemory=3072M
>> IndexMemory=1024M
>> LockPagesInMainMemory=1
>> DiskLess=1
>>
>> [shm default]
>> SigNum=10
>>
>> [ndbd]
>> Id=2
>> Hostname=host1
>>
>> [ndbd]
>> Id=3
>> Hostname=host2
>>
>> [ndbd]
>> Id=4
>> Hostname=host1
>>
>> [ndbd]
>> Id=5
>> Hostname=host2
>>
>> [shm]
>> NodeId1=2
>> NodeId2=4
>> ShmKey=2424
>>
>> [shm]
>> NodeId1=3
>> NodeId2=5
>> HsmKey=3535
>>
>> Some questions that I have.
>> 1) Do I need to define links for each direction. I.e. do I need to 
>> have a [shm] section for NodeId1=2, NodeId2=4 and another [shm] 
>> section with NodeId1=4 and NodeId2=2? If so, do I need to define a 
>> separate Shmkey, or use the same key? Any other things I need to 
>> configure? What order do I need to start things in? Is it different 
>> then normally?
>>
>>
>
>


-- 


Jon Stephens - jon.stephens@stripped
Technical Writer
MySQL Documentation Team
Sun Microsystems AB
MySQL and Software Infrastructure Group
Liljeholmen (Stockholm), Sweden
Summer: UTC +02.00 / Winter: UTC +01.00
Mobile: [phone]
Skype: plastic-fish
MySQL: www.mysql.com
Sun: www.sun.com