List:Cluster« Previous MessageNext Message »
From:Hendrik Woltersdorf Date:June 4 2010 1:45pm
Subject:Antwort: Re: uneven data distribution
View as plain text  
I did some research in the logfiles. This started after the last cluster
software upgrade from 7.0.7 to 7.0.13.

Until the upgrade the memory usage was:

2010-05-14 08:54:09 [MgmSrvr] INFO     -- Node 11: Data usage is 6%(21577
32K pages of total 327680)
2010-05-14 08:56:52 [MgmSrvr] INFO     -- Node 10: Data usage is 6%(21583
32K pages of total 327680)
2010-05-14 08:57:21 [MgmSrvr] INFO     -- Node 11: Index usage is 5%(7082
8K pages of total 131104)
2010-05-14 09:01:32 [MgmSrvr] INFO     -- Node 10: Index usage is 5%(7085
8K pages of total 131104)

Right after the upgrade to 7.0.13:

2010-05-14 11:07:57 [MgmtSrvr] INFO     -- Node 10: Data usage is 4%(16226
32K pages of total 327680)
2010-05-14 11:11:40 [MgmtSrvr] INFO     -- Node 10: Index usage is 4%(6495
8K pages of total 131104)
2010-05-14 11:12:25 [MgmtSrvr] INFO     -- Node 11: Data usage is 19%(62601
32K pages of total 327680)
2010-05-14 11:16:12 [MgmtSrvr] INFO     -- Node 11: Index usage is 4%(6494
8K pages of total 131104)

Today I restarted the two datanodes and now it is:

2010-06-04 14:02:24 [MgmtSrvr] INFO     -- Node 10: Data usage is 19%(62562
32K pages of total 327680)
2010-06-04 14:02:24 [MgmtSrvr] INFO     -- Node 10: Index usage is 4%(6423
8K pages of total 131104)
2010-06-04 14:02:24 [MgmtSrvr] INFO     -- Node 11: Data usage is 19%(63917
32K pages of total 327680)
2010-06-04 14:02:24 [MgmtSrvr] INFO     -- Node 11: Index usage is 4%(6426
8K pages of total 131104)

Any ideas, how this can happen ?

regards

Hendrik Woltersdorf
XCOM AG


                                                                           
             Hendrik                                                       
             Woltersdorf                                                   
             <hendrik.woltersd                                          An 
             orf@stripped>               Cluster <cluster@stripped>  
                                                                     Kopie 
             04.06.2010 14:21                                              
                                                                     Thema 
                                        Antwort: Re: uneven data           
                                        distribution                       
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




Hi,

my next test: restart node 10, wich showed 5% data memory usage before.

Now both nodes show a data memory usage of 19% and an even distribution:

eti-dus-ha-cl1_root# ndb_desc -p -d hacom GLOBAL_ID
...

-- Per partition info --
Partition Row count Commit count Frag fixed memory Frag varsized memory
Extent_space Free extent_space
0 1578180 1578409 839942144 42106880 0 0
1 1574362 3158478 835551232 42008576 0 0


NDBT_ProgramExit: 0 - OK

Strange things happen inside of a cluster...

regards

Hendrik Woltersdorf
XCOM AG
Tel.: 0375/27008-580
Tel. mobil: 0178/5710078
Email: Hendrik.Woltersdorf@stripped

Inactive hide details for Hendrik Woltersdorf---04.06.2010 12:37:33---Hi,
here is the output for the largest table:Hendrik Woltersdorf---04.06.2010
12:37:33---Hi, here is the output for the largest table:
                                                                           
                         Hendrik                                           
                         Woltersdorf/DE/X                                  
                         COM                                               
                                                                           
                                                                        An 
                         04.06.2010 12:37                                  
                                                     Cluster               
                                                     <cluster@stripped. 
                                                     com>                  
                                                                           
                                                                     Kopie 
                                                                           
                                                                           
                                                                     Thema 
                                                                           
                                                     Antwort: Re: uneven   
                                                     data distributionMail 
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           


Hi,

here is the output for the largest table:

eti-dus-ha-cl1_root# ndb_desc -p -d hacom GLOBAL_ID
-- GLOBAL_ID --
Version: 1
...

-- Per partition info --
Partition       Row count       Commit count    Frag fixed memory
Frag varsized memory    Extent_space    Free extent_space
0               1575951         3683447         90505216
42991616                0               0
1               1572050         1579967         835551232
41943040                0               0





*** XCOM AG Legal Disclaimer ***

Diese E-Mail einschliesslich ihrer Anhaenge ist vertraulich und ist allein
für den Gebrauch durch den vorgesehenen Empfaenger bestimmt. Dritten ist
das Lesen, Verteilen oder Weiterleiten dieser E-Mail untersagt. Wir bitten,
eine fehlgeleitete E-Mail unverzueglich vollstaendig zu loeschen und uns
eine Nachricht zukommen zu lassen.

This email may contain material that is confidential and for the sole use
of the intended recipient. Any review, distribution by others or forwarding
without express permission is strictly prohibited. If you are not the
intended recipient, please contact the sender and delete all copies.

Hauptsitz: Bahnstrasse 37, D-47877 Willich, USt-IdNr.: DE 812 885 664
Kommunikation: Telefon +49 2154 9209-70, Telefax +49 2154 9209-900,
www.xcom.de
Handelsregister: Amtsgericht Krefeld, HRB 10340
Vorstand: Matthias Albrecht, Marco Marty, Dr. Rainer Fuchs, Dirk Werner
Vorsitzender des Aufsichtsrates: Stefan H. TarachAttachment: [text/html]
Attachment: [text/html]
Attachment: [text/html]
Thread
uneven data distributionHendrik Woltersdorf4 Jun
  • Re: uneven data distributionJohan Andersson4 Jun
    • Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun
Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun
  • Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun