List:Cluster« Previous MessageNext Message »
From:Hendrik Woltersdorf Date:June 4 2010 10:37am
Subject:Antwort: Re: uneven data distribution
View as plain text  
Hi,

here is the output for the largest table:

eti-dus-ha-cl1_root# ndb_desc -p -d hacom GLOBAL_ID
-- GLOBAL_ID --
Version: 1
Fragment type: 5
K Value: 6
Min load factor: 78
Max load factor: 80
Temporary table: no
Number of attributes: 4
Number of primary keys: 1
Length of frm data: 365
Row Checksum: 1
Row GCI: 1
SingleUserMode: 0
ForceVarPart: 1
FragmentCount: 2
TableStatus: Retrieved
-- Attributes --
ID_GLOBAL_ID Bigint PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY
AUTO_INCR
ID_SERVICE Int NULL AT=FIXED ST=MEMORY
GLOBAL_ID Varchar(40;latin1_swedish_ci) NULL AT=SHORT_VAR ST=MEMORY
DATE_INSERT Datetime NOT NULL AT=FIXED ST=MEMORY

-- Indexes --
PRIMARY KEY(ID_GLOBAL_ID) - UniqueHashIndex
REFERENCE_3_FK(ID_SERVICE) - OrderedIndex
PRIMARY(ID_GLOBAL_ID) - OrderedIndex
IDX_STRING(GLOBAL_ID) - OrderedIndex

-- Per partition info --
Partition       Row count       Commit count    Frag fixed memory
Frag varsized memory    Extent_space    Free extent_space
0               1575951         3683447         90505216
42991616                0               0
1               1572050         1579967         835551232
41943040                0               0


NDBT_ProgramExit: 0 - OK

The restarted node does not use MORE memory. The amount is the same before
and after the restart.

And yes, this is a table, where we delete periodically lots of rows and it
contains a varchar field.

regards

Hendrik Woltersdorf
XCOM AG
Tel.: 0375/27008-580
Tel. mobil: 0178/5710078
Email: Hendrik.Woltersdorf@stripped


                                                                           
             Johan Andersson                                               
             <johan.andersson@                                             
             oracle.com>                                                An 
                                        Hendrik Woltersdorf                
             04.06.2010 11:42           <hendrik.woltersdorf@stripped>      
                                                                     Kopie 
                                        Cluster <cluster@stripped>  
                                                                     Thema 
                                        Re: uneven data distribution       
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




Hi,

can you send the output from:

ndb_desc -p -d <databasename> <tablename>
on some of the larger tables to verify the distribution?

In general , you can get this discrepancy in datamemory usage if you have
1) deleted a lot of records containg VARCHAR/VARBINARY (var sized attrs)
and the deletion did not free up pages completely so they wasnt returned
2) restarted one data node, this data node will then "compact" the
var-sized area, and then you will have less DataMemory usage on the
restarted node.

Look at
http://johanandersson.blogspot.com/2009/03/memory-deallocationdefragmentation-and.html

http://johanandersson.blogspot.com/2009/08/optimize-table-on-cluster-revisited.html


But the problem you describe, that the restarted node use _more_
memory,that  i haven't seen before.

BR
johan

Hendrik Woltersdorf wrote:
> Hi,
>
> in one of our clusters we have a very uneven data distribution:
>
> ndb_mgm> all report MemoryUsage
> ndb_mgm> Node 10: Data usage is 5%(17307 32K pages of total 327680)
> Node 10: Index usage is 4%(6481 8K pages of total 131104)
> Node 11: Data usage is 19%(62941 32K pages of total 327680)
> Node 11: Index usage is 4%(6398 8K pages of total 131104)
>
> I did a node restart one node 11, but that did not change anything.
> We are using version mysql-5.1.41 ndb-7.0.13.
>
> Is this uneven distribution normal?
> Can I do something to distribute the data evenly?
>
>
> regards
>
> Hendrik Woltersdorf
> XCOM AG


--
MySQL Cluster Mailing List
For list archives: http://lists.mysql.com/cluster
To unsubscribe:
http://lists.mysql.com/cluster?unsub=Hendrik.Woltersdorf@stripped






*** XCOM AG Legal Disclaimer ***

Diese E-Mail einschliesslich ihrer Anhaenge ist vertraulich und ist allein
für den Gebrauch durch den vorgesehenen Empfaenger bestimmt. Dritten ist
das Lesen, Verteilen oder Weiterleiten dieser E-Mail untersagt. Wir bitten,
eine fehlgeleitete E-Mail unverzueglich vollstaendig zu loeschen und uns
eine Nachricht zukommen zu lassen.

This email may contain material that is confidential and for the sole use
of the intended recipient. Any review, distribution by others or forwarding
without express permission is strictly prohibited. If you are not the
intended recipient, please contact the sender and delete all copies.

Hauptsitz: Bahnstrasse 37, D-47877 Willich, USt-IdNr.: DE 812 885 664
Kommunikation: Telefon +49 2154 9209-70, Telefax +49 2154 9209-900,
www.xcom.de
Handelsregister: Amtsgericht Krefeld, HRB 10340
Vorstand: Matthias Albrecht, Marco Marty, Dr. Rainer Fuchs, Dirk Werner
Vorsitzender des Aufsichtsrates: Stefan H. TarachAttachment: [text/html]
Attachment: [text/html]
Attachment: [text/html]
Thread
uneven data distributionHendrik Woltersdorf4 Jun
  • Re: uneven data distributionJohan Andersson4 Jun
    • Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun
Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun
  • Antwort: Re: uneven data distributionHendrik Woltersdorf4 Jun