MySQL Lists are EOL. Please join:

List:Cluster« Previous MessageNext Message »
From:Mauritz Sundell Date:March 5 2018 11:02am
Subject:Re: Antwort: Re: Out of operation records in local data manager far
below MaxNoOfConcurrentOperations
View as plain text  
Hi Hendrik

Do you update some rows several times in same transaction?

In that case you will have skew load and since the local operation 
records are split evenly between the LDM threads that can cause out of 
operations records on a single LDM thread.

Mauritz

On 2018-03-05 11:04, Mauritz Sundell wrote:
> Hi Hendrik
> 
> On 2018-03-05 10:44, Hendrik Woltersdorf wrote:
>> Hi Mauritz,
>>
>> the table(s), where I see this error often, have:
>>
>> 1) a primary key index --> thats +1
> 
> No extra operations for primary key.
> Only secondary unique indexes should use extra operations.
> 
>> 2) no blobs
>> 3) 4) no foreign keys
>> 5) the primary key is one integer column
>> 6) the number of ldm threads should be 6 (MaxNoOfExecutionThreads=12) 
>> and ndb_desc shows 12 partitions on two datanodes in one node group
>>
>> We did not set MaxNoOfLocalOperations explicitely.
> 
> Ok, need to look elsewhere.
> 
> Can you provide more details of the statements of the failing transaction?
> 
> Mauritz
> 
>>
>> regards,
>> Hendrik Woltersdorf
>>
>> FinTech Group AG
>> Peter-Breuer-Str. 13
>> 08056 Zwickau
>> hendrik.woltersdorf@stripped
>> www.fintechgroup.com
>>
>>
>> FinTech Group AG, Frankfurt am Main - HRB Frankfurt 103516 - 
>> Gerichtsstand Frankfurt am Main - Vorstand: Frank Niehage (Vors.), 
>> Muhamad Said Chahrour - Aufsichtsratsvorsitzender: Martin Korbmacher - 
>> _www.fintechgroup.com_ <http://www.fintechgroup.com/>
>> Diese E-Mail enthält vertrauliche oder rechtlich geschützte 
>> Informationen und ist ausschließlich für den bezeichneten Adressaten 
>> bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser E-Mail oder 
>> dessen Vertreter sein sollten, informieren Sie bitte sofort den 
>> Absender und löschen diese E-Mail. Bitte beachten Sie, dass jede Form 
>> der Veröffentlichung, Vervielfältigung oder Weitergabe des Inhaltes 
>> dieser E-Mail unzulässig und nicht gestattet ist. Verfälschungen des 
>> ursprünglichen Inhaltes dieser Nachricht bei der Datenübertragung 
>> können nicht ausgeschlossen werden. E-Mails können manipuliert werden 
>> und Viren oder ähnliche Sicherheitsrisiken enthalten.
>> This e-mail contains confidential or legally protected information and 
>> is exclusively meant for the named addressee. In case you are not the 
>> intended addressee of this e-mail or its representative please inform 
>> the sender immediately and delete the e-mail. Please note that every 
>> form of publication, duplication or propagation of the content of this 
>> e-mail is forbidden and not allowed. Falsification of the original 
>> content of this message during the data transfer cannot be excluded. 
>> Furthermore, e-mails may contain viruses or similar security risks.
>>
>>
>> Inactive hide details for Mauritz Sundell ---05.03.2018 10:24:34---Hi 
>> Hendrik On 2018-03-05 09:03, Hendrik Woltersdorf wrote:Mauritz Sundell 
>> ---05.03.2018 10:24:34---Hi Hendrik On 2018-03-05 09:03, Hendrik 
>> Woltersdorf wrote:
>>
>> Von: Mauritz Sundell <mauritz.sundell@stripped>
>> An: Hendrik Woltersdorf <hendrik.woltersdorf@stripped>, 
>> cluster@stripped
>> Datum: 05.03.2018 10:24
>> Betreff: Re: Out of operation records in local data manager far below 
>> MaxNoOfConcurrentOperations
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> Hi Hendrik
>>
>> On 2018-03-05 09:03, Hendrik Woltersdorf wrote:
>>  > Hi,
>>  >
>>  > we run a MySQL-Cluster, currently version 7.4.15, where we get "Out
> of
>>  > operation records in local data manager" at approximately 55000 rows
>>  > updated or deleted in one transaction,
>>  > even though MaxNoOfConcurrentOperations=300000.
>>  >
>>  > Any ideas, why 300000 operation records are not enough for 55000
> rows?
>> Assuming there are no other concurrent transactions, this could be due
>> to extra implied load such as:
>>
>> 1) changes to unique indexes, each change takes an operation.
>>
>> 2) changes to blob columns, each change column can take one or more
>> operation depending  on size of value.
>>
>> 3) foreign keys causing lookups in parent table
>>
>> 4) foreign keys cascading into child tables
>>
>> Or it could be due to skew load due to:
>>
>> 5) Partial primary key used as partitioning key.
>>
>> 6) The number of partitions do not match number of LDM-threads.
>>      This should typically not happen unless explicitly expressed,
> or if
>> one have reconfigured cluster with more LDM-threads or adding nodes
>> without runnning reorganize partitions.
>>
>> One can also configure the number of operations records in local data
>> manager (LDM) with MaxNoOfLocalOperations.
>> By default that is slightly bigger than MaxNoOfConcurrentOperation.
>>
>> Mauritz
>>
>>
>>
> 
Thread
Out of operation records in local data manager far belowMaxNoOfConcurrentOperationsHendrik Woltersdorf5 Mar
  • Re: Out of operation records in local data manager far belowMaxNoOfConcurrentOperationsMauritz Sundell5 Mar
    • Antwort: Re: Out of operation records in local data manager far belowMaxNoOfConcurrentOperationsHendrik Woltersdorf5 Mar
      • Re: Antwort: Re: Out of operation records in local data manager farbelow MaxNoOfConcurrentOperationsMauritz Sundell5 Mar
        • Re: Antwort: Re: Out of operation records in local data manager farbelow MaxNoOfConcurrentOperationsMauritz Sundell5 Mar
    • Re: Out of operation records in local data manager far below MaxNoOfConcurrentOperationsTom Egan5 Mar
      • Antwort: Re: Out of operation records in local data manager far belowMaxNoOfConcurrentOperationsHendrik Woltersdorf5 Mar