List:Cluster« Previous MessageNext Message »
From:Graeme Wallace Date:November 26 2012 5:49pm
Subject:Re: Problem with multithreading and ClusterJ
View as plain text  
Ok tried reducing transaction size down to 4K rows. Works for 2 threads,
but its slower than doing one thread with larger transactions.

If i up the threads to 4, i get the same error as before

Nov 26, 2012 12:44:24 PM com.mysql.clusterj.tie.Utility throwError
SEVERE: Error in NdbJTie: returnCode -1, code 266, mysqlCode 146, status 1,
classification 10, message Time-out in NDB, probably caused by deadlock .


Am I just going about this wrong ? I assumed that as i have a cluster of 10
datanodes i would be able to have lots of threads and therefore lots of
writes going on at the same time. (Caveat - i read the marketing material
for 1 billion reads and writes a second :) )

regards,



Graeme



On Mon, Nov 26, 2012 at 8:44 AM, Magnus Blåudd
<magnus.blaudd@stripped>wrote:

> On Mon 26 Nov 2012 04:33:57 PM CET, Graeme Wallace wrote:
>
>> I've tried playing around with transaction size, but the error still
>> remains even if I make the transaction small ie 32K rows AND i make all
>> the
>> primary keys unique - so that each transaction should have a unique set of
>> keys.
>>
>> G.
>>
>>
>> On Mon, Nov 26, 2012 at 8:30 AM, Magnus Blåudd
> <magnus.blaudd@stripped>
>> **wrote:
>>
>>  On Mon 26 Nov 2012 04:09:42 PM CET, Graeme Wallace wrote:
>>>
>>>  I'm trying to make my app multi-threaded and running into
>>>> problems.
>>>>
>>>> I have a Session local to each thread that is attempting to write to the
>>>> db. I'm batching up many savePersistent() calls in a transaction - but
>>>> when
>>>> the transaction commits I invariably end up with
>>>>
>>>> Nov 21, 2012 7:22:29 PM com.mysql.clusterj.tie.Utility throwError
>>>> SEVERE: Error in NdbJTie: returnCode -1, code 266, mysqlCode 146, status
>>>> 1,
>>>> classification 10, message Time-out in NDB, probably caused by deadlock
>>>> .
>>>>
>>>> For reading the docs, it looks like there shouldn't be table locking
>>>> going
>>>> on - so i dont understand what resource is being held by one thread that
>>>> stops the others from being able to write at the same time.
>>>>
>>>> Any clues would be most helpful,
>>>>
>>>> regards,
>>>>
>>>>
>>>> Graeme
>>>>
>>>>
>>>>  Check out Matthew's reply in in this forum thread
>>>
> http://forums.mysql.com/read.****php?25,505712,505757<http://forums.mysql.com/read.**php?25,505712,505757>
>>>
> <http://**forums.mysql.com/read.php?25,**505712,505757<http://forums.mysql.com/read.php?25,505712,505757>
>>> >
>>>
>>> "Due to the distributed nature of NDBCLUSTER there is no automatic
>>> deadlock detection mechanism within the NDBCLUSTER engine. The only
>>> indication that the cluster gives that there is a *possible* deadlock is
>>> the "Lock wait timeout exceeded". This error occurs when a given lock
>>> operation takes longer than TransactionDeadlockDetectionTi****meout
>>> (default 1200 ms.). If you have large transactions that lock many rows at
>>> once or if you have long running transactions these can prevent this or
>>> transactions from completing quickly. Try to avoid large or long running
>>> transactions by breaking them into smaller chunks. The
>>> TransactionDeadlockDetectionTi****meout can also be reached when a node
>>> is
>>> overloaded but has not yet been ejected from the cluster for missing
>>> heartbeats longer than 4xHeartbeatIntervalDbDb (default 1500 ms. each). "
>>>
>>> / Magnus
>>>
>>>
>>
>>
>>
> Give it a try with even smaller transactions if possible. Normally better
> to use more threads with smaller transactions.
>
> / Magnus
>



-- 
Graeme Wallace
CTO
FareCompare.com
O: 972 588 1414
M: 214 681 9018

Thread
Problem with multithreading and ClusterJGraeme Wallace26 Nov
  • Re: Problem with multithreading and ClusterJMagnus Blåudd26 Nov
    • Re: Problem with multithreading and ClusterJGraeme Wallace26 Nov
      • Re: Problem with multithreading and ClusterJMagnus Blåudd26 Nov
        • Re: Problem with multithreading and ClusterJGraeme Wallace26 Nov
          • Re: Problem with multithreading and ClusterJMagnus Blåudd27 Nov
            • Re: Problem with multithreading and ClusterJGraeme Wallace27 Nov
              • Re: Problem with multithreading and ClusterJCraig L Russell27 Nov