List:Cluster« Previous MessageNext Message »
From:Graeme Wallace Date:November 22 2012 3:22pm
Subject:Re: Help with ClusterJ
View as plain text  
For some reason i had to recreate my table to get this to work. So now it
persists both new and existing records without the need for session.find().

However, I'm now trying to make my app multi-threaded and running into
problems.

I have a Session local to each thread that is attempting to write to the
db. I'm batching up many savePersistent() calls in a transaction - but when
the transaction commits I invariably end up with

Nov 21, 2012 7:22:29 PM com.mysql.clusterj.tie.Utility throwError
SEVERE: Error in NdbJTie: returnCode -1, code 266, mysqlCode 146, status 1,
classification 10, message Time-out in NDB, probably caused by deadlock .

For reading the docs, it looks like there shouldn't be table locking going
on - so i dont understand what resource is being held by one thread that
stops the others from being able to write at the same time.

Any clues would be most helpful,

regards,


Graeme



On Thu, Nov 22, 2012 at 12:25 AM, Magnus Blåudd
<magnus.blaudd@stripped>wrote:

> On 11/21/2012 10:12 PM, Graeme Wallace wrote:
>
>> I have a stream of data that i'm trying to persist into a cluster using
>> ClusterJ and I'm running into problems.
>>
>> For each ASCII record that i receive, I'm parsing into Java primitives
>> then
>> setting the fields on a Java representation that i've gotten via
>> Session.newInstance()
>>
>> what i would really like to do then is to call
>>
>> session.savePersistent()
>>
>> so that if the primary key already exists in the db the row gets updated,
>> and if the primary key doesnt exist a new row gets created.
>>
>>
> Sounds like "INSERT INTO vs REPLACE INTO in SQL" or "insertTuple() vs
> writeTuple() in NdbApi". Seems like ClusterJ which is implemented on top of
> NdbApi should be able to do the same thing(if instructed to).
>
>
> It actually looks like ClusterJ is using writeTuple, check out this part
> of SessionImpl.java:
>
>      /** Save the instance even if it does not exist.
>>      * @param instance the instance to save
>>      */
>>     public <T> T savePersistent(T instance) {
>>
>
>
>
> / Magnus
>
>
>  Unfortunately, this doesnt work as I get an exception back if the primary
>> key already exists.
>>
>> Is there a way around this without having to do a session.find() on the
>> primary key and then updating the fields on the object returned ?
>>
>> The volume of data that I'm trying to insert is quite large (hundreds of
>> millions of records) and this would seem like a significant overhead if i
>> had to do a session.find() for every record.
>>
>>
>>
>


-- 
Graeme Wallace
CTO
FareCompare.com
O: 972 588 1414
M: 214 681 9018

Thread
Help with ClusterJGraeme Wallace21 Nov
  • Re: Help with ClusterJMagnus BlĂ„udd22 Nov
    • Re: Help with ClusterJGraeme Wallace22 Nov