List:NDB Connectors« Previous MessageNext Message »
From:Monty Taylor Date:May 24 2007 11:16pm
Subject:Re: NdbRecord
View as plain text  
Jim Dowling wrote:
> Hi Monty,
> Two points.
> Firstly, in NDB/J you need to define the NdbResultSet before you execute
> the transaction.
> NdbOperation foo = transaction.getNdbOperation("table");
> foo.readTuple(NdbTransaction.LockMode.LM_Read);
> foo.equal("id",12);
> // Notice - no recattrs returned here
> foo.getString("name");
> foo.getLong("age"); // people can be very old
> NdbResultSet results = foo.getResutSet();
> foo.execute(NdbTransaction.ExecType.Commit);
> while (
> {
> // results.getX();
> }

Indeed. One should also execute the transaction and not the operation,
which I think we both missed. :)

But thinking about this... shouldn't getResultSet really be a method on
the Transaction rather than the operation? That was you could call it
after the execute, which is where most people want to call it anyway,
and call it in a loop until it's null just like in normal JDBC...

> Second point. With Blobs in NDB/J, the most efficient way to read them
> is to have the programmer set the size of the Blob before reading it.
> Then the JNI code allocates a byte array of that size. This byte array
> is a native c++ buffer, that is then pinned in C++ code until the
> transaction that reads in the blob's contents (in the NdbRecAttr in
> C++). If you didn't pin the byte array in C++, the jvm could move around
> the java byte array in memory, and the transaction wouldn't be able to
> set the contents of the buffer for the blob.
> After the transaction commits, the buffer in C++ is unpinned, and the
> java byte array is accessed using blob.getData();
> NdbOperation foo = transaction.getNdbOperation("blob_table");
> foo.getBlob(*"data"*,blobSize);  // blobSize is known in advance
> NdbResultSet rs = foo.getResultSet();
> trans.execute(..);
> NdbBlob blob = rs.getBlob(*"data"*);
> *byte*[] buf = blob.getData();
> Anyway, you could probably provide something similar using ndbconnectors
> in Java. Not sure about other languages.

Yes, I'm doing essentially the same thing for all the connectors now ...
although I've got a bug in it right now that I need to track down. But I
have one that's more important in either the NdbClusterConnection or Ndb

> However, I agree that the Ndb/J API is a huge step forward in
> programmability from the C++ API (which is beyond the ken of 99% of
> programmers).

> Jim
> Monty Taylor wrote:
>> I actually did have a look at it. Actually, I think using it will be
>> much harder for the connectors - at least using that API. I suppose it
>> might be possible to use it as the implementation behind the current
>> API, but the current API is really much friendlier.
>> Other than the memory management aspects (which I understand, and which
>> might be why we might want to use it in some places) are there any
>> functionality bits we could benefit from that we can't do with the
>> current API?
>> Here's where it gets difficult. As I understand it, with the NdbRecord
>> API, if I want to fetch a varchar(32) and int and a datetime from a
>> table, I need to allocate a single, appropriately size buffer and hand
>> that to NdbRecord. The API will then populate the buffer, and I am then
>> free to read the contents of the buffer as I choose, and free it when
>> I'm done. Which is great, if I'm MySQL. But what if I'm writing from
>> Python? (Or even Java) What do I pass to that function? Do I just get
>> back a List of bytes? It's a sort of fascinating thing to think about,
>> because the answer is not as obvious to me as wrapping the more OO old
>> API. (And I hope, for this reason, you'll overlook my
>> stream-of-consciousness ramblings here)
>> So, I've been merging the NDB/J stuff this last week, and let me tell
>> you what would actually be really useful from the connectors point of
>> view - NDB/J supports the following:
>> NdbOperation foo = transaction.getNdbOperation("table");
>> foo.readTuple(NdbTransaction.LockMode.LM_Read);
>> foo.equal("id",12);
>> // Notice - no recattrs returned here
>> foo.getString("name");
>> foo.getLong("age"); // people can be very old
>> foo.execute(NdbTransaction.ExecType.Commit);
>> NdbResultSet results = foo.getResutSet();
>> which makes for good programming. Right now, I'm faking this in Java. My
>> getString and getLong methods are just wrappers around
>> NdbOperation::getvalue() which store the RecAttr in a
>> Map<String,NdbRecAttr> belonging to the NdbOperation. Of course, this is
>> sort of silly, since on the C++ level the NdbOperation object already
>> owns the memory for the RecAttrs and probably has a private list of
>> them. So if I could just have a method (or two) on NdbOperation that
>> would let me get the RecAttrs owned by the Operation, I could do all of
>> this ResultSet stuff in the mapping layer rather than in the Java
>> wrapper layer. If we had that, we could even implement something like
>> NdbResultSet in the API...
>> And thus we reach the quandry that we will always reach - needs of
>> Application programmers and System programmers. I'm totally on board
>> with all the non-memory-copying we get from NdbRecord and how wonderful
>> that is.
>> Stewart Smith wrote:
>>> had a look at it?
>>> it's pretty cool - should be very useful for connectors and ORM foo.
>>> in fact... possibly best just to use it. (is pushed to telco now)

Monty Taylor
Senior Consultant
MySQL Inc.,
Get More with MySQL!
NdbRecordStewart Smith23 May
  • Re: NdbRecordMonty Taylor23 May
    • Re: NdbRecordJim Dowling24 May
      • Re: NdbRecordMonty Taylor25 May