From: Mika Raento
Date: March 23 2007 11:32am
Subject: Re: Integrating NDB async transactions with other event loops
List-Archive: http://lists.mysql.com/ndb-connectors/23
Message-Id: <4603BAD8.1090107@iki.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Monty Taylor wrote:

> 
> Got a good reply on this from Jonas: 
> 
> """
> Using ndb async transactions and select/(e)poll does not currently work.
> We dont provide a fd to use.
> And communication between ndbapi/ndbd is multi socket and multiple
> threads also
>   multiplex over same socket(s).
> 
> Adding to this, transport mechanism is configurable (tcp,shm,sci etc)
> and is
>   transparent to ndbapi programs, as well as *with-in* ndbapi.
> 
> Conclusion: Using ndb async transaction is select/(e)poll loop
>   is not possible wo/ adding that support explicitly to ndbapi...
> 
>   which btw is a cool idea...
> """

Thanks. That confirms my understanding of the basic transport mechanism. 
  I haven't looked _that_ deeply into it, but one option would be to 
have a single fd (a pipe) that NDB writes a single byte to indicate that 
now would be a good time to call sendPollNdb.

What I'm still looking into is how the threads are used. Basically I can 
work around the missing event support by calling sendPollNdb(0) with a 
reasonable interval, but should that or some other function do blocking 
IO in the same thread, that'll be a bad thing. I have a proof-of-concept 
of that working quite nicely with Danga::Socket.

Of course _totally_ async programming is hard anyway, as you tend to 
still have things like opening files which can block, but as long as the 
probability of blocking is low enough I can live with that. E.g., if 
sendPollNdb does some sync/blocking writes on a socket, that's not 
great, but if that socket very rarely blocks in practice, it'll be good 
enough.

The other thing that I'm still worried about is signal handling: for 
example I'm of course ignoring SIGPIPE in the async event loop and 
handling the return values from read/write instead. If the NDB code 
installs any signal handlers, that may be a problem.

Cheers,
    Mika