On Mon, 22 Nov 2004 16:25:21 +0300, Konstantin Osipov
<konstantin@stripped> wrote:
> * Olaf van der Spek <olafvdspek@stripped> [04/11/20 14:39]:
>
> > > > > > The number of reads was about 50000, for 25000 rows.
> > > > > > Isn't it (much faster) to use reads of 1, 2 or 4 kbyte?
> > > > >
> > > > > This is a design flaw in the client-server protocol.
> > > > > Currently it performs 2 reads per row - first row length is
> read,
> > > > > and then the row itself.
> > > > > Note that it is only the client issue, the server buffers result
> sets
> > > > > and sends them in big chunks.
> > > > > I'm looking if it's possible to fix it with help of cursors.
> > > >
> > > > Can't this be fixed without a protocol change?
> > > > Just buffer the reads. Read 1 - 64 kbyte from the socket into a local
> > > > single/double/ring buffer and then read small blocks from that local
> > > > buffer.
> > >
> > > It can not without help of non-blocking IO, as the actual number of rows
> > > is never known. The server sends result set metadata, then rows, and then
> >
> > I'm not sure, but doesn't a blocking read/recv return with a partial
> > result if at least a single byte is available?
>
> That's true, at least for TCP/IP and UNIX sockets.
> But our IO is layered. The library itself works with NET layer, and
> calls net_read(buff, size) which will loop until 'size' bytes have been read.
> The NET library in turn will redirect net_read call to the real transport,
> which can be one of TCP/IP, unix sockets, shared memory (windows).
> I will look if it's possible to fix the NET layer to use buffered read
> for some transports (for example for TCP/IP).
> But the most robust solution is to fix the protocol so that we know in
> advance how many bytes are pending.
Are there any updates on this issue?
I've got another case where 150000 rows need to be read as fast as
possible, but I think the multitude of system calls is slowing it
down.
