List:Internals« Previous MessageNext Message »
From:Michael Widenius Date:March 8 2009 12:29pm
Subject:Re: remote binary bulk inserts from background thread in MySQL
server
View as plain text  
Hi!

>>>>> "Sergei" == Sergei Golubchik <serg@stripped> writes:

Sergei> Hi, Andrej!
Sergei> On Mar 05, Andrej van der Zee wrote:
>> Hi,
>> 
>> I was wondering if it is possible to efficiently bulk-insert binary
>> rows in a remote MySQL server. In my setup, currently I collect
>> low-level resource data for each MySQL query. I modified the
>> server-code to read hardware counters and write the results in
>> (binary) bulk inserts into tables in the same database server. This is
>> done in a background thread. But, now I would like to  write the
>> binary data to a remote MySQL server as efficient as possible, i.e.
>> without (de)serializing the binary data to strings before it goes over
>> the wire. To make a long story short, is it possible to do binary bulk
>> inserts to a remote MySQL server from within a background thread in a
>> MySQL server? How could I do this?

Sergei> Either by using the client API and prepared statements - then you won't
Sergei> need to convert your binary data to strings, and you can bulk-insert many
Sergei> rows at once.

At this moment, the fastest way to do it is to write things to a file
on the client side and then issue a 'load data local infile' to read
the file.

If there are many rows, the binary to string to binary are convertes
are neglected compared to the effiency increase you get by using the
local infile protocol.

Regards,
Monty

Thread
remote binary bulk inserts from background thread in MySQL serverAndrej van der Zee5 Mar
  • Re: remote binary bulk inserts from background thread in MySQLserverSergei Golubchik6 Mar
    • Re: remote binary bulk inserts from background thread in MySQLserverMichael Widenius8 Mar
      • Re: remote binary bulk inserts from background thread in MySQL serverAndrej van der Zee23 Mar
        • Re: remote binary bulk inserts from background thread in MySQL serverSela Lerer23 Mar