MySQL Lists are EOL. Please join:

List:General Discussion« Previous MessageNext Message »
From:Marc Knoop Date:July 23 2004 6:41pm
Subject:Consolidating Data
View as plain text  

I have several (~12) web servers which all record web metrics to their own 
local mysql database.  I would like to consolidate data from each web server 
database to one master DB to within the hour.  I wish to avoid running 
multiple instances of mysql on the "master" server, so replication is not an 

What are the best practices for managing the consolidation of data?  Is it 
best to export the data on each web server and perform frequent bulk loads 
on the "master" server?  Or, is it better to have a robust Perl script on 
the "master" server that is responsible for pulling records from each web 
server?  I estimate 10,000 to 30,000 records per web server, per day with 
the average row size of 100 Bytes.  The web servers are all in remote 

The end goal is to have all web metrics available on *one* server from which 
a reporting server (M$ SQL server). 

Lastly, are there any experts on this list willing and available to code and 
document this, given more details, of course? 

Consolidating DataMarc Knoop23 Jul
  • Re: Consolidating DataSGreen23 Jul