At 02:34 PM 6/9/2004, you wrote:
>I need some help please! I have 60GB of proxy logs to parse and load into
>a mysql database. I've written a parsing script but I'm stuck now on how
>to load the data in.
>I have a database called PROXY_LOG with 2 tables:
>user_id date_time url_id size
>url_id url category
>The data values I have for each record are:
>user_id date_time size url category
>u752359 2004-04-02 12:33:04 3403 http://www.mysql.com/index.html
>The problem is I'm not sure how to load the data into the 2 tables and set
>the url_id. Could anyone point me in the right direction please?
Is this a one time operation? If so, you could create a temporary
table with all the fields:
>user_id date_time url_id size url category
and use Load Data InFile to load the data into this one table.
Then create the two smaller tables using:
create table USER_TABLE select user_id, date_time, url_id, size from
create table URL_TABLE select url_id, url, category from BIG_TABLE;
drop table BIG_TABLE;
So there you have it in 3 lines of code. :-)
P.S. If you could break the text file into two files to begin with, then
two Load Data InFile statements would handle things nicely.