List:General Discussion« Previous MessageNext Message »
From:Jacek Becla Date:March 30 2009 4:37pm
Subject:Re: dynamically splitting a table through one full table scan
View as plain text  
Hi Kabel

Yes, I did, it won't do the job for us. I didn't explain the whole 
usecase: we are dealing with a 50-billion row table which we want
to split into 1-million-row tables, and then dynamically break each
of these into smaller pieces in order to speed up n^2 near-neighbor
joins. If we partition the 1 million row table from the very beginning,
we will end up with unmanageable number of tables/files.

thanks,
Jacek



> Have you looked into MySQL partitioning?  If you're using version 5.1, 
> it might really help.. just partition the big table on chunk ID.
> 
> http://dev.mysql.com/doc/refman/5.1/en/partitioning.html
> 
> kabel

Thread
dynamically splitting a table through one full table scanJacek Becla28 Mar
  • Re: dynamically splitting a table through one full table scanChris W28 Mar
  • Re: dynamically splitting a table through one full table scankabel28 Mar
    • Re: dynamically splitting a table through one full table scanJacek Becla30 Mar