List:Internals« Previous MessageNext Message »
From:Loic Dachary Date:November 22 2000 12:43pm
Subject:Speedup inserts that involve updating fulltext index
View as plain text  
       Hi,

       I'm trying to help Sergei improve the performances of the fulltext
index updating. He suggested that I could try to maximize fulltext index
update speed in the case where a bunch of inserts are done on an existing
index. The basic idea is that it's much faster to insert already sorted
keys in the MYISAM index than to update it at random. The myisam/sort.c
function do a good job for that purpose when completely rebuilding the index.
The idea is to also take advantage of these functions when inserting rows.
    
       Here is a copy of the last mail I sent to Sergei. I'm not really 
asking for something specific to someone but rather hoping to get 
ideas or comments.

       Cheers,

---

       The INSERT DELAYED mechanism allows to group random inserts together
and a single function handles the actual insertion when appropriate. For now
I will assume this is good enough for our purpose.
       The problem is that even if grouped by INSERT DELAYED,
each insert implies an update of the fulltext index. I will therefore modify
the delayed_insert::handle_inserts function so that all fulltext index updates
are skipped when inserting the records. When all the records are inserted I
will call a modified version of find_all_keys that will work only on the 
previously inserted records.
       It occurs to me that we might as well do the same for all the indexes
types, no need to be selfish :-)

       As always, let me know if this sounds reasonable.

       Cheers,

-- 
Loic   Dachary         http://www.dachary.org/  loic@stripped
24 av Secretan         http://www.senga.org/      loic@stripped
75019    Paris         Tel: 33 1 42 45 09 16        loic@stripped
Thread
Speedup inserts that involve updating fulltext indexLoic Dachary22 Nov
Re: Speedup inserts that involve updating fulltext indexsasha23 Nov