MySQL Lists are EOL. Please join:

List:General Discussion« Previous MessageNext Message »
From:Steve Rapaport Date:February 8 2002 7:47pm
Subject:Re: Distributed Fulltext?
View as plain text  
Also, I have to ask the question:  

Why is it that Altavista can index terabytes overnight and return 
a fulltext boolean for the WHOLE WEB 
within a second, and Mysql takes so long? 

On Friday 08 February 2002 11:50, Steve Rapaport wrote:
> I second the question.  It could also reduce the size of the
> fulltext index and the time taken to update it.
>
> -steve
>
> > On Thursday 07 February 2002 20:53, Brian  wrote:
> > > Has anyone made a suggestion or thought about ways to distribute
> > > databases which focus on fulltext indexes?
> > >
> > > fulltext indexes do a good job of indexing a moderate amount of data,
> > > but when you get a lot of data to be indexed, the queries slow down
> > > significantly.
> > >
> > > I have an example table, with about 90 million rows.. and has a
> > > fulltext index on a varchar(100) field. A single-word query which would
> > > return approx 300k results takes an average of 15 seconds. A query with
> > > smaller results (~ 10k) can be as quick as 1 sec.. which I would
> > > consider acceptable.
> > >
> > > Has any thought about splitting the data into distributed files or even
> > > machines? ie: something as simple as 'words' starting with 'X' are
> > > split into a-h, i-p, q-z... or something more advanced? (maybe mysqld
> > > could automatically split results based on (#results per unique 'word'
> > > / desired # of 'split files/machines') Would such a system give any
> > > advantages to searching speed and concurrenct query scalability? I
> > > haven't looked at the fulltext internals.. so I don't know if such
> > > "query routing" could take place or not.
> > >
> > > If nothing else, does anyone else have experience with a table of this
> > > size or even larger? What kind of tuning have you done?
>
> ---------------------------------------------------------------------
> Before posting, please check:
>    http://www.mysql.com/manual.php   (the manual)
>    http://lists.mysql.com/           (the list archive)
>
> To request this thread, e-mail <mysql-thread98891@stripped>
> To unsubscribe, e-mail
> <mysql-unsubscribe-steve=petabit.com@stripped> Trouble
> unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

---------------------------------------------------------------------
Before posting, please check:
   http://www.mysql.com/manual.php   (the manual)
   http://lists.mysql.com/           (the list archive)

To request this thread, e-mail <mysql-thread98923@stripped>
To unsubscribe, e-mail <mysql-unsubscribe-cyon=bestweb.net@stripped>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

Thread
Distributed Fulltext?Brian DeFeyter12 Feb
Re: Distributed Fulltext?Steve Rapaport12 Feb
Re: Distributed Fulltext?Brian DeFeyter12 Feb
Re: Distributed Fulltext?Steve Rapaport12 Feb
Re: Distributed Fulltext?Brian DeFeyter12 Feb
Re: Distributed Fulltext?Alex Aulbach12 Feb
Re: Distributed Fulltext?James Montebello12 Feb
Re: Distributed Fulltext?George M. Ellenburg12 Feb
Re: Distributed Fulltext?Steve Rapaport12 Feb
  • Re: Distributed Fulltext?Mike Wexler12 Feb
Re: Distributed Fulltext?alec.cawley12 Feb
Re: Distributed Fulltext?Steve Rapaport12 Feb
  • Re: Distributed Fulltext?Mike Wexler13 Feb
    • Re: Distributed Fulltext?hooker14 Feb
Re: Distributed Fulltext?Steve Rapaport12 Feb