On Mon, Sep 13, 1999 at 04:37:32PM -0700, d e l d h a n o a scribbled:
> Hello,
>
> I have a question concerning spiders and robots. Is a spider/robot a little
> program written in PHP which searches the web for sites and then enters them
> into a databse automatically?
>
Writing a robot using PHP would be a neat trick.
Spiders are written in many languages, usually C/C++ or even Perl.
> If so I would really like to learn how to program one. Does anyone have any
> tutorials/articles on this subject so I can learn?
>
> Also, does anyone know of any open source code for spiders/robots?
>
There are enough spiders in the world, please don't create another.
Yes, there are spiders out there that are open source. HT://Dig
(www.htdig.org) is one that comes to mind.
You can do a search on Yahoo for interesting web pages. One that I used
to visit often when I was researching these things is:
http://info.webcrawler.com/mak/projects/robots/robots.html
Chris...
--
What's the point? NT is not the solution.
NT is the problem.
Computers make very fast, very accurate mistakes.
PGP fingerprint: 063FCE320681C336 78C164FC9B2F91EA
