1. 

   mr_flea First Sergeant

   which search spider is this?

    

   mr_flea, Feb 8, 2004

   #1
2. 

   mr_flea First Sergeant

   Yep, that's the one. Which search engine does it serve?

    

   mr_flea, Feb 8, 2004

   #2
3. 

   DavidGP MajorGeeks Forum Administrator - Grand Pooh-Bah Staff Member

   http://www.robotstxt.org/wc/active/html/worm.html
   
   This search indexer is one of the firts and an oldie at that, from around 1994.

    

   DavidGP, Feb 9, 2004

   #3

Your name or email address:

Do you already have an account?

• No, create an account now.
• Yes, my password is:
• Forgot your password?

Stay logged in

---

Downloads All In One Tweaks \ Android \ Anti-Malware \ Anti-Virus \ Appearance \ Backup \ Browsers \ CD\DVD\Blu-Ray \ Covert Ops \ Drive Utilities \ Drivers \ Graphics \ Internet Tools \ Multimedia \ Networking \ Office Tools \ PC Games \ System Tools \ Mac/Apple/Ipad Downloads

Other News: Top Downloads \ News (Tech) \ Off Base (Other Websites News) \ Way Off Base (Offbeat Stories and Pics)

Social: Facebook \ YouTube \ Twitter \ Tumblr \ Pintrest \ RSS Feeds

---