Word

Description

Crawler

a program that searches the Internet seeking publicly accessible information and Web documents. The information might be stored in a database and made available through a search engine. Crawlers are also known as spiders, bots and wanderers. see spider.