The reptile program that fundamental tutorial searchs engine to optimize a foundation piece

" SEO of network sale tutorial " , chinese stationmaster stands ( is exclusive and online release.

If have,make him use, contact Zhang Dongwei of person of this book rights and interests please.

One, what is program of search engine reptile

Network reptile (be called again webpage spider, network robot, more often) of the person that call webpage angle, be one kind according to certain regulation, the program of automatic capture World-Wide-Web information or script. Additionally a few seldom use name still has ant, automatic index, simulator is vermian perhaps.

These handle capture of the network that be called or the spider is creeping. A lot of sites, search engine especially, use reptile to provide newest data, it basically is used at offerring it to had visited a carbon of the page, next, search engine can is opposite gotten page has index, in order to offer quick visit. The spider also can be used on Web execute a few missions automatically, check a link for example, affirm Html code; also can use information of some kind of specific type on capture webpage, for example capture email address (use at rubbish mail) normally.

Spider of a network is a kind of robot, or software acts as agent. Substantially, it begins from a group of URL links that want a visit, call these URL the seed. Reptile visits these links, its make out of these pages all exceeding link, add this URL list next, can call retrieve forward position. These URL are visited repeatedly according to particular strategy.

2, the sort of reptile program and resolution

The reptile process that searchs engine to expedite them goes content of website of visit, index, but because search engine,clique reptile program calls in website performance can be affected on certain level. In your server log file, see the method that visits every time and corresponding IP address, if be reptile program,call in, criterion User-agent can show Googlebot or the search index such as MSNBot props up reptile program name, every search engine has his User-agent.

3, home’s main reptile program


/ Search/spider.htm

Gu Ge

/ Bot.html




/ Help/webmaster/spider/


/ Webspider.htm

/ Soso-image-spider.htm

Search dog

/ Docs/help/webmasters.htm


/ Msnbot.htm