Spiders a site and creates output files containing (1) a list of URLs in the site, (2) a Google sitemap, (3) an XML summary file with title, description, and keywords for each page, and (4) links to URLs external to the spidered domain. Robots.txt compliant.
Tags: site spider, xml file, output file, title description, google sitemap |
|||||||||||||||||
Copyright © 2005-2006, Exefind.com