What is Spider Crawler?
Spider crawler allows Google and other spider robots to efficiently walk through your site by following the trail of links that spider crawler generates for the spiders to follow. New and improved to increase the keyword ranking based upon your menu elements.
Using our exclusive BDPDT technology, we support any and all versions of DNN with the same code base of SpiderCrawler. Feel confident that no matter what version of DNN you are running on, you can always run the latest version of our software.
What makes Spider Crawler different than a Site Map Module?
- We have taken into account Google's Page Ranking Algorithm to come up with the best way of producing the links to maximize your Page Rank to whatever page you wish it to go to.
- Optimized URL re-writing using /Portal Name/Tab Name/ format.
- We allow you to optionally display the links to a spider (parsed by UserAgent), and also provide a url without the parameters with them (using DNN's url rewriter logic)
- We allow you to optionally choose pages not to link to - pages that might detrimentally affect your overall site Page Rank - such as product pages with many outbound links will reduce your Page Rank - if there is no reciprocal link back to you.