Home About us Services SEO Plans Contact us Articles FAQ Link




Extractor Tools

Search Engine Marketing >> Articles >> Yahoo Articles >> Yahoo Slurp

Yahoo Slurp

Yahoo! Slurp, A Yahoo!'s web-indexing robot. The Yahoo! Slurp crawler gathers documents from the Web to put up a searchable index for search services using the Yahoo! search engine. These documents are discovered and crawled as other web pages contain links directing to these documents.

As part of the crawling effort, the Yahoo! Slurp crawler will take robots.txt standards into account to make sure we do not crawl and index those pages that you would not like to have returned via Yahoo! Search Technology. If a page is not allowed to be crawled by robots.txt standards then it is neither considered for inclusion nor placed in the search engine's database.

It follows HREF link and it does not follow SRC links. This means that Yahoo! Slurp will not retrieve or else index individual frames referred to by SRC links.

Yahoo! Slurp has support for frames also makes an attempt to crawl complex URLs such as those generated by forms, content generation systems, also dynamic page generation software.

How Can I Prevent Yahoo Slurp From Subsequent Links From A Particular Page Or Archiving A Copy Of A Page?

It obeys the noindex meta-tag. If you place:
<META NAME="robots" CONTENT="noindex">
-in the head of your web document, Yahoo! Slurp will get back the document, but it will not index the document or position it in the search engine's database.

<META NAME="robots" CONTENT="noindex">

It will get back the document, but it will not index the document.

<META NAME="robots" CONTENT="nofollow">

It will not follow any links that are present on the page to other documents.

<META NAME="robots" CONTENT="noarchive">

Yahoo maintains a cache of all the documents that we fetch, to allow our users to access the content that we indexed (in the event that the original host of the content is beyond your reach, or the content has changed). If you do not desire us to archive a document from your site, you can position this tag in the head of the document, and Yahoo will not give an archive copy (Cache) for the document.

Yahoo Slurp will not just indexes the title and meta tags, but also the full text of webpages. So including quality content in the webpage is as significant as including keywords in the title and meta tags.

Yahoo searches for pages and as soon as it finds a page with the required keyword, it lists the page in its SERPs. The position of the page depends on the content. But there are some chances for a page with the required keyword being left out owing to deprived content or because Yahoo could not find the page.

How to attract Yahoo slurp to crawl the website?

There are three ways you attract the Yahoo Crawler in crawling the site:

1. Get links from sites that are frequently crawled by the Yahoo Robot, if that is done Yahoo regular visits the site and crawls it, Regular Yahoo visit is a good sign and helps lots of getting good Ranking.

2. As yahoo says you can trigger the Yahoo Robot by browsing a website by the Yahoo companion toolbar and Yahoo says this will trigger the Yahoo slurp Bot.

3. Through the notorious PFI/PPC program sitematch. This type of inclusion guarantee's an inclusion into the Yahoo index, so no trouble of using it.

Yahoo Webrank



SEO client speaks