Search Engine Spider Web Crawler Bot concept.

You Don’t Need Robots.txt On Root Domain, Says Google

by


Google’s Gary Illyes shares an unconventional but valid method for centralizing robots.txt rules on CDNs.

  • Robots.txt files can be centralized on CDNs, not just root domains.
  • Websites can redirect robots.txt from main domain to CDN.
  • This unorthodox approach complies with updated standards.



Source link

Related posts:

  1. Google: Don’t Choose Cheap TLDs, Avoid Spam Risks
  2. Google: We Don’t Use Third-Party SEO Tool Scores For Rankings
  3. Domain History: Is It A Google Ranking Factor?
  4. Google On Topical Authority: Don’t Worry About It

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

We Know You Better!
Subscribe To Our Newsletter
Be the first to get latest updates and
exclusive content straight to your email inbox.
Yes, I want to receive updates
No Thanks!
close-link

Subscribe to our newsletter

Sign-up to get the latest marketing tips straight to your inbox.
SUBSCRIBE!
Give it a try, you can unsubscribe anytime.