Meta Tag Robots and X-Robots-Tag Header Guide

Parameter Meta robots tag X-robots-tag Type HTML meta tag HTTP header Scope Applies specifically to the HTML page it’s included in Applies to the HTTP response for diverse file types, including HTML, CSS, JavaScript, images, etc. Why you should use the meta robots tag and X-Robots-TagLet’s examine how the robots meta tag and the X-Robots-Tag help in search engine optimization and when you should use them. Meta robots directives and search engine compatibilityThe robots meta tags and X-Robots-Tag use the same directives to instruct search bots. For example:<meta name="googlebot" content="noindex, nofollow"> <meta name="googlebot-news" content="nofollow">The robots meta tag: syntax and utilizationAs we’ve said before, the robots meta tag is inserted into the page’s HTML code and contains information for search bots. Errors in configuring the robots meta tag and the X-Robots-Tag can lead to incorrect indexing issues and website performance problems.