Google explains the use cases for its different crawler types

Google has now added new details that explain the three categories its Google crawlers fall into, they include Googlebot, special-case crawlers and user-triggered fetchers. Types of Google crawlers. At the top of this Googlebot page, Google listed these three crawler types:Googlebot – The main crawler for Google’s search products. Special-case crawlers – Crawlers that perform specific functions (such as AdsBot), which may or may not respect robots.txt rules. This now explains how Google crawlers act, when they respect the robots.txt and how to identify them better.