Let's take a look at these attributes in more detail.
Name
The name attribute controls whether crawlers and bots (user agents, also known as UAs) follow the instructions in.
To tell all crawlers to follow the instructions, use:
name="robots"
In most scenarios, you'll want to use this as the default, but tunisia mobile database you can use as many different meta robots tags as needed to specify instructions for different crawlers.
To give instructions to different crawlers, it is simply a matter of using several tags:
<meta name="googlebot" content="noindex">
<meta name="googlebot-news" content="nosnippet">
There are hundreds of different user agents. The most common ones are:
Google: Googlebot (you can see a complete list of Google crawlers here )
Bing: Bingbot (you can see a complete list of Google crawlers here )
DuckDuckGo: DuckDuckBot
Baidu: Baiduspider
Yandex: YandexBot.