Meta tags are HTML elements in a page’s head that can describe the page, control crawler behavior or supply social-preview metadata, depending on the tag.
Meta tags have different jobs
Meta tags are machine-readable HTML elements in a page head. Some describe a page, some direct crawlers, and some provide social-preview data. They should be selected by function: a robots directive controls indexation eligibility, a meta description may be used as a search snippet, and social metadata supports link previews. None turns thin or unavailable content into a useful search result.
The title element is related but is not itself a meta tag. Search engines can generate title links from several visible and structured signals, so a title and heading should agree with the page rather than repeat a phrase mechanically. Meta descriptions are suggestions for snippets; Google can use other visible text when it better matches the query.
Robots directives and snippets
| Element | Primary function | Test |
|---|---|---|
| meta robots noindex | Tell crawlers not to index an HTML page | It is present in the crawled HTML and the page is crawlable |
| X-Robots-Tag | Send a robots directive in HTTP headers | Response header is returned for the intended URL |
| meta description | Suggest page-summary text | It accurately reflects visible content |
| Open Graph metadata | Supply a social preview | A debugger sees the final canonical page values |
For a PDF or image, X-Robots-Tag is often the applicable directive because there is no HTML head. For an HTML page, meta robots and header directives should not contradict each other. A page blocked in robots.txt may not be crawled to see a noindex rule, so blocking and deindexing are separate decisions.
Inspect the response
- Fetch the canonical URL and inspect both HTML and response headers.
- Check title, description, canonical and robots values against the visible page purpose.
- Test alternate hosts, trailing slash variants and filtered URLs for contradicting directives.
- Use URL Inspection as later crawler evidence, not as the only implementation test.
- Recheck after caching or deployment changes, because templates can overwrite page-level values.
Example: a paginated category may have a useful title and description for its current products but should not inherit a noindex from a filter template. Compare the actual response, the canonical target and sitemap inclusion before changing a tag.
Conflicting signals
Common defects are a 200 page with an error message, a canonical to a redirected URL, a noindex page placed in a sitemap, and a generic description copied across hundreds of pages. The remedy is not more tags; it is one intentional state across HTTP response, visible content, canonical, robots directive, internal links and sitemap.