Keyword research tool: build a semantic core

Collect, clean, cluster and grow a keyword pool per project: Yandex Wordstat demand, SERP clusters, exact frequency and competitor mining, with an offline plan before every paid call.

Try it

SpiderHead semantic core: the stage rail with free and paid stages and the offline plan of a paid stage.

Thirteen stages, one accumulating core

Each stage changes a status and records the reason, so a phrase is never deleted and every decision can be explained. Free stages sit between the paid ones, so you check the pool before you buy the next answer.

  • FreeFree, runs locally
  • PaidPaid provider call
  • WebPublic web pages, no provider quota
  1. Free

    Import

    Bring what you already have: a CSV or one sheet of an XLSX. Map query, frequency, group, subgroup, URL and comment, then add, update or replace. The preview shows new, existing and duplicate rows before anything is written.

    phrases
  2. Paid

    Collect · Yandex Wordstat

    Expands your seeds by region and re-expands strong phrases up to the depth you set. The queue is deduplicated, and --resume continues an interrupted run without asking for the same phrase again.

    phrases, expansions, provenance
  3. Free

    Clean

    Applies the project filters (intent, stop words, informational patterns), flags frequency outliers and groups phrases by lemma. A phrase with observed impressions or positions is never cut by frequency rules.

    status, route, lemma group
  4. Free

    Graph

    Builds a word graph of the pool. Homonym candidates go to review and candidate clusters land in anomalies.md, before you pay for anything else.

    anomalies.md
  5. Free

    Site match

    A phrase belongs in the core only when the site offers it. Phrases without any term from catalog.txt go to review as off-catalog.

    review list
  6. Paid

    Cluster · Yandex Search API

    Fetches one top 10 per phrase and caches it, so a repeated run pays only for new phrases. Phrases that share pages form a cluster; --method louvain regroups the cached results to break long chains.

    SERP cache, clusters, landing candidates
  7. Free

    Relevance

    If no site of your kind from vendors.txt appears in a phrase's top 10, the topic is probably off scope. The phrase goes to review; nothing is deleted.

    review list
  8. Free

    Competitors

    Reads the cached results: which domains hold your clusters and which words they use in titles. Candidates go to synonyms.txt.

    competitors.md, synonyms.txt
  9. Web

    Mine

    Opens the most frequent competitor pages from the collected results and reads their title, description, H1 and first H2 headings. Topical two- and three-word phrases go to synonyms.txt.

    competitor_mining.md, synonyms.txt
  10. Paid

    Synonyms · Yandex Wordstat

    Checks the candidates in synonyms.txt. Only phrases with demand join the core; then collect --resume grows the pool from them.

    checked phrases
  11. Paid

    Exact frequency · Arsenkin

    Buys the exact !W frequency once per lemma group, at the end, when the pool has stopped changing. Above the project's budget gate it needs --yes.

    exact frequency, task receipts
  12. Free

    Report and Excel

    report.md and clusters.csv for reading, an XLSX workbook (Summary, Core, Info, Clusters, Rejected) for the client, pool.csv and pool.json for other tools.

    report.md, XLSX, CSV, JSON
  13. Free

    Landing and insights

    After launch, pick each cluster's landing page from measured candidates, then read weekly cluster signals from your local Search Console, Webmaster and Metrika data and turn chosen ones into checklist tasks.

    landing, weekly signals, tasks

The loop that grows the core

The core is not collected once. Competitors and mining feed new candidates, synonyms checks them, collection resumes and clustering runs again. Exact frequency is bought last.

  1. collect
  2. clean · graph · site match
  3. cluster
  4. competitors · mine
  5. synonyms
  6. collect --resume
  7. exact, once
  8. report

Where the data comes from

The collection pipeline runs on Yandex data. Google demand, rank tracking and search analytics are separate toolkit commands that work next to the core.

Inside the pipeline

Paid

Yandex Wordstat

collect · synonyms

Base frequency, refinements and similar queries through the Yandex Cloud API. API key and folder ID; hourly quota.

Paid

Yandex Search API

cluster

One top 10 per phrase and region, cached with its fetch time and region. The async endpoint only.

Paid

Arsenkin

exact

The exact !W frequency the Wordstat API does not give. Billed at task creation; the task_id is kept, so a timed-out result is fetched again, not bought again.

Next to the core

Paid

DataForSEO

google-keywords

Google search volume, competition and seed expansion. Sandbox by default.

Paid

Wordstat one-offs

keywords-expand · keywords-seasonality · regions-tree

Expand one phrase, read its demand by month, browse the region tree (the tree is free).

Free

Topvisor

topvisor-read

Reads projects, keywords and recorded position history from your existing account. Paid checks cannot be launched from the toolkit.

Free

Search Console · Webmaster · Metrika

semantics insights

Weekly cluster signals from data the toolkit already synced locally. The insights step itself makes no provider calls.

Pay once, see the plan first

Every paid stage has an offline plan: the region, the engine, the query source and an upper bound of requests or tasks. No provider client is created by a plan.

A plan before every paid call

--dry-run --json prints the plan. The run itself needs --confirm-paid, even when the estimate is zero.

Receipts, not retries

Wordstat attempts, SERP fetches and Arsenkin tasks are reserved before the call. An ambiguous request is never resubmitted automatically.

Caches that stay

Cached SERPs and paid receipts survive a replace of the pool, so clustering again pays only for new phrases.

One spend journal

Each charge lands in spend-report with the semantic project and stage it belongs to.

When a provider price cannot be established offline, the plan says so instead of guessing a number.

Import and export

In

  • CSV or XLSX, one selected sheet; headers mapped automatically when unambiguous
  • Fields: query, frequency, group, subgroup, URL, comment, plus base, quoted, exact, impressions and Yandex/Google positions
  • A Category / Cluster path splits into group and subgroup
  • add keeps existing phrases, update applies the given fields, replace swaps the pool and is refused when empty
  • Bounded input: 32 MiB per file, 100,000 rows, 128 columns

Out

  • XLSX workbook: Summary, Core, Info, Clusters with competitors, Rejected
  • report.md digest and clusters.csv matrix
  • pool.csv and pool.json for other tools
  • Scope: the whole core, one cluster (--cluster) or clusters without a landing (--scope no-landing)
  • A dry run lists the exact files and row counts first

Real output from the CLI

Captured from the synthetic example project that ships with the toolkit. Its phrases are Russian, as Yandex demand is, and its frequencies are invented. No paid call was made.

Free stages, one after another
$ seohead semantics clean --project ./core
$ seohead semantics graph --project ./core
$ seohead semantics sitematch --project ./core
clean: kept=34 review=4 dropped=3 | lemma groups=41 (median base=720, outlier>36000)
graph: homonyms=0, to review=0, candidate clusters=7 -> anomalies.md
sitematch: catalog terms=11 | off-catalog->review=8 | kept and matching the site=26
The offline plan of a paid stage
$ seohead semantics cluster --project ./core --dry-run
"mode": "estimate", "stage": "cluster", "paid": true,
"provider": "yandex_search",
"request_estimate": {"unit": "requests", "count": 26, "kind": "upper_bound"},
"money_estimate": null,
"money_unavailable_reason": "provider pricing is not available from this offline preflight",
"paid_calls_made": 0
Where the core stands
$ seohead semantics status --project ./core
dropped=3 | kept=26 | review=12 | _total=41 | _exact=0 | _expanded=0 | _clusters=0
provider units: {'yandex_requests': 0, 'arsenkin': 0}
next: seohead semantics exact --project ./core
Export plan, nothing written
$ seohead semantics export --project ./core --scope no-landing --dry-run
"mode": "preview", "scope": "no-landing", "rows": 41,
"planned_files": ["core/semantics/pool.csv", "core/semantics/pool.json"]
bash
seohead semantics status --project ./my-core

Install first: installation guide. Every command also runs as an MCP tool for AI agents.

What you need

  • Three to ten seed phrases and the project filters in project.yaml
  • catalog.txt with the site's sections, vendors.txt with sites of your kind
  • Yandex Cloud API key and folder ID for Wordstat and the Search API
  • An Arsenkin token for exact frequency, only at the end
  • Optional: an existing keyword list as CSV or XLSX

Limits

  • Collection and clustering run on Yandex data; Google demand is the separate google-keywords command
  • Money is not estimated offline when the provider price is unknown; the plan says so
  • Landing candidates need a SERP clustering run and the project's own domain
  • Insights read only data already synced into the local sources database

Hand the core to an agent

Ten MCP tools expose the same stages and reads. A paid call returns the plan and needs confirm_paid=true to send work.

  • seo_semantics_statusCounters, provider units and the next stage
  • seo_semantics_runFree stages
  • seo_semantics_mineCompetitor pages, no provider quota
  • seo_semantics_paidPaid stages: plan first, then confirm_paid=true
  • seo_semantics_clustersPaged cluster list
  • seo_semantics_cluster_showOne cluster with its landing evidence
  • seo_semantics_cluster_queriesPhrases of a cluster with the retained top 10
  • seo_semantics_set_landingAssign or clear a landing page
  • seo_semantics_insightsWeekly signals by cluster and source
  • seo_semantics_insights_tasksChosen signals into the checklist

Questions

Does it collect Google keywords?
Not in the collection pipeline: it runs on Yandex Wordstat, the Yandex Search API and Arsenkin. Google search volume and seed expansion come from the separate google-keywords command through DataForSEO, and you can import Google positions into the core as a column.
Which stages cost money?
collect and synonyms (Yandex Wordstat), cluster for phrases without a cached top 10 (Yandex Search API) and exact (Arsenkin). mine reads public competitor pages and uses no provider quota. Everything else is free and runs locally.
Can a run charge me twice?
The toolkit is built so it does not. Paid requests are reserved before the call, an interrupted request is not resubmitted automatically, a paid Arsenkin result is fetched again by its task_id, and cached results stay when you replace the pool.
Do phrases get deleted?
No. A stage changes a phrase's status (kept, review, dropped and so on) and records the reason and the stage. You can always see why a phrase left the core and bring it back.
Can I start from an existing keyword list?
Yes. Import a CSV or an XLSX sheet with your own columns mapped to query, frequency, group, subgroup, URL and comment. Positions and impressions you import protect a phrase from frequency cuts.
How is the landing page for a cluster chosen?
After a SERP clustering run, and only when the project's own domain is known, each candidate URL is scored by the share of cluster phrases whose top 10 contained it. You pick the landing; the choice is written with an audit record.

Try it on your own site