Claude Keyword Clustering: Automate Intent & Cleanup

The complete AI workflow. The API stack that powers it, the methodology that shapes it, and the step by step process that runs it. Including the parts that still require your judgment.

Keyword clustering is one of the most time consuming tasks in SEO. It is also one of the most broken when you try to hand it to AI without a proper system behind it.

I have spent the last several months building an AI assisted SEO workflow using Claude and Cursor. Keyword clustering was the first workflow I tackled deliberately. In this guide I am going to show you the complete system. The API stack that powers it, the methodology that shapes it, and the step by step process that runs it. This includes the parts that still require your judgment, because any guide that pretends AI eliminates all of that is not being honest with you.

By the end you will have the exact setup, the API connections, the four prompts, and the review method to run this yourself on a real keyword list.

01What Fellow SEOs Say About Keyword Clustering

The honest version.

Before I explain the system, I want to share what actually motivated building it.

Over the past year I have had conversations with fellow SEO professionals. Agency owners, in house strategists, freelancers. We talked about where their workflows break down. Keyword clustering came up consistently as one of the highest effort, lowest satisfaction tasks in the toolkit. Not because it is technically difficult, but because it is cognitively exhausting in a way that scales linearly with the size of the keyword list.

Here is what they told me, in their own terms.

I still have to manually eye ball the final sheet every time. The tools group by surface similarity. A human has to verify whether a cluster actually represents one searcher need or two.

I need to review intent manually no matter what tool I am using. The tool handles the volume. The human handles the meaning.

The tools dump thousands of errors or groupings with zero context. Raw clustering output without prioritisation or business context is noise dressed up as structure.

When two tools give me different clusters for the same list, I have to go back to the SERPs to decide who is right. Tool disagreement is not an edge case. It is standard operating procedure.

The pre clustering work is never automated. Competitor research, brainstorming topic ideas the tools would never surface, initial manual grouping. The clustering tool enters the picture after significant human work has already happened.

Sensitivity tuning kills me. Running the tool, deciding the output is too tight or too loose, adjusting, re running. Iterative judgment masquerading as a technical process.

The pattern across every conversation: tools handle the volume and the pattern matching. Humans handle the meaning. The tools save the easy 70 percent and leave the exhausting 30 percent. That 30 percent is the part that requires the most focused attention of anything in the workflow.

02The Methodology. Principles Behind the System

Before I show you how to build this, I want to explain the methodology that shaped it. These are the principles I applied when designing the workflow. Understanding them matters because they determine why the system is built the way it is. They will help you adapt it when something does not work the way you expect.

03The API Stack

The system uses three external data sources. Each one serves a different checkpoint in the workflow. For my full stack of recommended SEO software and platforms beyond what is listed here, see my tools recommendation page.

{
  "mcpServers": {
    "dataforseo": {
      "command": "npx",
      "args": ["-y", "@dataforseo/mcp_server"],
      "env": {
        "DATAFORSEO_LOGIN": "your_email",
        "DATAFORSEO_PASSWORD": "your_api_password"
      }
    }
  }
}

Create this folder structure on your computer:

seo workspace/
├── CLAUDE.md          ← your SOP, read every session
├── data/              ← keyword data lives here
└── briefs/            ← all outputs saved here automatically

Open Cursor → File → Open Folder → select seo workspace.

Create a new file called CLAUDE.md directly in seo workspace. This is the file Claude reads at the start of every session. It encodes your identity, your methodology, your Quality Gate rules, and the workflow instructions.

Paste the following and edit the parts in brackets:

# CLAUDE.md, SEO Workspace

## Who I Am
I am an experienced SEO professional, not a developer, 
not a beginner. I understand search strategy, entities, 
intent, and content deeply.
Do NOT explain basic SEO concepts to me.
I am in a learning phase. Show your reasoning at each 
step and flag uncertainty visibly.

## System Mandate
Two jobs on every task simultaneously:
Force Multiplier, eliminate manual repetitive work.
Quality Gate, catch expert mistakes before they ship.
Human in the loop. Always. Nothing is auto published.

## The Honest 70/30 Split
You handle the 70 percent: volume processing, semantic 
grouping, SERP overlap checking, data cross referencing.
I handle the 30 percent: intent judgment, business context 
decisions, publishing decisions, entity relevance review.
Your job is to make my 30 percent faster and more 
structured. Not to pretend it does not exist.

## Co Relevance Supervision Rule
You map entity attributes at scale I cannot match.
But statistical co occurrence is not genuine relevance.
Flag any entity or keyword grouping where correlation 
to the core topic is uncertain. Label: REVIEW NEEDED.

## API Connections Available
GSC API: live ranking data, position, impressions
Ahrefs/Semrush API: volume, competitor keywords, gaps
DataForSEO SERP API: live SERP overlap, SERP features

## Keyword Clustering Methodology

### Core Rule: 40 to 60 percent SERP Overlap
Use DataForSEO SERP API to check live top 10 URLs.
4 to 6 shared URLs = same cluster (one article)
Under 4 shared URLs = separate clusters
Flag unverified overlaps as ⚠️ SERP CHECK NEEDED

### JTBD Lens, Apply Before Every Grouping
Before clustering any keyword, ask:
What functional struggle is the user trying to solve?
Syntax: verb + object + clarifier

### The 8 Judgment Checkpoints
CP1: Mixed intent detected → ⚠️ INTENT REVIEW NEEDED
CP2: Suggest the split, never split automatically
CP3: SERP overlap unconfirmed → ⚠️ SERP CHECK NEEDED  
CP4: Overlapping clusters → ⚠️ POSSIBLE DUPLICATE
CP5: After first output, report if sensitivity needs 
     adjusting and recommend re run parameters
CP6: Before clustering 50+ keyword lists, ask:
     (1) competitor keywords added?
     (2) community research keywords added?
     (3) initial grouping done?
CP7: Recommend Own Page / Merge / Kill with confidence
CP8: Check GSC API data, Path A or Path B decision

### Ranking Decision Tree
PATH A: not ranking → evaluate and recommend new article
PATH B ranking + intent matches → optimise existing page
PATH B ranking + intent mismatches → new article needed
NEVER recommend new page without checking existing 
coverage first. Cannibalisation is the expert mistake.

## Output Rules
Save all outputs to /briefs/
Naming: YYYY MM DD clustering [topic].md
Use mandatory table format on every output
End every output with CONFIDENCE & FLAGS section

Open Cursor chat (Ctrl+L Windows / Cmd+L Mac). Confirm Claude is selected, not GPT 4. Send each prompt and wait for the response before sending the next.

Read my CLAUDE.md and confirm you understand:
1. Who I am and how to treat me
2. The Force Multiplier and Quality Gate mandate
3. The 70/30 split and what each side handles
4. The keyword clustering methodology and 8 checkpoints
5. Which APIs are connected and what each one does
6. The JTBD lens and how to apply it before grouping

Tell me what you know before we start.

Claude should describe your SEO experience level, the Force Multiplier and Quality Gate system, the 40 to 60 percent overlap rule, all 8 checkpoints, the three API connections, and the JTBD syntax. If it misses anything, correct it before proceeding.

Context for this clustering session:

SITE: [your domain]
NICHE: [your niche]
TOPIC FOCUS: [e.g. "NHS car discount schemes"]
AUDIENCE: [who you are writing for]

BUSINESS CONTEXT (critical, read before grouping):
[List any terms that need special handling:
Brand names that look like generic terms,
Eligibility or informational terms you rank for
but that are upstream of your product,
Existing tool pages or calculator pages,
Terms where audience intent is specific to your 
business model]

DATA SOURCES TO USE:
Pull live ranking data from GSC API for this domain
Pull competitor keyword gaps from Ahrefs/Semrush API
against these competitors: [list 2 to 3 competitors]
Use DataForSEO SERP API for overlap verification

Before clustering, run Checkpoint 6. Ask me the 
three pre clustering input quality questions.

Answer Claude’s three questions honestly. If the input list is incomplete, fix it before running. Weak input into a good methodology still produces weak output.

Now run the full keyword clustering workflow.
Follow the 6 step methodology and all 8 checkpoints 
from CLAUDE.md exactly.

Step 1: Read keyword data and identify topic structure
Step 2: Apply 40 to 60 percent SERP overlap rule via 
        DataForSEO
Step 3: Apply JTBD lens per cluster. What is the 
        functional struggle being solved?
Step 4: Check SERP features via DataForSEO
Step 5: Recommend Own Page / Merge / Kill per cluster
Step 6: Apply Ranking Decision Tree using GSC data

Requirements:
Flag ⚠️ INTENT REVIEW NEEDED on every mixed cluster
Flag ⚠️ SERP CHECK NEEDED on every unconfirmed overlap
Flag ⚠️ POSSIBLE DUPLICATE on overlapping clusters
Apply JTBD syntax to each cluster's intent column
Use the mandatory table format from CLAUDE.md
End with full CONFIDENCE & FLAGS section
Save to: /briefs/2026 clustering [topic].md

Show reasoning on uncertain clusters.
Never present estimated data as confirmed.
Start with Step 1.

Here are my review decisions:

INTENT REVIEWS:
Cluster [name]: CONFIRMED as [intent]
Cluster [name]: SPLIT into:
  Group 1 [intent]: [keywords]
  Group 2 [intent]: [keywords]

SERP CHECKS:
Cluster [name]: CONFIRMED, [X] shared URLs verified
Cluster [name]: SPLIT, under 4 shared URLs

PUBLISHING DECISIONS:
Cluster [name]: OWN PAGE, new article approved
Cluster [name]: PATH B, optimise [existing URL]
Cluster [name]: KILL, not worth targeting
Cluster [name]: MERGE with [other cluster name]

Produce the final approved cluster table with all 
decisions applied.
Save to: /briefs/2026 clustering FINAL [topic].md

This is where your SEO expertise earns its place in the system. Here is how to work through the flags efficiently.

Reviewing ⚠️ INTENT REVIEW NEEDED

Open Google and search the primary keyword. Look at the top 5 results. Are these informational or transactional pages? Does Claude’s JTBD classification match what Google is actually rewarding?

If the SERP shows mixed signals, some informational, some transactional, the cluster needs splitting. Tell Claude which keywords go where. If the intent is consistent throughout, confirm the cluster.

Reviewing ⚠️ SERP CHECK NEEDED

Even with the DataForSEO API, some overlaps will be flagged as uncertain. Unusual SERPs, very new keywords, personalised results. For these, open Google manually for both keywords. Count shared URLs in the top 10. Four or more means same cluster.

Reviewing JTBD Classifications

This is the most important review step and the one most content on clustering skips entirely. Read each cluster’s JTBD statement. Does it accurately describe what the searcher is trying to accomplish? A wrong JTBD produces a wrong page. Wrong format, wrong angle, wrong CTA.

Making Path A or Path B Decisions

Your GSC API data tells you:

• No position data: not ranking, new content decision. Is this cluster within your topical authority zone? Is the competition realistic?
• Position 11 to 50: ranking but underperforming, your highest ROI optimisation list. Does your existing page’s intent match the cluster?
• Position 1 to 10: already ranking well, low priority unless CTR is poor.

04Honest Limitations

Most AI SEO content skips this section. I think it is the most important one.

It does not understand business context automatically

Every business context insight required for accurate clustering needs human input before the workflow runs. If you do not provide this in Prompt 2, you will get technically correct clusters that are strategically wrong. Things like brand names that look generic, eligibility terms that are upstream of your product, or existing tool pages that should never get a new article competing with them.

SERP features change

The DataForSEO SERP API pulls live data, but what is ranking today may not be ranking in three months. Cluster formats should be re evaluated periodically, not treated as permanent decisions.

Statistical co occurrence is not genuine relevance

Even with the SERP API confirming URL overlap, Claude will occasionally group keywords that share SERPs for reasons unrelated to topic similarity. Seasonal content, news spikes, Google testing. Your intent review catches these. Do not skip it.

The 30 percent is non negotiable

Anyone telling you AI eliminates the judgment layer in keyword clustering is either selling a tool or has not processed a real keyword list at scale. The system makes the 30 percent faster and more structured. It does not make it disappear. Your experience is not optional. It is the mechanism that makes the output worth using.

05Summary. The Complete Stack

APIs

• Google Search Console API: ranking data, Path A or Path B decisions
• Ahrefs API or Semrush API: volume, competitor gaps
• DataForSEO SERP API: overlap verification, SERP features

Workspace

seo workspace/
├── CLAUDE.md
├── data/
└── briefs/

Prompts in sequence

1. Confirm understanding of CLAUDE.md
2. Provide context and answer CP6 questions
3. Execute the six step workflow
4. Apply your review decisions

Your review covers

• Intent verification on flagged clusters
• SERP overlap spot checks on uncertain groupings
• JTBD classification review
• Path A or Path B decisions using GSC data
• Final publishing decisions

Output

A validated cluster map in /briefs/ feeding directly into Content Gap Analysis. Workflow 2 in the system.

06Frequently Asked Questions