How to automate keyword research without losing search intent

Automating keyword research gets you to a list of potential candidates. However, then you have to determine whether writing that page of text is worth it for you.

For example, if you have a tool for small research teams to organize their customer interview notes, a description of “AI tool for research” is very sparse. Describe who would use this tool, what input they would put into it, what they get out of it, and what features already exist. For example: “Product managers upload interview notes and get a list of the recurring themes in reviewable form.” Note that you would not include features that are in the roadmap for this app.

Set the boundaries before collecting keywords

Select a country for your research. Search volume and SEO difficulty can vary greatly from country to country. Establish minimum requirements for the number of analyzed keywords that will be measured, a minimum volume, and maximum difficulty for the analytics. These can be used as a filter for your first cut of potential keywords, but don’t assume that all remaining keywords are viable.

It is also important to consider the price of a single search. Many SEO tools charge in the order of $150 / month for a subscription. So if you only need to do a single search for a single app or to test a single market then a usage-based run would likely be more cost effective. Note however that even a single search can consume a large amount of model and compute credit and so it is wise to set a budget for such a run.

Run this brief through the Automated Keyword Research app. The app will output the automated keyword research for the search brief, including the measured keyword variants, estimated search volume and SEO difficulty, as well as Parent Topic groups and reasons why particular search queries were excluded from the output. Note that the rejected rows are still included in the output for later review by the researcher. The Automated Keyword Research output can be downloaded as either JSON or Markdown files.

Read the groups, not just the biggest number

As an example of what this might look like, the following results might all look “relevant” for a search for help summarizing customer interviews: “customer interview analysis”, “summarize customer interviews”, “customer research template”. But the last of these could simply be looking for a free downloadable template. Open up the search results for each of the phrases that you found useful, and review the URLs that users are being sent to. Can you provide the same value in an article, tutorial, or even a product page for your existing tool?

Additionally, notes within the Parent Topic groups can reveal overlap in terms of what users are looking for. A single useful page can serve an entire group of related variants (all referring to the same type of outcome). Also, note that low difficulty scores alone do not guarantee ranking well for search terms, relevance is also crucial.

Remember the cases where Recurse’s Keyword Research run did not find a necessary measurement. These are indicated by “Unknown” in the tables. A run may fail to gather sufficient data for many reasons. Even though the run’s findings for such a case are extremely limited, they are still valuable for developing an understanding of what was measured and what remains a mystery.

Turn the inventory into an editorial decision

For each of your search phrases, create a single sentence that tries to answer the following four questions: Who is searching for this? What are they trying to do? What would a really great article on this topic look like? Does your product help with this search phrase in any way? If your product only relates in the last sentence of a long article then this probably isn’t the topic for you.

Automating this type of data collection reduces repeated work and makes research trade-offs visible. Start with Recurse’s keyword research task to carry your keywords, country and language into agent setup. Your agent checks available APIs and costs before running the research.