---
title: "AI SEO Tools: My Toolset With Tests and Pitfalls - Łukasz Rogala"
description: "Which AI tools I use for SEO: language models, search data, site crawling and my own scripts. I show tests with numbers and the pitfalls I hit."
url: "https://rogala.pro/en/blog/ai-seo-tools/"
language: "en"
author: "Łukasz Rogala"
published: "2026-10-18T12:15:00+02:00"
modified: "2026-10-18T12:16:57+02:00"
---

[Blog](https://rogala.pro/en/blog/)

# AI SEO Tools: My Toolset and the Tests That Shaped It

[Łukasz Rogala](https://rogala.pro/en/#about) · October 18, 2026 · 18 min read

**My toolset of AI tools for SEO is language models, paid data on Google results, a program for crawling sites and my own scripts.** A model is good at organizing data, describing it and writing code. A result from one SERP snapshot or one model answer can still be random. That is why I show the tests that changed how I use these tools. I do not make a ranking from 1 to 10, because I did not compare these tools side by side on the same data.

**I ran the tests in this post from July to October 2026, on my own data and on data from projects whose names I do not give.** I pay for the paid tools in this post, and I give their names without links. I co-run Linko, a link platform, and at VerticalSEO, my agency, we build our own SEO tools. I write about them with that caveat. I give model names in the versions from the day of the test wherever I recorded them, because versions change every few weeks. I describe results from small samples as preliminary.

## What does AI do well in SEO, and what does it not?

**A language model reads, organizes and writes well, but it counts badly from memory and cannot see data that you do not give it.** On September 26 and 27, 2026, I tested language models on 17 types of SEO tasks, each at an easy and a hard level. I built some of the types on the basis of 103 job ads for SEO specialists. The ads most often listed technical audits, reporting and data analysis. Among tools they most often listed Search Console, Google Analytics 4 (GA4), Ahrefs, Semrush and Screaming Frog.

**The strongest models scored almost 100% on nearly every type of task, so the differences showed only in the weaker models.** The top group included, among others, Claude Opus 5.5, Fable 5.1, Codex and Gemini 3.1 Pro. On counting from a file at the hard level, Claude Haiku 4.5 got 2 points out of 10. Gemini 3.8 Flash with a low reasoning level got 1 out of 10, and the local Gemma 4 12B 0 out of 10. In tasks where the model could calculate the data with a Python script, all the cloud models tested got full marks. Even the weakest of them did. The data in those tasks are Screaming Frog exports, 45,000 GA4 events and a server log of 160,000 lines. That is why in my work a script that the model writes and runs does the counting.

**Texts in Polish were graded blind by two models, and here the differences were large.** The graders did not know which model had written the text. The scale had 5 grades, and 5 meant a text ready to send to a client without corrections. Only Claude Opus 5.5 and Claude Sonnet 5 got a 5 on the hard writing task. Gemini 3.1 Pro got a 2, because it made up a “conversation yesterday” with the recipient. The grader Claude Opus was stricter than the grader Gemini, both for Claude's answers and for the other models'.

**The same model can give a different result when the same test is run again.** Claude Haiku 4.5 got 62 and 39 points out of 102 in two test runs. In the post on the [SEO audit](https://rogala.pro/en/blog/seo-audit/) I described why an AI model will not do a full site audit on its own.

## What is my toolset, and what do I use each tool for?

**I divide the tools into six uses in my work, not by vendor.** The same model appears in several uses, because I use it one way for reading data and another way for checking. For each use I give the test after which I changed how I work.

**Every use in my toolset has a test that changed how I work**

A six-by-two matrix. Rows: language models, local models, results data, site crawling, content and semantics, checking. Columns: tools and tasks, and the test that changed how I work.

*Tests from September and October 2026 on my own data and on data from projects whose names I do not give. I treat results from small samples as preliminary.*

## Which language models do I use, and for what?

**I do the main work in Claude Code, that is, Claude running in a terminal with access to files, scripts and data.** Claude writes and runs scripts, reads exports and prepares drafts of reports. Codex (OpenAI) and Gemini (Google) get the same materials for a second check.

**I check texts and reports with a second model from another vendor, in two rounds.** The first round looks for factual errors and gaps. The second round checks only the corrections, because new errors appear when things are corrected. In October 2026, in one project, the second round found a wrong total number of links in a guide, the one from which a percentage was calculated: 79 instead of 76. That error appeared only during the corrections. My checking script did not find it.

**A model used as an LLM judge can measure something different from a person, so I test it on texts that I have already graded.** On October 7, 2026, I gave two models 87 descriptions of Google patents to grade. Among them were 8 descriptions that I had earlier judged ready myself, and the judges did not know it. Fable, a model from Anthropic, judged 5 of those 8 ready. Codex with a high reasoning level judged 1 of 8 ready.

**These two results apply only to the texts that I judged ready myself.** I did not grade the remaining 79 descriptions, so I do not know how many of the rejections were justified. Codex rejected 51 of the 87 descriptions in total, because it compared every sentence of a description with the claims, that is, the legal scope of the patent. The agreement between the judges was close to chance. Cohen's kappa was 0.06, where 0 means agreement by chance and 1 full agreement.

**I use local models on my own computer with a graphics card for bulk labels, because I do not pay for each item.** On October 1, 2026, I tested local models on four simple tasks. The data were pages of three Polish online stores. Gemma 4 31B recognized the page type correctly in 98.5% of 403 pages. The page types were product, product list, article or other. A script with rules got 84% of those pages right.

**Gemma read prices and judged the consistency of numbers almost without errors, but part of the reference answers in this test came from a model, not from a person.** The reference for page type was labels from a script corrected by the Claude Opus model. Gemma read the price correctly from all 177 product cards. The reference was the price from structured data, consistent with the text of the page. In 99.3% of 134 cases Gemma correctly judged whether a number in a sentence matches the data. Half of those sentences had a deliberately changed number. Gemma also recognized the type of demand in a keyword (offer, outside the offer or advice) in 97.7% of 218 keywords. The reference here was labels from the Claude Opus model.

**The same model family failed when comparing numbers between two files in a real offer.** On October 12, 2026, two versions of Gemma 4, 26B and 31B, compared 63 rows and reported only false contradictions. They concluded, for example, that “-34%” and “one third” are contradictory values. The only real problem in that text was not about numbers but about an exaggerated word next to a result from a table. So it was not part of Gemma's task. Now a script checks whether every number from one file appears in the other. It is a check of whether the numbers are present, and their meaning is checked by a person or a second model.

## Which tools do I use for ranking and demand data?

**I take data on Google results from DataForSEO, Search Console and Ahrefs, and each of these sources has a different pitfall.** DataForSEO sells snapshots of Google results and demand data, charged per query. From September 17 to October 14, 2026, I ordered 17,076 tasks there for 99.89 USD. Among them were 13,085 snapshots of Google results.

**One SERP snapshot is not enough to measure ranking, because successive snapshots of the same keyword look different.** On October 7, 2026, I pulled results for 94 keywords three times, an hour apart, and twice within the same minute, for 1.57 USD. Then I compared the domains from the top ten in two snapshots of the same keyword. The shared domains made up about 15% of all distinct domains from both snapshots, as a median. For snapshots from the same minute it was about 8%. One keyword had 4 variants of results as a median. That is why, instead of a position, I calculate the share of snapshots in which a domain is in the top ten. There are about 10 snapshots, and I send them all at once.

**This measurement invalidated my earlier test, in which I compared the ranking from one snapshot with citations from five snapshots.** The test result “passed” turned out to be false, so I suspended the test. AI Overview, that is, the AI answer above Google results, was stable in this measurement: it appeared in 97-99% of snapshots, and the list of cited pages did not change.

**Search Console shows clicks measured by Google itself, but an export from the interface cuts the table to 1,000 rows.** A thousand queries sorted by clicks is the most clickable part of the data. On September 20, 2026, in one site, an export from the interface showed a CTR of 4.28% for positions 1-3 in the USA. CTR is the share of impressions that ended in a click. Data from 16 months pulled without the 1,000-row limit, that is, 123,498 queries, gave 1.40%. I pull such data through the Search Console API, that is, an interface for programs. The second way is a report in Looker Studio, Google's free tool for reports, with the Search Console connector.

**I use Ahrefs for links, keywords and competitors, but I check its traffic estimates in Search Console.** The traffic column in Ahrefs Rank Tracker is an estimate: estimated demand multiplied by an assumed click-through rate. In one site this estimate was 10-50 times higher than the clicks in Search Console, and in another about 3 times lower. Rank Tracker also counts blocks on the results page as positions. In one case a drop from 1st to 13th place meant only that an AI Overview had appeared above the results.

## Which tools do I use for technical audits?

**For audits I use Screaming Frog and headless Chrome, and an AI model reads the results but does not replace crawling the site.** Screaming Frog goes through a site the way a search engine robot does. Headless Chrome shows the page after JavaScript has run, so I compare the two versions of the page. In one audit, crawls with and without JavaScript run in parallel gave a false signal about rendering. Since then I run them one after another.

**When the firewall of an audited site blocks my crawling program, I fetch single URLs through the Apify Web Fetch service.** In a test on October 12, 2026, the service read 8 of 8 pages that I could not fetch from my IP address. Two of them belonged to clients' sites, and six to public review sites, comparison sites and online stores. It cost 0.00155 USD per URL. The service sometimes reported a successful fetch with code 200, though it had fetched only a verification page. So I treat a very short text or a text about verification as a block and fetch again.

**This service does not show what a real Google robot or AI robot sees.** It connects from a different IP address, and even when I give the name of a robot, the other headers reveal an ordinary Chrome browser. So I check robot blocks in Search Console and with my own script that compares the server's responses for different robots. In the post on the [llms.txt file](https://rogala.pro/en/blog/llms-txt/) I described how I check by IP address whether a request was sent by a real robot.

**I built an assistant for technical audits on my agency's audits from 2020-2026, and I measure it on audits that a person did earlier.** The assistant is an AI agent that collects data about a site and checks it against a catalog of rules written down from those audits. I approve the scope and priorities of the audit myself, at the start of the work and before the audit is sent. I will add the result of the measurement here after the second measurement on a new site.

## Which tools do I use for content and semantics?

**I assign a keyword to a page with hybrid search, because the earlier version of my tool hit only a third of keywords.** On October 14, 2026, I took keywords from Search Console for two sites, without keywords containing the brand name. For each keyword the reference was the page that got at least 60% of the clicks for that keyword and was among the crawled pages of the site. The earlier version of the tool used the MiniLM model for embeddings, that is, turning text into a vector of numbers. It pointed to the reference page first for 34% of keywords.

**Hybrid search combines three lists of pages and hit the reference page for 80.5% and 77.1% of keywords.** The first list comes from the e5-large model. The second is search by words in the content, and the third by words in the title, H1 heading and URL. I combine the lists with the RRF method, which rewards pages that rank high on several lists at once. The best single embedding model was different on each of the sites, and the hybrid won on both.

**Hybrid search hit the page with the majority of clicks more than twice as often as the earlier version of my tool**

Horizontal bars for four methods. Each has two bars: the share of keywords for which the method pointed first to the page with the majority of clicks, separately for two sites. MiniLM 33.9% and 34%, e5-large 70.8% and 54.9%, search by words in the content 67.1% and 63.2%, hybrid search 80.5% and 77.1%.

*Test of October 14, 2026. The reference page is a page with at least 60% of the clicks for a given keyword in Search Console, excluding keywords with the brand name. The MiniLM result concerns the earlier version of the tool with a text-splitting error. Hybrid search combines e5-large with search by words in the content and in the title, H1 heading and URL of the page. The result measures agreement with clicks in Google, not which page best answers the query.*

**In this test I found an error in my own tool: the model did not see the end of most text fragments.** The script split the text into fragments of 80 words, while the model accepts 128 tokens, that is, pieces of words. Polish words split into more tokens than English ones, so 89-98% of fragments were cut off. The 34% result therefore concerns the earlier version of the tool together with this error, not the MiniLM model itself. The test measures agreement with clicks in Google, not which page best answers the query.

**I extract entities, that is, names of companies, products, people and concepts, from texts with the local GLiNER2 model.** In my test from July 2026 the model had an F1 of 0.94 on Polish texts, the same as on English ones. F1 combines precision and recall, and 1 means a perfect result. This result is for labels that do not overlap. When the labels “company” and “brand” were in one query together, F1 dropped to 0.85. The model runs on an ordinary processor, about 20 paragraphs a second, with no per-page fee.

**I do not use GLiNER2 to recognize the intent of keywords, because in my test it was right in only 33-42% of cases.** It was often wrong with a confidence of 0.99, that is, almost the highest confidence it reports.

**I made the topic map of this series of posts from entities on 71 competitor pages from Poland and the USA.** The script extracted 110 entities and 1,764 triples, that is, records in the form entity, attribute and value. The data cost 0.072 USD. The map showed the roles of the posts, pairs of texts to connect with links, and topics that this site lacks.

## Which tools do I use for link building?

**For link analysis I use Ahrefs, but I check links marked as lost live before drawing conclusions.** In October 2026, in one project, Ahrefs showed 360 Polish domains as lost. 140 of them still linked with the same URL. About 70 URLs answered with a 403 error to a fetch by an ordinary program, so I checked them in a browser controlled by a script (Playwright).

**I co-run Linko, a link platform, so I do not describe it here like a tool of another company.** In the post on [link building](https://rogala.pro/en/blog/link-building/) I described how I assess sites before buying publications. In the post on [link building in the AI era](https://rogala.pro/en/blog/ai-link-building/) I described how AI helps prepare publications and contact with publishers.

## Which tools do I use to measure AI visibility?

**I described the tools for measuring a brand's AI visibility in a separate post, so here I give only the rule.** I count the brand's mention in an answer and the citation of its page separately. I ask each question several times, because the answers of assistants change. In the post on [brand visibility in AI](https://rogala.pro/en/blog/ai-visibility/) I described the measurement, the number of repetitions and the pitfalls of the tools. In the post on [AI SEO and GEO](https://rogala.pro/en/blog/ai-seo-geo/) I described what I do on the site so that AI assistants can cite it.

## What SEO tools have we built ourselves?

**When no tool answers my question, I write my own script with the help of AI.** Most of these scripts are for internal work: crawling, pulling data, entities, checking texts and numbers. Before I start using a script in my work, I test it on data with a known correct answer.

**At VerticalSEO we are preparing 16 SEO tools that will be available through the site and through an API, also for AI agents.** The tools are not published yet. The tests from October 8, 2026, checked whether the tools work according to the specification. All 610 of 610 API tests passed, and 325 of 325 tests of tools for agents in the browser. These tools work in the WebMCP standard, which exposes a site's functions to AI agents. The tests do not yet measure whether the tools give accurate SEO conclusions.

**A test of the tool for checking entities showed that a name alone is not enough to point to an entity.** I was looking for Allegro, a Polish marketplace, but the tool picked from the Google Knowledge Graph, that is, its database of entities, a piece by Vivaldi. Under my first and last name it picked an actor. Now the tool confirms an entity only when Google gives the page being checked for it. A confirmation is also an identifier pointed to by Wikidata or by the structured data of the page.

**I am preparing LemmeGSC, my dashboard for Search Console data, for release as open source.** The dashboard pulls data from Search Console without the 1,000-row limit. Among other things it shows keyword cannibalization, that is, several pages of one site ranking for the same keyword.

## Which free SEO tools are enough to start?

**Search Console, Looker Studio, the free version of an AI assistant and the free version of Screaming Frog are enough to start.** Each of these tools has a different role:

- Search Console shows clicks, impressions and queries from Google;
- Looker Studio with the Search Console connector pulls this data without the 1,000-row limit;
- the free version of an AI assistant groups keywords and writes drafts and scripts, if you give it the data;
- the free version of Screaming Frog crawls up to 500 URLs, which is enough for a small site.

**Paid tools for keywords and links are worth adding only when you know what question they are supposed to answer for you.** A free AI assistant does not know your Search Console data. It gives the most when it gets an export from you and a request for a script that will calculate the result.

## How do I test an AI tool before I trust it?

**I test a tool on data for which I know the correct answer.** Not every test in this post had all the elements of the list below. In the test of local models, the reference for page type was corrected by a model, not by a person. Today a test has these elements:

- a ground-truth set from a person's decisions or from Google data, for example the page with the majority of clicks for a given keyword;
- the expected result written down before the test is run;
- a repeat of the same test run, because one run of a model or one SERP snapshot can be random;
- cases with a planted error, to check whether the check catches it;
- cost and time per item;
- numbers checked by a script, not by a model.

**The data for the tests cost me little, and the most work goes into the ground-truth set.** The test of SERP snapshots cost 1.57 USD, and the data for the topic map of the series 0.072 USD. A ground-truth set requires a person's earlier judgment. The costs given do not include that work.

**I record the result of a test together with what the test did not check.** For assigning a keyword to a page I know the agreement with clicks in Google, but I do not know whether the page it points to best answers the query. For local models I checked four simple tasks, not full jobs. Without such a record, the result of a test is easy to take as broader than it is.

**I will check which tasks in your SEO can be sped up with AI tools, and where a script or a person is needed.** Tell me which tasks you repeat every week and what data you use. I will reply with where I would start.

[Get in touch](https://rogala.pro/en/#contact) · [LinkedIn](https://www.linkedin.com/in/lukaszrogala/?locale=en_US) · [WhatsApp](https://wa.me/48570081225)

## Frequently asked questions about AI SEO tools

### Can ChatGPT do SEO?

**Yes, with data that you give it, but not as a source of data on rankings and demand.** In my tests I checked mainly Claude, Codex and Gemini, not ChatGPT itself. An assistant of this kind can summarize, group keywords, and write drafts of texts and code, but the result has to be checked. Without data from Search Console or from a keyword tool it does not know your site's rankings or the number of searches. Numbers given from memory can be made up.

### Will AI replace SEO specialists?

**My tests did not check this, because they concerned single tasks, not a specialist's whole job.** In those tasks the strongest models often got full marks. Wrong conclusions came from incomplete data and weak checking: from one SERP snapshot, from a truncated export from Search Console and from a model that compared numbers by itself. A person or a script designed by a person found those errors.

### Does Google penalize AI-generated content?

**Not for using AI alone: Google says that appropriate use of AI does not violate its guidelines.** Google's policy on scaled content abuse covers creating many pages mainly to manipulate rankings, not to help users. That policy applies to pages created in any way, including with the help of AI. Google also recommends manually checking the facts in AI-generated content before publication, including titles, descriptions and structured data.

## Where to start?

**Start with one task that you do every week and with the data you already have in Search Console.** Give the model the full data and ask for a script that will calculate what you now calculate by hand. Compare the result with a dozen or so cases that you check yourself. Only when the results match, run the script on the whole data. In the post on [SEO strategy](https://rogala.pro/en/blog/seo-strategy/) I described which Search Console data I check at the start of work.

External sources (checked on October 18, 2026):

- Google Search Central: Google Search's guidance on using generative AI content on your website (developers.google.com)
- Google Search Central Blog: Google Search's guidance about AI-generated content (February 8, 2023) (developers.google.com)
- Google Search Central: Spam policies for Google web search (developers.google.com)

the Screaming Frog SEO Spider pricing on the vendor's site (limit of the free version: 500 URLs). Own data: a test of the GLiNER2 model (July 2026), a test of 17 task types on language models (September 26-27, 2026), a test of local models on four tasks (October 1, 2026), a test of LLM judges (October 7, 2026), a test of the variability of DataForSEO SERP snapshots (October 7, 2026), a test of Apify Web Fetch and a comparison of numbers in an offer (October 12, 2026), a test of assigning a keyword to a page (October 14, 2026), DataForSEO usage (September 17 - October 14, 2026), the topic map of the series (October 18, 2026).

[Łukasz Rogala](https://rogala.pro/en/#about)

SEO strategist and consultant

Łukasz Rogala is an SEO strategist and consultant, CEO of VerticalSEO and co-founder of Linko. Since 2014 he has delivered 360° SEO: strategy and audits, technical SEO, content and link building, semantic SEO and brand visibility in AI search (GEO).

[About](https://rogala.pro/en/#about) · [Talks](https://rogala.pro/en/#talks)

Read next

- [AI Visibility: How I Measure It and With Which Tools](https://rogala.pro/en/blog/ai-visibility/)
- [SEO Audit: How I Audit a Website Step by Step](https://rogala.pro/en/blog/seo-audit/)