You don't need a $100/month subscription to Ahrefs or SEMRush to spy on your competition. The most valuable document on your competitor's website is public, free, and likely hiding in plain sight.
It’s called the XML Sitemap.
A sitemap is a blueprint of every URL a website wants Google to rank. By analyzing it, you can see their entire content strategy, their publishing frequency, and their keyword clusters. In the past, analyzing thousands of URLs manually was impossible. Today, with ChatGPT, it takes about 5 minutes.
This guide will teach you a method for competitor keyword analysis free of charge, using nothing but a browser and AI.
Step 1: Finding the Hidden Map
First, pick your top competitor. This should be a site that ranks where you want to rank. To find their sitemap, try these URL patterns in your browser address bar:
- domain.com/sitemap.xml
- domain.com/sitemap_index.xml
- domain.com/post-sitemap.xml
Most modern websites (especially those on WordPress or Shopify) use `sitemap_index.xml`. Click on it. You will likely see a list of other sitemaps. Look for the one labeled `post-sitemap` or `blog-sitemap`. This is where the articles are.
Step 2: Extracting the Data
Once you open the post sitemap, you will see a wall of URLs. Don't be intimidated. We just need the text.
The "Lazy" Method: Press `Ctrl + A` (Select All) and `Ctrl + C` (Copy). It might copy some dates and metadata, but ChatGPT is smart enough to ignore that.
The "Pro" Method: If the list is huge, you might want to use a free browser extension like "Link Grabber" to extract just the URLs into a clean list.
Step 3: The "X-Ray" Prompt
This is where the magic happens. We aren't just asking ChatGPT to read the list; we are asking it to reverse-engineer the strategy.
Paste your list into ChatGPT (use GPT-4 or Claude 3 for best results) followed by this prompt:
Copy This Prompt
"I am pasting a list of URLs from a competitor's sitemap below. I need you to perform a keyword analysis on these URLs. Please structure your response into 3 parts: 1. **Core Clusters:** Group these URLs into 3-5 main topic categories. What are they writing about the most? 2. **Keyword Extraction:** For each URL, guess the primary target SEO keyword based on the slug. 3. **Gap Analysis:** Based on these topics, what 'sub-niches' or questions are they seemingly ignoring? Here is the list: [PASTE URLS HERE]"
Step 4: Decoding the Strategy
ChatGPT will output a structured report. Here is how to interpret what you see:
1. Identify the "Money Pages"
Look for URLs that contain words like "vs", "review", "best", or "alternatives". These are high-intent commercial keywords. If your competitor has 10 articles comparing "Product A vs Product B," that is a signal that this specific cluster makes money.
2. Spotting the Clusters
You might notice they have 50 articles about "iPhone cases" but zero articles about "iPhone screen protectors." This is a Content Gap. You can swoop in and dominate the screen protector niche while they are distracted.
3. Publishing Velocity
If the sitemap includes dates (Last-Modified), ask ChatGPT: "Analyze the dates. How often are they publishing? Did they stop posting about a specific topic recently?"
If they stopped posting about a topic 6 months ago, it might mean that topic wasn't profitable. Learn from their failure without spending a dime.
Step 5: From Analysis to Revenue
Now that you have their keyword list, don't just copy it. Improve it. This is the concept of "Information Gain" we discuss in our SEO Strategy Guide.
Take their top 5 performing keywords (based on your analysis) and run them through our tools:
- Check Profitability: Use our LTV Calculator. If you rank for this keyword and get customers, can you afford the acquisition cost?
- Check Intent: Are people looking for a simple definition, or a tool? If they are looking for a tool, build a simple calculator (like we did) instead of writing a 2,000-word essay.
Conclusion: Intelligence Over Budget
Competitor keyword analysis free of charge is possible if you understand that data is public. The sitemap is the most honest document a company produces.
While they spend thousands on tools, you can use AI to read their map, find the holes in their defense, and build a fortress in the gaps they left behind.
Frequently Asked Questions
What if the sitemap is blocked?
Some sites block `sitemap.xml` in their `robots.txt` file. In this case, you can check `robots.txt` (domain.com/robots.txt) to see if they list a custom sitemap URL there. If not, you may need to rely on Google searches like `site:competitor.com` to see their indexed pages.
Does this work for YouTube channels?
Partially. You can copy the "Videos" tab list of a YouTube channel and use the same prompt. Ask ChatGPT to analyze the video titles to find the content clusters.