Crawled - currently not indexed: Why it happens and how to make Google love your site again
Learn how to identify why your pages aren't being indexed and get them on the SERP.
Updated February 27, 2024.
- Google Search Console is a vital tool for understanding your website insights.
- Various factors affect Google's ability to index your pages.
- Inspecting your URLs can uncover why they are not being indexed.
- Creating unique and valuable content helps to avoid "crawled - currently not indexed" issues.
Trying to understand some of the jargon and intricate tools that Google Search Console offers can be overwhelming, but understanding your search analytics is key to optimizing your pages and improving your overall site health. When you come across a page that isn't indexed, it's important to figure out why.
» Interpret key page analytics with an all-in-one page builder.
What does "crawled - currently not indexed" mean?
If a page is marked as "crawled - currently not indexed," it means Google has found and crawled the page but hasn't added it to search results yet. Importantly, this status doesn't guarantee that Google will index it in the future, leaving you wondering about the fate of your crawled but unindexed pages.
How to find your page indexing report
When you log in to Google Search Console, you'll find a page indexing report by clicking on "Pages" in the left-hand panel.
This report reveals which pages Google has indexed, meaning they will appear in search results pages.
It also notifies you of pages that are not indexed and can give you a variety of page statuses. The "Why pages aren't indexed" section on this page lists the different reasons why your website's pages aren't indexed.
» Explore the difference between paid search and SEO in conversion funnels.
Crawled vs. discovered: What's the difference?
Here, we have another index status, "discovered - currently not indexed" that seems very similar.
The key differences are:
- Crawled: This means Google has looked at your page but decided not to add it to the search results.
- Discovered: Google knows the page exists but hasn't crawled it yet. It's like a virtual queue—your page is waiting to be crawled.
Why the wait?
Google has a "crawl budget." This budget allocates how much resources are given to crawling your website. If you have a smaller site with a lower crawl budget or the budget is already exhausted, some pages must wait to be crawled.
Common reasons for not indexing
Several factors can cause a page to receive the "crawled - currently not indexed" status. One reason could be that the data on Google Search Console is outdated or inaccurately reports the page as not indexed. Other reasons Google might not index a page include:
- Low-quality/thin content
- Duplicate content
- The overall health of your website
- Poorly designed sites
- Redirects and broken links
- Unimportant pages (e.g., pagination pages)
Unfortunately, if you run an e-commerce website with many similar product pages, Google may mistakenly perceive this as duplicate content and choose not to index it.
While crawling your page, Google collects all the links found on the page and sends them into a crawl queue. Some non-essential URLs, like image or RSS feed links, could also fall into this non-indexing category. » Explore how Entail can solve SEO for you.
How to find the reason behind "crawled - currently not indexed"
While Google doesn't provide specific reasons for pages receiving the "crawled - currently not indexed" status, you can conduct your own investigations using Search Console.
By reviewing non-indexed URLs and identifying patterns, you can uncover potential issues. For instance, consistent URL issues containing "/product/review" might indicate a problem preventing indexing.
Follow these steps to find out why your pages receive the "crawled - currently not indexed" status:
1. To start, go to the "Why pages aren't indexed" section of the "Pages" overview. You can view all of the reasons for unindexed pages here.
2. Click on the "Crawled - currently not indexed" status and it will take you to a new page showing all of the pages under this status.
3. Go to the "Examples" section below the graph.
4. Click on the URL you want to inspect.
5. Click "Inspect URL."
6. Check the status of each page for trends. The URL inspected below has no information for sitemaps, referring pages, whether indexing is allowed, or canonicals.
7. Live test your URL to see a screenshot of what Google has found.
8. This URL renders a plain blue screen telling Google there is no content and shouldn't be served on the SERP.
Not indexed happens
It's quite common for websites to get this status on some of their pages as there might be delays between crawling and indexing. But it's good practice to be sure of the reasons and update any broken links or other errors on your site.
» Learn how AI can badly affect SEO.
How to get your pages indexed
Some pages aren't indexed because they don't really give any information to users. They could simply be there for the website's functioning or to offer automatic feeds. Others are serious issues you need to deal with so that important content pages can be indexed.
False index status report
Sometimes you might find that the URL you inspect is actually indexed and the system is reporting on old data. The report will rectify this in time and requires no additional investigation. Use the site search operator to make sure the URL is not indexed.
To use the site search operator, type the following into your search bar:
If you see that your page isn't indexed, you'll need to do some more investigating.
Feed and paginated URLs
Feed and paginated URLs do not require additional work.
- Feed URLs: They are left out of the index as they lead to an XML document. This isn't useful in the search results. You can identify these by looking for .../feed at the end of a URL.
- Paginated URLs: These URLs are pathways for Google's crawlers. They are necessary to the crawling process but don't need to be indexed themselves. Ensure that no "nofollow" tags are contained on these pages as it will stop crawling to pages further on the path.
This is where you need to put in some work. If Google crawls your page and finds minimal content, it will not be indexed.
This can sometimes happen if you have a product page with only an image and a title. Other pages with single lines of text would also not be indexed according to Google's conditions for valuable content.
If it's a product page you want indexed, add more unique descriptions about the product. This can signify to Google that the page is valuable to the user experience.
» Entail's content marketplace can help you develop expert content.
Google filters out duplicate content to prevent the same content from being served in searches. This should be high on your to-do list if it's happening on your site.
To make sure this is what's happening to your page, you would need to perform a Google search using a piece of the content on your page.
- Type "&num=100" into the search bar after your piece of content. This will show you the top 100 results for that content. Use a paragraph or a few sentences of content.
- Type "&filter=0" into the search bar after your piece of content to see the unfiltered results.
If your URL is showing up in the unfiltered results then it is likely that you are being filtered out of the SERP because of duplicate content.
Try to create an original spin on the content you're creating. Remember, Google loves unique, authentic content. To get indexed and rank, you must develop high-value content that users find interesting and helpful.
Your content deserves to be indexed
The importance of indexing cannot be understated. If you want to become a thought leader in your domain, you need people to see your content.
By following Google's guidelines and producing valuable and unique content, you can signal that you are an authority. Google, in turn, will index your pages and serve you up to users.