It’s been a while since you decided to build a website for your brand. Since then, you’ve put much effort into ensuring that it works. You’ve chosen a domain host and a name, gone through several site templates, and built yours. After all these efforts, your site is now live.
But wait, how do you get your site to show on Google? You’ve published your pages. But as expected, they aren’t on Google’s result pages. It’s almost like you don’t exist. Well, things would remain that way until you get your site crawled by Google.
In other words, you have to let the search engine know you have a site and want it indexed. But how? And is it even necessary? After all, most SEO articles say the first step to successfully ranking on Google is to know your target audience and create content tailored to them. Are they wrong, or is there a step missing? To put it simply, yes. The first step to show up on Google's SERPs is to get your site crawled. In this post, you'll discover the best step-by-step method for that. But first, let's consider why.
Why Should You Get Your Site Crawled? (Importance of Indexing in SEO)
Before answering that, consider how Google operates to answer a user’s query:
The first thing it does is travel across the web through connected hyperlinks, searching for webpages it has never seen before or updated ones. The process of this travel and search is known as Crawling.
Once it finds a new or updated site, it saves or stores the page in its database (This process is Indexing).
As a user, when you make a search query, Google draws pages from its database that match your search. These pages are organized and sorted in order of relevance. We know this as Ranking.
With that in mind, SEO is the practice of making your site more relevant to a search query to be ranked higher. But before that, comes crawling and indexing - the two processes that alert Google of your existence.
But as a new website, Google won't find your page automatically as it travels through external links. That's because nobody may link to your site immediately. That's why you need to alert the bot (also called Google crawler or spider) manually. And the best tool for this is Google's search console. Let's see how it works:
How Do You Get Your Site Crawled By Google Using Search Console?
Google Search Console is your go-to tool for matters concerning crawling and indexing. It allows you to manually submit your site to Google's crawlers, telling them to visit your web pages. In the following guide, you'll learn how to do this in two major ways:
But before that, here's how to quickly sign in and get started on Search Console:
➔ First, visit the login page.
➔ After clicking Start Now, you’ll need to add your website. You’ll be asked to choose between two property types. First, the domain property (a property that also includes all subdomains and different protocols) and a “URL prefix” property (only includes links with the protocol and prefix you specify).
➔ Next, you'll need to verify your property through one of the available methods provided. Some methods include an HTML file upload, HTML tag, Google Analytics, and DNS record. Note that domain property types only allow for DNS record verification.
➔ If for some reason you can't verify, you can ask someone who already has access to grant you access to the site on the search console.
Once verified, it’s time to get your site crawled by Google. Two main ways include:
1. Request Indexing
➔ Once you’re in, navigate to the top search bar and enter the link to the webpage you want to index.
➔ Next, you’ll see whether Google has already crawled the page or not. If not, you only need to hit the “Request Indexing” icon as seen below:
➔ After that, you’ll see a message confirming your request has been sent.
Remember, this only tells Google to visit your page. It does not guarantee your site will get crawled immediately. You may need to wait a few hours or even days before Google indexes the page.
Regardless, this is the most straightforward method of getting Google to crawl your website. The next method may be more complex and not suitable for small websites.
2. Submit A Sitemap
A Sitemap is a file containing the list of webpages, media, and other content on your website. It helps search engines understand how your site is arranged, thus helping them crawl it better. Sounds good, right? But you should know that a sitemap is not necessary for all websites.
This is especially so if your site is small and does not contain a lot of media content. Above all, you might choose to ignore it especially if your internal links are well-structured.
If you decide it’s for you, then go ahead and create a well-structured sitemap. You can also make use of sitemap generators for this. Once done, it's time to submit it to the Search Console.
➔ Navigate to and click on Sitemaps on Search Console.
➔ Enter the URL of your sitemap in the space provided and submit it to Google.
That’s it. This way, Google will be able to visit any important pages associated with your sitemap and update them.
Wrapping Up: Crawling and SEO!
Getting your website crawled and indexed by Google is a vital step you should not overlook. It may sound techy and challenging. But as seen in this article, it is really easy to do that. All you need to do is to get your site verified on Google Search Console. Afterward, using any of the two methods above, you can tell Google of your site’s existence.
Only after this should you go ahead and start creating content for your SEO strategy. And here's one thing to remember. With every post you publish, you'll need to either request an index for that page or update your sitemap to include the post's URL. By doing so, you'll be ensuring that Google is not overlooking your content. For tips on how to create solid SEO content that ranks, see our resources on SEO Content and SEO Optimized Articles.