how do search engines work

SEO 101: How Search Engines Work

So much of your business depends on being visible in the search engines. You’d like to try to understand this thing that has so much power over your success.

But figuring out how search engines work can be really confusing. And it’s not just you.

There’s a whole industry based around trying to understand which sites rank for which reasons, and even the information we do know is changing all the time.

We can’t provide an extensive rundown of how the Google algorithm works for you (no one can), but we can provide some basic information on how search engines work that may help remove some of the mystery. Here are a few of the main things you should know.

HostGator Website Builder

The Search Engine’s Goals

The first thing to understand about how search engines work is that their priority is providing the best possible results for what the searcher is looking for.

When it comes to the natural results, the search engine is not concerned about how much a particular website owner might want to grab those top spots or think you deserve it – they only care about the people searching.

By providing the information people need, a search engine can ensure those people keep coming back to use the site again. We know how well that’s worked for Google – many of us use it every day.

That primary goal leads into the secondary goal that generates the company’s profits: making money on ads. Businesses pay to advertise on search engines in large part because they know a huge number of people use them every day. As long as Google keeps its users satisfied and coming back, advertisers will continue to keep the company profitable.

So the search engine’s main concern is therefore how to make sure the results it delivers provide the most useful information for the consumer’s query. That’s where things start to get complicated.


Search Engine Index

For a search engine to be able to identify the right web page for every possible query (or come as close as possible to such a lofty goal), it has to have a record of all the possible web pages online, along with some understanding of what’s on each of them. To do that, search engines create a massive index of web pages.

This index attempts to identify and organize every website and web page on the web in a way that allows it to draw connections between the keywords people search for and the content included on each page.

On top of all that, it needs to be able to assign relative quality to different web pages that cover the same topic.  This is tricky since all of this is happening with machines. People have a hard enough time agreeing on what constitutes “quality” content, but search engines have to determine it based on factors that machines can measure objectively.


Website Crawlers

The first challenge of creating a search engine index is identifying all the web pages out there.

This part of the job is up to website crawlers. Each time a website crawler discovers a page, it crawls the page, collecting all the relevant information on it needed for the search engine index. With that page added to the index, it then uses every link on that page to find new pages to crawl.

how search engine web crawlers work

Website owners can speed up the process of getting a website crawled by the search engines by submitting a sitemap and using internal linking. This is the easy part.


Search Engine Algorithms

The second challenge of the search engine index is the much more complicated one: attributing relative value to all of the web pages.

If the website crawler finds 100,000 pages that include content on them it deems relevant to a specific term, how does the search engine decide what order to deliver those results in?

That’s where the search engine algorithm comes into play. Engineers at each of the big search engine companies have spent untold hours developing a complicated algorithm that uses a number of factors to assign relative value to websites and web pages.


Ranking Factors

While there are many different factors that go into determining exactly why one page will rank over another – many more than we can summarize here, and more even than the greatest SEO expert knows – we have an idea of some of the most important search engine ranking factors Google and the other search engines take into account:

  • Links – Links are the most important ranking factor, especially external links (those that point from one site to another) because every time another website links to yours, it signals to Google that there’s something authoritative or valuable on the page being linked to. When a web page that has a lot of other websites linking to it links to another site, that link is even more valuable because of the high authority the website already has. While everything else on this list matters, a LOT of determining rankings is based on the number and quality of links that point to a website.
  • Website age – Older websites are generally seen as being more trustworthy and authoritative than new ones.
  • Keywords – Search engines are always trying to provide the most relevant results, so they look for terms on the page related to the query of the person searching. The more you use related keywords, the more it signals to the search engine that your content is relevant.
  • Mobile usability – Google has been upfront about using mobile usability as a ranking factor. If your website looks awesome on desktop, but has never been optimized for mobile use, then it could hurt you in the rankings.
  • Page speedPeople are impatient and therefore so are the search engines. A slow-loading page will rank lower because of it.
  • Behavior data – Google tracks what people do once they get to the search engine results page (SERP). If someone clicks on a page and immediately backtracks – that’s a signal that the page didn’t provide what they were looking for. If instead they spend time on the page or even click through to different pages on the site once they get there, then it shows Google that the site provides value.

Google and the other search engines have provided some information about the ranking factors they use, but they generally keep pretty quiet about how their search engine algorithms work. They don’t want people trying to manipulate the results – something that’s long been a problem with black-hat SEO practitioners.


Search Engine Optimization

While there’s definitely a lot we don’t know about how search algorithms work, everything we do know has come to form the basis of the whole field of search engine optimization. SEO is competitive and you’re limited in what you can do to grab the rankings you most want for the keywords relevant to your business, but there’s still a lot you can control and do.

Our series on SEO basics will dive deeper into some of the ranking factors you can control and how to optimize your website to perform better in the search engines. Check back soon for the rest of the series.

Don’t miss the rest of the articles in our SEO 101 series!

Want to boost your website rankings? Get expert help with HostGator’s SEO services.

Kristen Hicks is an Austin-based freelance content writer and lifelong learner with an ongoing curiosity to learn new things. She uses that curiosity, combined with her experience as a freelance business owner, to write about subjects valuable to small business owners on the HostGator blog. You can find her on Twitter at @atxcopywriter.