how search engines work

Search Engine Optimisation in GodalmingYou don't need to know how a search engine works but if you are wondering . . .

What is a Search Engine?

A search engine is a database system designed to catalogue the words found on websites. Search engines have programs called 'spiders' (also known as a bots, crawlers or robots) which automatically visit a website and copy the text content back to the search engine's database. The search engine will process the copies of the files and reduce them to key data for inclusion into the database.

When you use Google (and other search engines) you are not searching the internet but a filtered text copy of it held in huge databases. If the spiders have not found or 'indexed' your website properly then it won't be found when someone does a search.

I can't make the spiders visit your website however, I can help them to find all the content when they do. A new site can be listed in a matter of days, but sometimes it takes a few weeks. Once found, then the more your site is updated the more often the spiders will visit as they are hungry wee sprites eager to find new information.

How do Search Engines find my site?

Several elements of a web page are used by search engines to index a website. The domain name, the page title, the keywords, the description, the heading tags (visible headings in your page like chapter titles in a book) and most importantly the text content. The keywords and description are found in a special hidden section called 'Meta Tags'. The 'Keyword' Meta Tag used to be important, however it is far less so now. File naming conventions can also help, for example using keywords to name a page, a folder or adding 'Alt' tags to an image - these are the words that appear when your roll your mouse over an image in Internet Explorer.

Nowadays the 'keywords' are regarded as those used within the visible text of a page, not the Meta Tags. Copy needs to be written to make use of keywords that are most searched for. For example 'website' might be searched for ten times more often than 'websites' and so would be a better choice - this is 'keyword optimisation'. It is possible to research which words are used more often and so tailor the copy within a page to achieve higher rankings. Using different words with the same meaning can make the copy more interesting to read and help with SEO. If the copy on a page feels repetitive and laden with keywords then the search engines will pick up on this and probably penalise your site. This called 'keyword stuffing'.

Search engines continually get 'smarter' by using semantic approaches and relationships between websites. It is now important to use the correct phrases and context-sensitive copy. Keywords alone are not enough. Engines are more language based - for example they try hard to find and answer to the question 'Who is the Queen of England?' rather than just match the words. The answer should be Queen Elizabeth II - no other queens should come up as the sentence had the word 'is' not 'was'.

Are there really hundreds of search engines?

No. Just a handful - less than 6. Years ago you could submit your site to lots of engines that have long since vanished. Many connected to the same database and were just branded front ends. 

Inbound links

Inbound links are magical things . . . if you can get thousands of sites to link to yours Google will think it's a very important place to visit! In this case the SEO on your site does not matter. In the extreme it explains why some websites for famous people and companies are number 1 regardless of their content. The quality of inbound links matters too. If you only have a handful but they are on very popular websites this will drive your ranking up. Achieving both of these goals is impossible for most customers however, you should always ask good quality websites to link to yours.

Is the way my site is built important?

The structure of your site is very important. Search engines often only read the first 30Kb of a web page, so if your page links and copy are buried too far down, or at the bottom of a large page of javascript the spiders may not index your pages and content properly. Cheap website software is often guilty of creating pages 'drowning in code'.

By separating content from structure (i.e. using CSS designs like this site) this problem is massively reduced; the search engine has far less structural code to wade through. To do this properly requires a thorough knowledge of the code behind the pages. Framesets and inline-frames are avoided by most professional designers because search engines can't find their way through them to read the content of the pages.

What about Flash sites?

In terms of of search engine optimisation, Flash sites are considered a poor option. It's also best not to create navigation menus in Flash, and if it has to be done then I provide alternative HTML text-only menus.

I am happy to build websites completely in Flash if you wish, but I do not offer any SEO on these products. I have noticed over the last few years that the demand for Flash-only websites has fallen as the desire for SEO performance has become more important for clients. You can however have SEO sites that have Flash content - such as audio and video players.