SEO Advice

Google announced yesterday that they have enhanced the Webmaster tools and now allow you to examine internal and external links on your site much more thoroughly than you can with the link: command.

After playing around with the new enhancements, so far I have to give it a thumbs up. The new features allow you to examine every single link to and from your site, not just the links that sometimes show up with the link:command. In addition, you can look at internal and external links to and from specific pages and export any of this data to CSV format.

Well, its been a while since Google last updated pagerank for web sites. This is a special day where webmasters around the world hold their breath to see what kind of love Google will give to webby creations. Yes, pagerank is really irrelevant when it comes to SEO and rankings but its still one of many indicators and web site owners love the numeric gratification or distress over the lack of it.

Some existing sites are currently showing a pagerank of 0, some new sites are showing an actual pagerank so its pretty clear that Google is in process of updating their data centers. Hold your breath and hopefully the update will be done soon.

Yahoo! Answers celebrated its first birthday yesterday answering any query a web user may have. Over in the Google camp, they recently shut down Google Answers and are no longer accepting questions. I guess Yahoo! can celebrate being top dog for once although I am sure they would prefer it would be a more prestigious tool such as their search.

This might not be a startling revelation of any kind, but just revalidates how critical being part of trusted neighborhoods and having strong linking from established, related sites. You can tell how well Google trusts the content of your web site by how deep Google will spider and index your web site.

Check your web logs and see what Googlebot is doing. Is it spidering deep within your site? Does a site:www.yourdomain.com query on Google display only your home page? If you answered yes to these questions, chances are that there is a low trustiness factor from Google and it just doesn’t think your site is very important.

If your site is new, then you probably know why this is happening. Go and get some backlinks to your site. If your site is established, then you probably have a problem some where. Maybe you are involved in some bad neighborhoods that Google has little respect for. Dump your reciprocal link programs, get some good content and start getting legitimate links to your web site. Easier said than done, but this is the foolproof solution.

Google launched a custom search engine this week that allows you to define what web sites you want to have included in the search and some other nifty features. From Google:

• Specify the sites you want to include in searches
• Place a search box and search results on your website
• Customize the look and feel to match your website
• Invite your community to contribute to the search engine
• Make money from relevant ads in your search results

Google accounts for approximately 50-90% of search engine traffic. On sites that I have been involved with, this number seems to average around 75-80% of total search engine traffic. Based on these numbers, I would focus exclusively on optimizing your web site for Google and not pay too much attention to Yahoo or MSN.

As reported by The Register, there is a new worm that infects Yahoo Instant Messanger and causes fraudulent clicks on web sites containing adsense advertisements. The adsense ads focus on mesothelioma, a rare cancer caused by asbestos exposure that also happens to have very high cost per click values. My guess is that its unlikely the worm creators will ever get paid.

Its a well known fact that Google looks at URLs when determining keyphrase relevance to the web pages in your site. Always use hyphens and not underscores in your URLs. This includes files names and directories. The reason why is because Google disregards underscores (_) but interprets hyphens (-) as a space.

So if you had a web page all about search engine marketing, you would name the file like this:

search-engine-marketing.html

And not

search_engine_marketing.html

The latter example with the underscores is interpreted as searchenginemarketing whereas the example with the hyphens is seen as search engine marketing.

There is a lot of talk about the Google sandbox these days in the blogosphere and various SEO and SEM related web sites. First of all, what is it? Well, quite simply, the Google sandbox is the equivilent of search engine purgatory where new sites exist for up to a year. During their sad existance in the sandbox, they don’t rank (or rank well) for keyphrases that they should. Why does the sandbox exist? Its believed that it’s there to prevent new, mainly spammy sites from popping up and ranking well right away. It forces sites to have to earn their rankings by waiting a while.

The thing about the sandbox is that there is no real way to avoid it. Most sites fall in no matter what kind of quality links they have pointing their way.

There is one loophole: it appears as though keywords or keyphrases that exist in your domain name will rank well even though others do not. This site is a great example. It is new (just over a month old) and and does not rank at all for any seo related keyphrases. Although suprisingly enough, it ranks 11th in Google for seo advice.

I guess it should be no surprise that people don’t take security seriously. This doesn’t really have anything to do with SEO but I am going to share anyways because it was a small source of entertainment this Friday morning.

As reported on The Register, it is very easy to do a search on Google for unprotected web cams using queries such as:

http://www.google.com/search?q=inurl%3ACgiStart%3Fpage%3DSingle

OR

http://www.google.com/search?q=axis+inurl%3Aview%2Findex.shtml

A lot of people setting up these inexpensive cameras aren’t using the included security features to the delight of so called “video hams”.

From the Register:

IP cameras, widely available from around £70 and not to be confused with web-cams that connect directly into PCs, connect into corporate computer network or domestic broadband systems and are capable of transmitting live footage or individual images across the internet. However these systems are often set up without bothering to configure security settings (such as password protection), an omission that means nominally private cams can be viewed over the net by so-called “video hams”.

By using search engines such as Google, snoopers can locate and view hundreds of unprotected cameras, according to Robert Schifreen, IT security consultant and author of the book Defeating The Hacker. Insecure cameras can be easily located using search strings such as “inurl:CgiStart?page=Single” or “axis inurl:view/index.shtml”, to produce interesting results (as illustrated by Schifreen here).

The problem is compounded by the inclusion in many systems of so-called “PTZ” features, which allow surveillance cameras to be panned, tilted and zoomed remotely so that the operator can focus on a person or activity of particular interest. Wi-Fi versions of video surveillance cams have made it possible to install systems in a wider number of locations, further exacerbating the issue.

There was an interesting article written by Brandt Dainow entitled How accurate is Google Adwords? The article looks at the revenue made by Google with its Adwords CPC engine, accuracy of clickthroughs and click fraud. Definately worth a read.

Unlike Steven Colbert who is well known for his truthiness, I just spotted a small Google fib. I was just playing around with Google Sitemaps and noticed a link saying “How can I improve my site’s ranking?”. I’ll bite.

On the resulting page contains the line “We don’t manually assign keywords to sites, nor do we manipulate the ranking of any site in our search results.” That is interesting because I would call removing a site from its search results a pretty good example of manipulating its ranking (such as what happened with webposition.com and many other documented instances).

Hopefully this site doesn’t not not get “manipulated” for its cheekiness. More on Google Sitemaps later.

This was posted the other day to Google’s webmaster blog regarding how googlebot spiders web sites and the use of robots.txt files:

If my site is down for maintenance, how can I tell Googlebot to come back later rather than to index the “down for maintenance” page?
You should configure your server to return a status of 503 (network unavailable) rather than 200 (successful). That lets Googlebot know to try the pages again later.

What should I do if Googlebot is crawling my site too much?
You can contact us — we’ll work with you to make sure we don’t overwhelm your server’s bandwidth. We’re experimenting with a feature in our webmaster tools for you to provide input on your crawl rate, and have gotten great feedback so far, so we hope to offer it to everyone soon.

Is it better to use the meta robots tag or a robots.txt file?
Googlebot obeys either, but meta tags apply to single pages only. If you have a number of pages you want to exclude from crawling, you can structure your site in such a way that you can easily use a robots.txt file to block those pages (for instance, put the pages into a single directory).

If my robots.txt file contains a directive for all bots as well as a specific directive for Googlebot, how does Googlebot interpret the line addressed to all bots?
If your robots.txt file contains a generic or weak directive plus a directive specifically for Googlebot, Googlebot obeys the lines specifically directed at it.

For instance, for this robots.txt file:

User-agent: *
Disallow: /

User-agent: Googlebot
Disallow: /cgi-bin/

Googlebot will crawl everything in the site other than pages in the cgi-bin directory.

For this robots.txt file:

User-agent: *
Disallow: /

Googlebot won’t crawl any pages of the site.

Calgary based Cambrian House performs publicity stunt by delivering 1000 pizzas to Google’s headquarters. Read their press release or watch the video.

Google SOAP Search API (beta) has serious problems with accuracy. If you compare the results returned from the API to results from manually searching on Google, they are rarely the same. The is a serious problem.

It is against the Google terms of service to scrape the search engine results to obtain keyword positions for your web site. Unfortunately, this is the only accurate method of obtaining this information. Scraping search engine results takes up precious server bandwidth and resources that from Google’s perspective they want to be used for actual surfers searching for web pages and not for your web site analysis that earns them zero advertising revenue.

Bottom line is that SEO firms and many others use web page scraping as it is the only method of getting an accurate picture of whats happening on Google. Until Google fixes their API, expect this to continue.

When you hire an SEO company to optimize your web site and help improve your rankings, choose one that you have good communication with and that will explain to you exactly what steps they are taking to optimize your site.

There are a couple key reasons for this. First of all, it makes business sense. You should know what your hard earned dollars are being spent on. All too often, search engine optimization firms operate as a “black magic” entity charming snakes in a backroom to get your web site a higher search engine ranking. In actuality, the only optimization they have ever done is submit your site to Yahoo and the occasional web site change. Maybe the money spent doesn’t have as much value anymore when you know the specifics of what they have done.

The most critical reason is that you are ultimately responsible for any illicit practices that your SEO firm uses. If they use keyword stuffing or cloaking (more on these later) and your web site gets removed from Google, ultimately its your business that suffers. Imagine the scenario where traffic completely disapears from one of the major search engines; this has the potential to destroy your business. An example of this was with Traffic-Power.com who was de-listed from Google (and most if not all of their SEO clients).

Search engine optimization firms play an important role but you should be wary of their practices. Some operate on the up and up, some use black hat methods. Always know what your SEO firm is doing at all times and make them provide you with a synopsis of any optimization they do.

Google has an information page for webmaster that examines guidelines for properly getting your site indexed in Google’s search engine. This is a must read, especially for new webmasters.

Read Article

When desiging your web site, try and keep the directory structure as simply and high level as possible. Do not embed your content deep in a folder structure more than a couple nested folders. If possible and you have a simple site, try and keep all your web pages in the root directory.

This is because Google prioritizes content based on folder level. All other things being equal, a web page in the root of your web site is seen as more important as a web page nested a folder or more deep.

Just like in real life with people, first impressions mean a lot to Google. If you are launching a new web site, make it count.

Try and get a few quality links to your web site from other established, content related web sites. Make sure you have a well organized, search engine friendly web site with atleast 5 pages of optimized and related content. Finally, make sure your domain name is registered for a term of atleast a few years. All of these factors combined will contribute to a decent starting page rank and could avoid the dreaded Google sandbox.

Yes, I know. Pagerank is not as an important factor as it used to be but it’s still a solid indicator from Google as to the relevance and importance of your web site. Its a lot easier to obtain a higher pagerank down the road when you start off at a 5 or better right off the bat.

There is no reason why every single site shouldn’t be in dmoz. After all its free. Yes, it does take some patience and time to get category editors to list your web site and if your site is spammy in nature, chances are they won’t list it (and rightfully so).

The effort of listing your site can help you build quality inbound links. Not only that but one link at dmoz is worth atleast two because Google Directory also uses the content for its own dmoz based directory. So take the time and effort and make sure you are in dmoz. You can also find local directories that will provide back links to your web site such as Find Here.

One seo tip that is often overlooked is domain name registration length. Google looks to see how long a domain name is registered and will give you a higher pagerank because of it. If you are committed to your web site and Internet presence then there is no reason why you can’t justify registering your domain name for 5 or 10 years. Not only will this give you a boost in Google but it will also secure your web identity for years to come and avoid having to renew your domain year after year.