You can disable sending referrers in Firefox with this method:

1. Get Firefox.
2. Install the Web Developer Toolbar
3. Restart Firefox.

On the Web Developer Toolbar, click on Disable and then on Disable Referrers and your browser will no longer send referrers.

Sometimes you will have to turn referrers back on to leave comments in certain blogs, depending on what kind of spam detection software the blogs are running.

Here is a screenshot of how to disable sending referrers on Firefox with the Web Developer Toolbar.

Related posts:

1. 12 1/2 Essential Firefox Extensions for SEO
2. MSN Keyword Referrer Spam Continues, Distorting Analytics Everywhere
3. Are Your WordPress Plugins Secretly Injecting Links in Your Blog?

UPDATE: I recommend just installing vbSEO instead of the solution below because you can add Google Analytics directly through the vbSEO interface. vbSEO also adds many other critical SEO improvements to the basic vBulletin installation. However, the solution below can still be used to add other types of tracking code to vBulletin, like Clicky Stats, and excellent free Web 2.0 stats program.

Here are some step-by-step instructions for adding Google Analytics (or other tracking code, like Clicky) to vBulletin:

In your vBulletin admin control panel, click on Style Manager and navigate your way through the options to “edit templates”. Double click on the footer template. Add your Google Analytics or other tracking code right at the bottom of the footer template. That will enable tracking for the main part of your forum.

To get the tracking code into the archives, click on “Add New Plugin”, and from the select box choose archive_complete. Enter the following code in the textarea and make the plugin active:

$output .= "<script src=\"http://www.google-analytics.com/urchin.js\" type=\"text/javascript\">
</script>
<script type=\"text/javascript\">
_uacct = \"UA-XXXXXXX-X\";
urchinTracker();
</script>";

If you are using another analytics system, just replace the Google Analytics code with it, being sure to escape all double quotes with a backslash.

vBSEO

Also, if you are using vBulletin, I highly recommend using vBSEO to optimize your forums. vBSEO will create clean, search engine friendly URLs on your forum and 301 redirect the old URLs to the new URLs. It will also provide other SEO benefits like remove the redirects in RSS feeds, remove the links to /index.php, and add trackback and pinging features. vBSEO can also add Google Analytics code to your forum…

Related posts:

1. How to Install Linux on an External Hard Drive
2. How to Extract Bot Activity From Logfiles
3. Is Rook Analytics Metapredict?

The hits come from IP addresses beginning with 65.55.165.* and the referrer strings have this format:
http://search.live.com/results.aspx?q=firefox&mrt=en-us&FORM=LIVSOP

The keyword (highlighted above) is always one word, and usually appears on the Web site somewhere, with the exception of the porn related keywords.

MSNdude posted a comment on the Webmaster World thread about this issue:

Thanks for all the feedback on this thread.

First, we appreciate the concerns and issues that have been raised and apologize for any incovenience this might have caused.

Second, we want to explain what this is all about. The traffic you are seeing is part of a quality check we run on selected pages. While we work on
addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index.

Please keep the feedback and thoughts coming as we will use this to help improve this process and make sure that it impacts your sites as little as possible.

thanks
- msndude (msd)

How can MSN call this a “quality check”? They are sometime using explicit porn keywords that do not appear anywhere on my sites.

I recommend reading through the entire Webmaster World thread about this MSN referrer issue.

I wonder if MSN is doing this for old-fashioned referrer spam reasons. The keywords that they use in the referrer string are related to the site’s content. So Microsoft sends keywords that they think the webmaster might be interested in so that the webmaster will click through to MSN Live Search.

The reason that I suspect that it might be traditional referrer spam is because MSN is trying very hard to get people to use MSN Live Search, even to the point of buying Google AdWords. Every time you click through your logs to MSN Live Search, it boosts Live.com’s traffic stats.

Also check out the post about it on andrewu.co.uk, Odd Referrer Spam-Like Behaviour From Microsoft.

If you notice this behavior from Microsoft’s bots on your sites blog about it and let people know about the problem.

I’m going to call this referrer spam until MSN speaks out about the issue and stops pinging my logs with porn keywords. They seem to have been silent except for that one odd comment on Webmaster World by “msndude”.

Update: MSN continues assault on logfiles

I also recommend reading this excellent smackdown which gives details about how this Microsoft bot is even downloading Adsense JavaScript from Web sites.

Related posts:

1. MSN Keyword Referrer Spam Continues, Distorting Analytics Everywhere
2. MSN Live Search Only Has Partial Support for Wildcards in Robots.txt
3. MSN Live Search Indexing URL Fragments

Google Analytics and other JavaScript-based analytics programs cannot capture data about robot activity on a site. But there is an easy way to get data about crawler activity on a site by grepping (searching) raw log files and analysing the data in a spreadsheet.

This technique works best in Linux or Mac OS X. I’m only familiar with Linux so the example below assumes that you are running Ubuntu or another Debian-based GNU/Linux distro. If you don’t have Linux and can’t install it somewhere, you could also run the commands directly on your remote Linux server over SSH and then download the resulting CSV file.

Open up a terminal and navigate to the directory where your raw logfile is located. In this case the raw logfile is called access_log.

Then type the following line:

egrep '(Googlebot|Yahoo!|msnbot)' access_log | cut -d' ' -f1-30 --output-delimiter=, >search_engines.csv

You will then have a CSV file of search engine activity that you can open in a spreadsheet for analysis.

This is what the script does:

1. egrep is used to extract all lines from the logfile that contain Googlebot, Yahoo!, or msnbot. You can add other bots if you want their data also. access_log is the name of the logfile and can be changed to whatever your logfile is named.
2. cut is a command that splits the logfile into columns using a space as the column delimiter (-d' '). It extracts fields/columns 1 to 30 (-f1-30), though there are probably about 20 to 25 columns. It then reassembles the columns with a comma as the output delimiter.
3. The script then saves the output as a CSV file called search_engines.csv.

Once the CSV file is open in a spreadsheet, delete the columns that you don’t want. Then go to the menu item Data —> Filter —> Auto Filter. You will then be able to sort the output by search engine and by response code. For example, you could filter the data to see only 302 response codes that Googlebot encountered. Or 404 errors that msnbot encountered.

This is just a crude solution that should work for general log files. You can customize it if necessary. Many IIS logs are not constructed well and need some other manipulation, but that is a post for later.

Related posts:

1. How to Add Google Analytics To vBulletin (including archives)
2. How to Find Thousands of Potential EDU Backlinks with GNU/Linux

I couldn’t remember what the script was so I typed cetrk.com into Google and couldn’t find any useful information. I eventually remembered that it was a CrazyEgg tracker and I decided to post about it so that other people could find information about it.

CrazyEgg is a JavaScript tracker that among other features shows a nice heatmap overlay on your pages that show where visitors are clicking:

The basic version with a limited number of page views is free at crazyegg.com.

Related posts:

1. Propeller.com & Cialis
2. 12 1/2 Essential Firefox Extensions for SEO
3. Google Search URL Parameters