How to stop and remove spam bot referrals showing up in your Google Analytics account? Have you ever logged in to your Analytics account and seen a massive sudden spike in sessions? At a glance this could also make you think that your site has had a massive influx of customers and therefore you would be hoping to make more sales. However, on a closer inspection at the source of these sessions you may see a massive spike in referral traffic coming from sites that you don’t know. Many times these sources are from what is known in the industry as “spam bots”. Of recent more and more websites are being crawled by these referrer spam bots, making it appear as though more people have visited your website. These negative spam seo bots can also skew your Analytics data by a considerable amount, especially as these bots can appear as though they have visited many times, have a 100% bounce rate and provide no other positive data, which in turn can throw out all of your other data (e.g. conversion rate, pages per session etc.) What is a spam bot? A ‘bot’ is an automated program/engine that executes actions without the need for manual input. They’re written to carry out repetitive jobs that would take a real person hours/days/months to complete, whilst taking the bot just minutes. In this particular blog, the referral bots are created to visit (or crawl) your website for one reason or another. The difference in legitimate bots and illegitimate ones The genuine bots are mostly from search engines such as Google or Bing and crawl your website to index your site, to show all your pages in their search results. All bots have the ability to disable analytics tracking which is why these ‘good’ bots won’t appear in your results, the spam bots intentionally leave the tracking in. To understand why the spam bots do this, we need to look at who is causing these spam referrals. The Most Common Spam Referral Bots Are:
free-share-buttons.com social-buttons.com event-tracking.com get-free-traffic-now.com semalt.com
There are several variations around these sites but the intent is simple; their URL appears in your Analytics, your curiosity makes you follow their URL to find out what they’re about, which then gets them free advertising that you have had to work for. The services these sites offer are illegitimate. For example, one spam referral you may see is called see-your-website-here.com. As the name suggests, they can get your own website to appear in other people’s Analytics results by paying them a fee. Leading US digital design, development and strategy firm Viget, have created a list of the most common spam referral bot sites by TLD and country code. This list is constantly updated. Why are spam referrals a problem? Crawling uses up your website’s resources and can slow the website down. Firstly, a slower website can negatively affect how your website ranks, as Google factors in site speed into its ranking algorithm. Secondly, if your website slows down at any point it can increase your bounce rate and lower your conversion rate (and performance in general). Happily, Google lets you choose what time of day and how often it should crawl your site. That allows you to set a time when you have the least amount of visitors, causing the least amount of disruption to your real visitors. (We can help you set that time of day through Google Webmaster Tools)
How can I find out if I have session visits from referral spam? Once you have logged in to your Analytics account click ‘Acquisition’ on the left hand side menu, then ‘All Traffic’, and then ‘Referrals’. Once you’ve loaded that page the data will include ‘Source’ in the first column. In that column you’ll probably be able to spot some of the examples we mentioned earlier, or a variation on them. You may have several pages of sources – Click the right arrow at the bottom right of your data to see more sources
How to hide and block spam referrals in Analytics The quickest and easiest fix is to hide these websites from showing in your Analytics data, which will prevent your data from being skewed. To do this we need to create an advanced filter in analytics, but don’t worry, it’s not as complicated as it sounds once you know how. We’ve kept it as simple as possible with this step by step guide: 1. 2. 3. 4. 5. 6.
Log in to your Analytics account Click Admin in the top menu From the Account then Property drop down menus, select the account/website you want to work on Under the View drop down select Create new view Under Reporting View Name put a simple memorable name such as Spam Free Ensure the Reporting Time Zone is correct for your time zone, then click Create View
7. 8. 9. 10. 11. 12.
This will take you back to the Admin screen. In the View drop down menu select your Spam Free view Click Filters Click + New Filter Name the filter something like Exclude Spam Bots Under Filter Type select Custom, then under the Filter Field drop down select Referral In Filter Pattern we want to include the illegitimate websites. Earlier on in this blog we gave five example spam websites. Here they are in the format we need for this filter: free-share-buttons\.com|social-buttoms\.com|event-tracking\.com|get-free-traffic-now\.com|semalt\.com The filter needs to have no spaces and before each ‘.’ needs to be a ‘\’. Each URL is separated by a ‘|’ 13. Click Save and you’re done We created a new view called Spam Free so that it can run alongside your normal view. It’s good practice to have three views – one that is ‘straight out of the box’, i.e. with no changes in terms of filters etc. The second view is to test changes that you may want to make, such as a filter. The third view is to copy any tested (and working) changes over to. As this is probably going to be your first change, we can go straight to creating the Spam Free view without the need for the testing view. The real fix to remove spam bots Some of these spam referral websites are repeat offenders – you’ll see them pop up in your referral traffic regularly. Others will be one hit wonders. The benefit of the above method is that it’s quick to do, however there are several draw backs. This method has to be monitored and updated regularly for it to keep being effective. If a new spam referral pops up the filter needs to be updated to include the new referrals. The other main downside is that these bad bots are still visiting your site, we’ve just hidden them from Analytics data. That means that they will still be taking up your server’s resources and impacting your website’s load speed. The good news is that we have a fix for that too, the bad news is that it’s a little more complicated than the previous method and will require a reasonable understanding of the syntax involved in .htaccess files (assuming you’re on an Apache server). If this bit has already turned into complete jargon then no need to read on, you can just contact us and we can do the hard work for you! Using .htaccess to block spam websites Before we go on with the explanation, it’s worth pointing out that your .htaccess file is very powerful and incorrectly completing the arguments can block your entire website. The benefit is that this stops each website listed visiting your website completely. For this explanation we are using the same five example spam referrals we mentioned at the start of this blog. This is how your .htaccess file should look after including these sites:
RewriteEngine on RewriteCond %{HTTP_REFERER} ^http://.*free-share-buttons\.com/ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*social-buttons\.com/ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*event-tracking\.com/ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*get-free-traffic-now \.com/ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*semalt\.com/ [NC] RewriteRule ^(.*)$ – [F,L]
You will have other lines of code in your .htaccess file already, possibly including the RewriteEngine on line. That only needs to be in your .htaccess once and should appear in the 1st line of the file. The final line above should be written as is. Upload this new .htaccess file to the root on your domain and you’re done. No more spam referrals! As with the advanced filter in Analytics, you’ll need to update this as often as possible but that only takes 5 minutes once you’re familiar with how to do it. Do you have issues with spam referral sessions in your analytics? Do you see massive spikes in Google Analytics traffic? Do you want to fix those annoying spam referral issues once and for all, but don’t know where to start? If you answer yes to any of these then we can help you fix these issues. Visit Swindon digital marketing agency toinfinity for more information or call 01793 357230.