Xoops Tips - TIPS - Webmaster - Prevent Prefetching

Webmaster : Prevent Prefetching

Posted by tl001 on 2005/12/8 7:20:36 (4006 reads)

One of the most STUPID and annoying creations by Mozilla/Firefox is prefetching. IMHO, one day because of this idiotic feature, Firefox may end up being banned by most webmasters. Unless, M$ stupid enough to follow the mistakes of Mozilla.

A browser (firefox) with prefetching option ON or with Goolge accelerator/prefetcher will download every links founded on your site. It behaves exactly like a bad/rogue crawler trying to crawl every links of your site.

Prefetching is a nightmire for any web sites, especially small ones with limited bandwidth and resources. To stop prefetching, you may want to add the following lines of codes into header.php

before
include_once XOOPS_ROOT_PATH.'/class/xoopsblock.php';

//stop prefetching
if ((isset($_SERVER['HTTP_X_MOZ'])) && ($_SERVER['HTTP_X_MOZ'] == 'prefetch')) {
    header('HTTP/1.0 403 Forbidden');
    echo '403: Forbidden<br><br>Prefetching not allowed.';
    exit;
}

To add an extra protection, you may also want to add the following lines into .htaccess (before your RewriteRule)
Quote:

RewriteCond %{HTTP:x-moz} ^prefetch [OR]
RewriteCond %{X-moz} ^prefetch

The codes have been tested to be effective. Corrections, sugguestions or improvements are welcome.

Reference:
http://www.ilovejackdaniels.com/php/block-prefetching/

The comments are owned by the poster. We aren't responsible for their content.

Poster	Thread
Chappy	Posted: 2005/12/11 1:13 Updated: 2005/12/11 1:13
Just popping in Joined: 2004/8/17 From: Rowlett, TX Posts: 20	Re: Prevent Prefetching I implemented this and it seems to work fine...as far as I can tell. No fuss, no mess... Thanks for the heads up...

tl001	Posted: 2005/12/31 11:42 Updated: 2005/12/31 11:43
Webmaster Joined: 2004/6/10 From: Posts: 282	Re: Prevent Prefetching It seems the codes added into header.php are not as effective as we would have expected. The following codes seem to be working. (remove the codes in header.php) Add a new rule into .htaccess (your server must allow mod_rewrite) `#new rule - ban prefetch RewriteCond %{HTTP_USER_AGENT} "Firefox/1.0.7" [OR] RewriteCond %{HTTP:x-moz} ^prefetch [OR] RewriteCond %{X-moz} ^prefetch RewriteRule (.*) noaccess_to_prefetch_accelerator.php [L]` Please note that noaccess_to_prefetch_accelerator.php is a non-existant file so the server would return a 404 code to the browser. Based on reading from the net, this seems to be the proper way of doing it. It will stop Firefox 1.0.7 and most times Firefox 1.5 with Prefetch ON. It still does not stop those browsers with accelerator ON, but the server will return 404. It will alleviate the heavy pressure on the datasebase because of the rapid-fire database queries generated by the evil prefetch. It is not perfect - it would be if Firefox 1.5 with or without prefetch were to be banned altogether. Sometimes, ...

sven423	Posted: 2006/2/21 3:42 Updated: 2006/2/21 3:42
Just popping in Joined: 2006/2/21 From: Posts: 1	Re: Prevent Prefetching Well, I just had to register to comment on this... Have a look at: http://www.mozilla.org/projects/netlib/Link_Prefetching_FAQ.html There you can see: the webmaster is fully able to control which sites are being prefetched by using rel="prefetch" in link-tags. Ones without rel="prefetch" are not going to be prefetched. So why do you want to block Firefox? People on other sites (http://www.entartete-kunst.com/?p=657) are already starting to complain about that. Do you really want to block 20 per cent of your visitors? The only thing left is that Google uses prefetching for the first result in its searches. and that's a really nice thing which makes surfing a lot faster, imho. I'm also a webmaster of several sites. And I don't care about this little bit of traffic due to prefetching if it makes the web better for users.

tl001	Posted: 2006/2/21 8:44 Updated: 2006/2/21 14:52
Webmaster Joined: 2004/6/10 From: Posts: 282	Re: Prevent Prefetching Quote: There you can see: the webmaster is fully able to control which sites are being prefetched by using rel="prefetch" in link-tags. Ones without rel="prefetch" are not going to be prefetched. How so? Google/Firefox must be extremely nice to you than to me?? This site has the simplifiedURLs implemented and prefetch was crawling the site like a rapid-fire extremly bad behaving crawler and even worse, some of the prefetching totally disregarded previously crawled contents. Quote: So why do you want to block Firefox? People on other sites (http://www.entartete-kunst.com/?p=657) are already starting to complain about that. Do you really want to block 20 per cent of your visitors? I am not blocking firefox, only firefox with prefetching ON. If it would take banning firefox to save myself from being banned by my host, then I would do that. A fast and speedy response depends on two factors: your Internet connection and server specs. Nowadays, almost everyone has a broadband connection and many of the servers are pretty fast. Even prefetching does work, what would you gain while in the process of making a mess with the host server? Those using prefetching, to me, are a selfish bunch with no regard what their actions would do to the servers. Losing those visitors is not a concern to me. To increase servers' responding speed is the responsibility of the webmasters. NOT Google's and Firefox's! So hands off.