Comments on: WPMU robots.txt done globally http://wpmu.org/wpmu-robotstxt-globally/ WordPress, Multisite & BuddyPress plugins, themes, news and reviews and special offers from the team at WPMU.org Thu, 09 Feb 2012 22:23:15 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Edmund http://wpmu.org/wpmu-robotstxt-globally/#comment-2527 Edmund Fri, 20 Mar 2009 17:02:38 +0000 http://wpmu.org/?p=364#comment-2527 It's pretty easy to activate the plugins in WordPress 2.71. It’s pretty easy to activate the plugins in WordPress 2.71.

]]> By: Purchase Domain http://wpmu.org/wpmu-robotstxt-globally/#comment-2153 Purchase Domain Sat, 14 Feb 2009 05:21:27 +0000 http://wpmu.org/?p=364#comment-2153 Thanks, there is more reason to comment than ever before! Thanks, there is more reason to comment than ever before!

]]> By: Annie Maloney http://wpmu.org/wpmu-robotstxt-globally/#comment-1928 Annie Maloney Tue, 27 Jan 2009 13:19:25 +0000 http://wpmu.org/?p=364#comment-1928 Nice Post. There are so many plugins out there and still so many people that are uneducated as to how they work and affect thier site, especially plugins that deal with the bots. Nice Post. There are so many plugins out there and still so many people that are uneducated as to how they work and affect thier site, especially plugins that deal with the bots.

]]> By: Foreclosure Pete http://wpmu.org/wpmu-robotstxt-globally/#comment-1850 Foreclosure Pete Tue, 13 Jan 2009 23:22:59 +0000 http://wpmu.org/?p=364#comment-1850 I agree that the correct SEO method is to use noindex follow using meta robots. There are so many seo things one needs to consider...oh boy I agree that the correct SEO method is to use noindex follow using meta robots. There are so many seo things one needs to consider…oh boy

]]> By: Robb http://wpmu.org/wpmu-robotstxt-globally/#comment-1372 Robb Sat, 08 Nov 2008 16:26:59 +0000 http://wpmu.org/?p=364#comment-1372 Will this work for only WP blogs hosted on the WP site, or also for those hosted on individual sites )like i have)? Will this work for only WP blogs hosted on the WP site, or also for those hosted on individual sites )like i have)?

]]> By: Andy Beard http://wpmu.org/wpmu-robotstxt-globally/#comment-1239 Andy Beard Thu, 16 Oct 2008 14:29:18 +0000 http://wpmu.org/?p=364#comment-1239 Your robots.txt example creates a ton of dangling pages. If you want pages out of the index and not sucking up "Google Juice" then the correct SEO method is to use noindex follow using meta robots. Your robots.txt example creates a ton of dangling pages. If you want pages out of the index and not sucking up “Google Juice” then the correct SEO method is to use noindex follow using meta robots.

]]> By: wep http://wpmu.org/wpmu-robotstxt-globally/#comment-1097 wep Fri, 03 Oct 2008 10:28:06 +0000 http://wpmu.org/?p=364#comment-1097 hi...sorry, i no understand, make one robots.txt in root directory and global.php in mu-plugins? hi…sorry, i no understand, make one robots.txt in root directory and global.php in mu-plugins?

]]> By: Jason http://wpmu.org/wpmu-robotstxt-globally/#comment-1026 Jason Mon, 29 Sep 2008 18:23:16 +0000 http://wpmu.org/?p=364#comment-1026 @Dr Mike: Yeah probably good call with the blogs.dir line. My impression was that there was no need for an Allow line anyway. Everything gets crawled unless you disallow it. I got that line from askapache.com So if you just have a robots.txt file in the root directory, then there is no customization as to the sitemap url? Maybe that's not needed either... As for the sitemap: I add a couple lines to my .htaccess file RewriteRule ^(.*/)?sitemap.xml wp-content/blogs.php?file=sitemap.xml [L] RewriteRule ^(.*/)?sitemap.xml.gz wp-content/blogs.php?file=sitemap.xml.gz [L] Then manually write the sitemaps to the appropiate blogs.dir folder. @Dr Mike:

Yeah probably good call with the blogs.dir line. My impression was that there was no need for an Allow line anyway. Everything gets crawled unless you disallow it. I got that line from askapache.com

So if you just have a robots.txt file in the root directory, then there is no customization as to the sitemap url? Maybe that’s not needed either…

As for the sitemap:

I add a couple lines to my .htaccess file

RewriteRule ^(.*/)?sitemap.xml wp-content/blogs.php?file=sitemap.xml [L]
RewriteRule ^(.*/)?sitemap.xml.gz wp-content/blogs.php?file=sitemap.xml.gz [L]

Then manually write the sitemaps to the appropiate blogs.dir folder.

]]> By: ovidiu http://wpmu.org/wpmu-robotstxt-globally/#comment-1004 ovidiu Mon, 29 Sep 2008 07:37:30 +0000 http://wpmu.org/?p=364#comment-1004 k, seems interesting but what sitemap plugin do you suggest using for wpmu? k, seems interesting but what sitemap plugin do you suggest using for wpmu?

]]> By: increase online traffic http://wpmu.org/wpmu-robotstxt-globally/#comment-1002 increase online traffic Mon, 29 Sep 2008 06:47:03 +0000 http://wpmu.org/?p=364#comment-1002 Will this also work on just regular wp sites? Will this also work on just regular wp sites?

]]>