Setting Up a Curated News Aggregation Site with WordPress

WPMU DEV’s Autoblog plugin is a powerful tool that pulls in RSS feeds and publishes them to your site. Many may think that just means it’s a device for splogs (spam blogs).

While a tool like Autoblog could be used for that, there isn’t much point in it. Search engines learned long ago that those types of sites don’t add much, and so they’ve devalued them into near oblivion.

For most tools, their value is only fully realized in the right hands. And that’s the case for Autoblog.

The Autoblog plugin as a splog creator is pretty much worthless.

But the Autoblog plugin as tool for a curated news site, on the other hand, is pure gold.

Content is STILL King – Take Advantage of It

Content was still king, but many began seeing the advantage of sharing other people’s content, not just their own. If you became known as a reliable resource, then people would come back to you again and again.

Eventually the big buzzword for that type of activity became “curating.” In other words, you acted like a museum curator, picking and choosing the best things to show to others.

And in many ways, that’s where we still are today. You can still find great stuff from following good “curators” on places like Twitter, for example. But you probably miss a lot of good stuff too. Twitter moves fast, and things are gone in a flash. In addition, all the good nuggets for one niche are likely buried in the middle of a lot of other stuff. Even if there are ways to separate them out, the speed and the volume tend to overwhelm if you don’t stay on top of it 24/7.

 

Enter the Niche News Aggregation Site

Think about a few decent-sized niches that come to mind. Is there ONE site you can go to for ALL the major news or interesting articles about those niches?

If you answered yes to that, then you’re probably in a minority.

The truth is that a site like that is fairly rare when you consider all the possible niches there are. There may be lots of sites with the major news for a niche, but if you’re a very heavy consumer of information in that area, more than likely you’ve had to do something like set up a feed reader with RSS feeds from a number of different sites.

Feed readers are great. But the truth is a lot of people don’t go to the trouble of setting them up. Or they do go to the trouble, and then they forget about them. Who knows why exactly? Maybe it has something to do with the layout and presentation of the information.

In any case, the fact that not as many people use this fantastic invention as much as they might is actually an OPPORTUNITY.

If you could curate the best information from a niche and put it all on one attractive, well-thought-out site, you would have something indeed – a one-of-a-kind resource that anyone interested in that niche could come to whenever they wanted to get caught up with the latest and the greatest.

And so that’s exactly what we’re going to go over in this post – building a news aggregation site with WPMU DEV’s Autoblog plugin.

Is This Completely Automatic – Set It and Forget It?

I’m guessing that a number of you are already asking the question above – is this a “set it and forget it” solution?

While the Autoblog plugin can most certainly keep on chugging down whatever path you set it on, if you’re looking to build a site that truly offers value (and by doing so will actually attract a large number of visitors), then this is not going to be a complete set it and forget it endeavor.

If you’d like to build a valuable site, then you will need to be in there “curating” – deleting this or that, moving things around when appropriate, setting certain stories to be highlighted, possibly going out and finding better graphics for others, etc. Some may even choose to rewrite everything that gets pulled in before it gets published.

In other words, you might think of the Autoblog plugin as a researcher – a very powerful researcher that will do a lot of heavy lifting for you. It will pull in the raw material, but you will still need to do some shaping and massaging in order to make a coherent and quality presentation of the material.

 

What Will Google Think?

I know a lot of you are asking this question too – what will Google think?

You know what – who cares what Google thinks?

Seriously.

Google says do what’s best for your audience. If setting up a one-of-a-kind curated resource for your niche isn’t good for your audience, then what is?

The truth is, unless you are rewriting the content into your own words (and maybe not even then), you probably won’t see much love from Google or other search engines. At least not in the beginning. But when people begin to appreciate what you’re doing and start sending you attention in the form of links and social media buzz, then that may change, even if your content isn’t original.

It may not change, however. You’ll need to go into this with your eyes wide open.

Don’t expect love from Google, and don’t spend a lot of time courting it. It may never come.

Court visitors through more personal means – i.e. social media, word of mouth, email lists, even advertising.

If you build it, unfortunately, they won’t just come. You’ll have to convince them to come. But if you build it well, once you convince them to come, they’ll come again and again.

How to Set Up Feeds with the Autoblog Plugin

First, we will go over setting up your feeds with the Autoblog plugin.

In later sections, we will go over some tips for finding feeds and using other plugins that may complement the Autoblog plugin.

Getting It Right from the Beginning

When you are first setting up your feeds, you will want to make sure you have them the way you like them from the beginning. Once you process your feeds the first time, you will not be able to reprocess them again right away. And so if you make a mistake and process the feed, you will have to wait until new posts are created on the external sites and then processed again on your site.

For example, say you forgot to specify the number of words in the excerpt, and the feeds are then imported without any words at all, just titles. You can delete those posts, but even if you manually reprocess the feed again, it will show that you’ve already processed the most recent posts.

In fact, you may want to set up a test site first in order to figure out exactly how you’d like to set things up. Either that or practice processing feeds that you really don’t want to keep. You can then simply delete the created posts and the feeds themselves once you’ve gotten things the way you want them.

Messing something up at the beginning is not a fatal mistake by any means, but it will create a little added frustration that you can easily avoid by practicing first.

Step By Step

For many of you, setting up a feed will be extremely easy, and the short instructions on the Settings page will be all you’ll need. 

We encourage you to still at least take a quick look at Steps 1 & 2 below. After that, you can watch the video for a complete rundown of setting up a feed, or you can consult the text version for some steps that might not be completely obvious.

STEP 1: Add Categories

Depending on how you want to set your site up, you may want to go ahead and add some categories to your site as you normally would.

There is an option, however, to automatically create categories on the fly based on the categories from the feed sites (i.e. the sites where the posts are being imported from). This is up to you. If you choose this option, over time, you will likely end up with a large number of categories. So if this is something that concerns you, be sure to consider it from the beginning. (Of course, you can always change your mind later.)

STEP 2: Choose Add-ons

Once you have the plugin installed and activated, the first thing you’ll want to do is decide whether to activate any add-ons or not.

In my case, I’ve decided to activate the Featured Image Import and the Twitter Add-on add-ons. The Twitter Add-on converts Twitter @ mentions, for example, to links.

STEP 3: Insert Feeds

Below we’ll go over a few of the settings that might need a little more explanation.

For those who would like to see EACH setting explained, you can check out the following video:

9b3n-YBSz2w

Text Version of Potentially Confusing Steps:

Next, you’ll want to add new feeds by going to “All feeds” on the left and then clicking the “Add New” button.

Next, add a title for your feed. This is a title for this one individual feed only, and it will not be public. The point of this title is to help you easily identify it later in case you need to edit it or delete it.

For example, if I were to import the feed for WPMU.org’s Themes category, I would title that feed “WPMU Themes.”

STEP 4: Complete the Fields

Most of the options after this are pretty self-explanatory, but we’ll run through a few that might need a little bit more explanation.

Post type for new posts: This lets you choose whether you want to publish these feeds as a Post or something else, such as a Page. “Post” is the default choice here, and that’s what most people will probably go with.

Assign posts to this category: In order to choose a category here, you will need to have already set them up.

Treat feed categories as: When you import posts from other sites, they will have most likely been put into categories on those sites. That information appears in the feed you import.

This is the section that lets you automatically add categories to your site based on those categories from the feed sites.

You can also choose to turn those category names into tags on your site if you like. For example, if a post is put into a category called “Blue Themes” on the other site, a tag called “blue themes” will be added to your site (but not a category called Blue Themes).

Remember to check the box that says “Add any that do not exist” if you’d like to add these categories to you site either as categories or tags.

Link to the original source: In this spot, you can craft text that will link back to the original source.  Of course, you can make something generic. Or you could make something specific to this feed, such as “See the full post at WPMU.org.”

Process this feed: In this section, you decide how often the plugin should go out and check if there’s a new post available. If you choose “Never,” you can still process the feed manually (i.e. you can still press a button and it will for updates right then).

If you are getting the feed from a site that doesn’t update that often, the “everyday” option will suffice. 

Once you’re finished, just hit the “Add feed,” button. 

Best Practices for Pulling in Feeds

If you plan on building a reputable site, then the following are recommended:

• Ask permission to use the feed.
• Use excerpts, not the full post.
• Link back to the original post.
• Use a do-follow link.

Plugins to Compliment Autoblog

Using Default Thumbnails

While you can pull in images from a feed, many feeds do not provide images. When that’s the case, no thumbnails will appear with those posts, of course.

Some may not like this look of thumbnails with some posts but not with others. And if you’re one, then there’s a solution for that – a plugin called Default Thumbnail Plus.

This plugin lets you set up different default thumbnails for each category. If a post does not have a thumbnail associated with it, the plugin will add one.

Here’s a quick look at the settings page. You can see a site-wide default thumbnail (the WordPress “W”), and then you can see individual category thumbnails.

(Note: if you are using Google News as a source, they set their thumbnails at 80×80 pixels. You may want to consider this when using thumbnails on your site.) 

Duplicate Posts

If you are using multiple feeds from multiple sources, then there’s a chance you may end up with the exact same post twice. For example, say you are pulling a feed from a Blog X, and then you are pulling a feed from Google News. Google News may also be pulling from Blog X, thereby giving you duplicate posts.

The Super Post Cleaner  plugin can help with that. It will delete duplicate posts from the database, leaving the first post in place. 

 

Finding Feeds

Other WordPress Sites

If you are pulling feeds from other WordPress sites, then here are a few tips for finding feeds from those sites.

(Note, some sites may be using an RSS service that will redirect these default URLs, but these should work for most sites.)

The feed for the entire site:

www.example.com/feed/

The feed for an individual category:

www.example.com/category/category-name/feed/

The feed for an individual tag:

www.example.com/tag/tag-name/feed/

The feed for an individual author:

http://example.com/author/author-name/feed/

The feed for all comments:

http://example.com/comments/feed/

The feed for comments on a single post:

http://example.com//my-post-name/feed/  

(Note: just add /feed/ to the end of the URL for the post)

Google News

If you are pulling from Google News, you might consider using a search operator like “allintitle” in order to improve relevancy. The allintitle operator will only return stories that have the words you specify in the title.

An example of using the allintitle operator would look like this …

allintitle: wordpress plugins

In the case above, Google should only return stories with words “wordpress” and “plugins” in the title.

Once you have performed your search in Google News, you will be able to find the RSS link at the bottom of the page.

RSS Search Engines & Directories

Lastly, you can also search RSS search engines and/or directories for feeds. You can perform a search to find some of these sources.

Here are a few to get you started:

• http://ctrlq.org/rss/
• http://www.rsssearchhub.com/
• http://www.search4rss.com/

You’re the Brains, Autoblog is the Brawn

As stated before, using Autoblog to create splogs is a fairly worthless endeavor. Even if you did pick up some traffic here and there, eventually you would be axed by the search engines because no one is going to be linking to you or recommending you.

But if you let Autoblog do the heavy lifting by going out and doing your research for you, then you’ll begin to appreciate what a valuable tool it is. There are things bots are good at, and there are things brains are good at. Let each do what they do best.

Check out the Autoblog plugin here. It’s simple, flexible, and powerful.

Photo: RSS symbol,Gears from BigStock. Photo credit: yeowatzup, Billingham, meneame comunicaciones,

Tags:

• blogging