Development Seed Blog

New Flexible Aggregator for Drupal Ready for Use

After being in development for about seven months, we released FeedAPI 1.0 nearly two weeks ago! This is really exciting for me and everyone else who has been craving a more flexible aggregator for Drupal.

We wanted FeedAPI to be as flexible and as fast as possible, and I'm happy to say that the end result has reached those goals. All feeds are represented by nodes. Feed items remain independent from the node system - you can choose, whether you would like to store them as nodes or fast flat database records. Per default, FeedAPI is configured to use SimplePie as a feed parser, which works perfectly in most cases. But if you have particular requirements, you can switch to the alternative Common Syndication parser or write your own.

A particularity nice thing about FeedAPI is that you can configure your feed processing preferences on a per content type basis. Cron processes are performance optimized and work on a time limit rather than a number of feeds limit - a big relief for tuning cron performance. FeedAPI also collects important statistics for monitoring wicked huge aggregating sites. Additionally, FeedAPI has a straightforward configuration and is easily extensible through an API. (Just to know: The content-type preset (among others) is entirely Alex's concept and work, and Jose did a lot to make FeedAPI faster.)

In the near future, we plan to further tune FeedAPI's performance and make it more feature-rich (as I type this blog entry, we are working on committing new features that aren't included in 1.0 :) )

For now, I'd like to share with you what you can do with FeedAPI 1.0:

• Install it on a PHP4 or PHP5 site with a MySQL or a PostgreSQL database
• Cache feed downloading to save bandwidth and time
• Turn feed items into nodes
• Turn feed items into lightweight items – use FeedAPI like the good old core aggregator
• Adjust cron processing of feeds based on elapsed time. No more php timeouts
• Get statistics about feed processing
• With Feed Element Mapper, map feed item elements to CCK fields or taxonomy
• Import your feeds from anywhere with OPML import
• Export OPML
• Inherit taxonomy and organic group settings from feeds to feed items
• Views support, having a predefined views for exploring feed items for a specific feed
• Content-type based configurable presets
• Promote items to the front page according to your settings
• 1-click feed creating block. Just supply the feed url and the feed is created
• Purge or not purge outdated feed items – it's your choice
• Automate tests (simpletest module yeay!) - easy checking for problems on your installation.
• Auto-detect feed URL, you do not have to know the RSS/ATOM link, you can just supply the site URL, for example http://drupal.org
• Access to a documented API, you can write parsers or processors and use FeedAPI’s power in feed management
• Ready-to-translate .po files in the package
• +1: low turnaround time issue queue-based support :D

I started work on FeedAPI last summer as part of Google's Summer of Code. It's been really fun for me to see how the project has grown, and how far a project like this can go. During the Summer of Code, many people helped design the project with great suggestions. FeedAPI’s structure was originally dreamed up as the following: Get feed | Parsers | FeedAPI core | Processors . We also wanted user to only supply one url, and then FeedAPI would do an auto-detection of feed types (rss feed or an ical calendar or whatever), and none of the parts were to be designed to depend on the node system. In the end, some of these ideas proved to be too idealistic, so when we faced performance problems and the drawbacks of the abstraction, we changeed the direction a bit, which is how we got to the current version of FeedAPI. Then during the beta releases, I received several ready-to-commit patches for FeedAPI and tons of well-documented bug reports, which were all very helpful.

I would like to say a big thank you to everyone who helped to make FeedAPI better. These words are not just “saying some kind words at the end of the blog post.” Everyone who helped in the process really made FeedAPI better.

As I wrote in the release notes, we’re now seeking translators. So if your native language is not English, you can start translating today. Well, maybe I should set a good example and provide the Hungarian translation soon :)