Development Seed Blog

We just finished our beta work on "Managing News," our name for a team aggregator that allows entire organizations to track, manage, analyze, and act on news. It's like a corporate version of Digg meets Bloglines that allows an entire team to monitor news together.

We'll demonstrate the system and the suite of tools we built for Drupal-powered intranets tonight at the Drupal meetup here in Washington, DC. Stop by the Science Club at 7:00 pm to see the system in action. We'll also make a dataset available of news coverage of eight key congressional and senate races leading up to last month's election. We thought the political folks in DC would find this interesting.

Analyzing and Acting on News

The set of Drupal modules that makes up Managing News is the result of two very exciting intranet projects we worked on with The World Bank (whose intranet is named Buzz Monitor) and World Resource Institute. Our goal was to create a system that saves staff time when tracking news, makes it easy to get news from nontraditional sources, and allows team members to stay up-to-date with all the websites and media sources they track to do their job well.

But Managing News is not just about tracking news. The system is designed to make it easier for people to act on news and push it out to the people who need to know what is happening. Tag clouds make it easy to visualize popular topics. Graphs on almost every page show the frequency of a term over the past 30 days and compare the frequency of multiple terms in the news.

Is it Joe's blog or is it Joe Trippi's blog? Each source shows a Technorati and Alexa ranking to help you quickly assess its value. Third party tool integration to services like Technorati, Alexa, and
Yahoo! allow the system to harness the best information by the best service providers and apply each service's added value as an analytical tool.

When you see a valuable article, the system allows you to act quickly. If you run a blog with a news section like NextBillion.net or even just an intranet, you can find articles by looking around Managing News and republish them on your website with just a click.

The system itself does a lot of organizing automatically. News articles and blog posts that come in from feeds are organized by date, pre-tagged
with Yahoo! Terms (a new content analysis service), and organized by their source. Managing News also allows teams to customize how you want
your articles organized by allowing them to tag, flag, and comment on each article.

Tag - when you read an article, you can tag it with a word or a phrase. For example, tag an article "to read" if you want to mark it to come
back to later, or "banking" if it is about banking. Users can add lots of tags to each article.

Comment - Want to put an article in context for one of your colleagues or just make a note for yourself to look over later? You can add
comments to every article.

Flag - If an article is significant, click the +1 button. Later colleagues read all the articles flagged with a +1 for any given day.

Teams Working Together

Teams have their own space in Managing News that allows them to meet up online and track news important to them.

For example, the NextBillion.net group who helped craft the notion of teams in the system might have a group to pool its feeds on specific
topics like "bottom of the pyramid" or "microfinance." Managing News lets them track all their feeds in one spot and use the space to talk
with each other about what to blog and from what angle. Additionally they can use the system to analyze trends and sources for them.

Groups don't need to fall along departments. The system can also organize teams around topics being covered by different departments at the same time. Say both Rob Katz from NextBillion.net and Sarah
Paraghamian from Electricity Governance Initiative want to track news
about solar power. While they are in different departments at WRI, they can set up a group on Managing News to include feeds that are important
to both of their teams and generate a space to interact and better communicate all with just a few clicks.

The Tools

We plan to release all the modules that we built for Managing
News. Alex Barth, the project's lead developer and the single reason the system works so well, has already started the release back to Drupal.

Drupal offers the base framework on which all functionality was added. On top of Drupal, the main modules that comprise the guts of the systems are

Leech - these modules take news feeds, parse them, and turn them into nodes, making them fully available to Drupal's robust node API

Leech_yahoo_terms – when news articles are turned into nodes, this module sends their text to Yahoo News, handles the response, and turns keywords into taxonomy terms (tags) for each article

Url_profile – this determines the original source of an article, boils it down to its baseurl, and follows up http redirects for finding a feed's base that came in from Feedburner, for example

• url_profile_technorati - this takes a baseurl and pulls information from Technorati using its API and the cosmos, bloginfo, and blogtags queries
• url_profile_alexa - this takes a baseurl and gets a thumbnail of the website and its traffic rank

These are the main modules that aggregate and profile news articles. We're currently cleaning up several other modules and documenting that we will release soon. It's also possible to build other functionality that uses third party services and existing and non-existing Drupal modules to further customize the system.

Working on an Awesome Project with Great Clients

Both the The World Bank and WRI uniquely contributed to the development of these tools. Pierre Wielezynski from The World Bank was like a new team member for us when it came to developing our strategy. He played a vital role in several of
the big source profiling ideas, like parsing out article sources from keyword feeds. Pierre also helped influence some of the exciting analytical tools that are incorporated into the site such as the Technorati and Alexa profiling and the different views graphing the activity of terms.

WRI took the system to a new level by helping to identify what was needed to have effective teamwork flows. Rob Katz and his team at NextBillion.net conceptualized how they needed to work together in
departments and across the organization to be more effective in getting news out. Not only did WRI help shape the way we architected the backend for teams but also had us build an awesome end point to the system - a republish system. You can click republish on any article and it will take you to a page where you can select the website you want to send it to and then blog directly from the Managing News system to have the content syndicated to the desired site via RSS. In the next few weeks WRI will finish up beta testing on the system and integrate it into their websites.

The Future

We did not build anything earth shattering. The notion of
internal team aggregators is gaining ground.

For example, we have seen tremendous fan fair and warm reviews of the Monitor110 system that does something similar to Managing News but is catered to hedge funds. Managing News is not Monitor110, which digs through 50 million websites. Managing News takes a more open, network-centric, Web 3.0 approach of plugging into existing intelligence systems like Technorati, Alexa, and Yahoo, which index hundreds of millions of websites. Future development could include more
user-generated and aggregated intelligence. More importantly, because of the open nature of Drupal and the Managing News system (it's all open source and GPL licensed), it is customizable to do exactly what you need and has the flexibility to adapt. Managing News is light and grows as you need it to.

We are continuing to work on this system, which directly correlates with our underlying mission to help progressive causes better communicate online. To be able to communicate organizations first need to listen. And to amplify their communications they need an offsite strategy that finds out who else is writing about what they are writing about and starts engaging them.

Our goal is to find investment from other progressive organizations that want to use this system. Hopefully other environmental or progressive political organizations will want to try this out in preparation for the national election in 2008. Financing will allow us to

• bug fix
• further document
• add on new feature requests
• upgrade to the new version of Drupal
• improve overall performance
• build an installer profile so Managing News can easily be deployed by other
  organizations

We'll see, but we're confident.