Development Seed Blog

Tools That Show What's Happening in a Glance

One of the features that sets the Managing News system apart from other aggregators is that it makes it easy to act on information. It does this by showing you the big picture of what’s going on within the system over different timeframes.

The latest graph we plan to integrate into Managing News is a line graph that will show the terms (with the highest and lowest rate of change) that have risen or fallen fastest over a period of time. This will quickly show you the newest hot topics and what is already old news. A big benefit of this graph is that it will cut out issues that are always talked about in your field.

Currently we have a bar graph on Managing News that shows the most talked about topics today, this week, this month, and in the past six months. Here’s a screenshot of it on the reproductive health organization RH Reality Check’s Managing News.

As you can see in the above graphs, Iraq is consistently talked about, and Iraq has rather consistently been talked about since the war began four years ago. But that’s likely not the news that’s going to influence your daily communications and outreach (unless there’s a change or policy shift). That’s why we’re planning to switch to the first graph – to really hone in and show the topics that are spiking up and down.

It’s also easy to see what topics are most significant to the team or organization running Managing News. Just look at the tag cloud.

This shows the top 20 most talked about terms by the sources in your Managing News. Currently you can look at this information over four different time periods: day, week, month, and six months. However we plan to nix the different time periods, at least temporarily, because it’s slowing down the speed at which the graphs can update themselves. Instead of showing tags over the time periods, we plan to show the most popular tags of all time.

As we use Managing News on larger scales, we’ve seen that the graphs are taking a longer time to load and update themselves. Currently it can take up to a few minutes for graphs to update themselves with the latest information, and that’s way too long. One reason for this slow down is simply the sheer volume of content in Managing News systems, and that’s the nature of any aggregator. RH Reality Check’s system is brand new and already has 86,000 articles in it, and it’s only going to grow.

The second reason is that the process to pull information from Managing News into the graphs is complicated. Currently it works like this:

Step 1: Look for articles in the required time period in a potentially huge pool of entries (86,000 in the case of RH Reality Check) to determine the most popular terms.

Step 2: Add up all term relations (the package of an article with each of its tags, currently 640,000 for RH Reality Check) in these articles and figure out which one’s are the most popular.

Step 3: Plots the information in the graphs.

That’s for one time period. Our current graphs each have four. Simply put, this process takes awhile, and we don’t want to wait for it to run its course.

This week Alex will be working on implementing an idea he has to simplify this process. He’ll set up Managing News to keep content cached in chronological order depending on the publish date of the article as the articles are pulled into the system. This means that when the graphs are pulling information, they can skip step 1. This change should make the two new graphs we’re adding to Managing News work quickly.

What do you think of the new graphing ideas? We’d love to hear your thoughts on what information would most help your team and what you think of our plan to forego different time periods to improve speed.