Programmatic Data Aggregation

About Programmatic Data Aggregation

One of the most important effects of the drop in price for computing power and storage is automation and data storage. When combined you get programmatic or automatic data aggregation.

Imagine a computer that pulls in copies of websites and databases all day and all night long without ever getting tired.

In fact it is possible that your group or network may have use of this kind of functionality. When you pull in content and data from other sources and republish it there can be a significant benefit because various search engines perceive your re-publication as the addition of new content to your site.

Besides this easy form of gaming the search engines, having loads of data at your disposal can keep your network more informed, and give them an opportunity to integrate new information, processing and finding relevance, within the context of your own organization, as opposed to solitary web browsing and the efforts required to re-publish and share the information.

The evolution of open data has made the opportunity for aggregation skyrocket. If you are so inclined, there's a good chance that you can make this work for you in some way.

Listed below are options you can use for your own aggregator.

Options

Drupal Options

• Feeds Module - http://drupal.org/project/feeds