Why Newswire?

Newswire is a peer-to-peer, fully decentralized system that brings news to your desktop, within seconds after it is published.  This technology gives the community the power to weave a collaborative infrastructure for the delivery of essential information to individuals in a robust, scalable and secure way. Newswire is a survivable system which will deliver news to subscribers even if large parts of the infrastructure are under attack or stress.

The development of Newswire followed discussions after the 9/11 attacks when flash-crowd effects made it impossible for many to reach essential news sites. Email and weblogs proved more effective technologies in that situation, although both still suffer from the centralized risks of overload and single-point-of-failure. A more robust approach is to use a peer-to-peer structuring of the system and provide strong incentives for people to collaborate on the delivery of information to everyone by giving up a bit of bandwidth and CPU cycles.

Newswire has the additional effect that it can significantly reduce the load on the websites carrying real-time news information by providing hints about when information has changed. This would change the frequent redundant poll behavior seen by many news sites into more effective visits.

What is in Newswire

Newswire consist of a set of software components packaged as a single application. The main application functionality allows the user to subscribe to a potentially large number of publishers, and to receive (short) messages from these publishers.  The message content may range from complete articles to short descriptions with links, similar to the individual entries in RSS feeds. Currently the only UI is a windows based outlook-style client, but other cross-platform UIs are possible as the core components are portable. See the screenshots for a view of an early alpha client.

Under the covers Newswire contains a distributed state sharing engine which uses epidemic communication to synchronize with other (local) participations. It maintains information about subscriptions and up-to-date network load information from local participants and aggregates of this information from other nearby zones and some remote zones. The messages are forwarded over a soap based overlay network where the paths are dynamically maintained using the collected network statistics  to ensure a reasonable balancing of the forwarding load. The system makes extensive use of public key technology to ensure that only messages are handled from authorized publishers and that the each participant can verify the origin of a message.

The Newswire Technology

What is the technology that makes Newswire so special? At the core is epidemic communication and state management to maintain distributed knowledge about  subscriptions, participant network capabilities and forwarding load. The epidemic technology was first developed by Alan Demers and friends at PARC, but has been completely revised at Cornell in the past years

The structuring into zones and virtual hierarchies is based on the small worlds phenomenon. When you organize participants into  small groups of local participants with some knowledge of other remote nodes you can construct very effective routes in a decentralized, autonomous manner.

These technologies are used to build a loosely coupled, dynamic overlay network, that continuously monitors network load and capacities to achieve a fair load.

Subscription information stored in Bloom Filters is aggregated such that simple forwarding decisions can be made anywhere in the network based on the location of the publication and the direction where possible subscriber are localed.

Software Availability

Our goal is to distribute Newswire as widely as possible to establish a global community that collaborates on real-time news delivery to every desktop. At this moment we are still operating in "internal" mode, but our first distribution limited to the Cornell community is targeted for early 2003. Concurrent with this we will work with a group of beta-users to test the wide-area deployment.

We are aiming to do a first open public beta distribution somewhere in the spring of 2003.