Academic Computing Today

The changing landscape

My role in academic computing changed when I left Carnegie Mellon in 1995. After seventeen years as a computing director I became a user again. With the major exception of digital libraries, I was no longer an insider. This section describes some of the trends that I see as an outsider. Undoubtedly my different viewpoint has obscured important developments, but there seems to have been a fundamental change in academic computing.

The underlying theme of the early years of academic computing was that universities were different. Because the computing industry was not providing the systems that they needed, they built their own. More recently, however, end-user computing has become universal and academic computing has merged back into the commercial mainstream. The projects of the 1980s brought an end to the period in which universities created their own computing environments. The excellent computing that Cornell now provides for its faculty and students is built almost entirely upon standard components. The computer on my lap, the networks it connects to, the programs that I run, the email and web servers are commercial products that are available to everybody.

In 1990, in a review of the Andrew project, I wrote:

"As we begin the next decade, nothing on the horizon suggests any change as dramatic as the move from timesharing. The Andrew environment, with personal computers and shared information, appears to be the right model for the years ahead. Many skilled people are needed to refine and improve this environment, but assuredly the next few years will see ever improving computing, which will be based around cheaper and more powerful hardware, better system software, more elegant applications, more dependable service, and a higher level of sophistication amongst users. The university will continue to wrestle with the organizational and financial issues, and will eventually find a balance between decentralized control and university-wide coordination.

"However, the next decade will see much more than an elegant completion to the Andrew project. If we simply extend hardware trends for ten years, we can see that personal computers will execute more than 100 million instructions per second and have screens so good that reading from them will be as pleasant as a printed book, that much of the campus network will be operating at speeds measured in gigabits per second, that it will be cheaper to store almost everything on computer disks than on paper, and that the distinction between video technology and computing will be blurred. These are more than quantitative changes; they are opportunities to open up new areas. The university has vigorous programs in many areas which are poised to take advantage of more powerful computing. These include electronic libraries, speech recognition, natural language processing, chess playing, image processing, and extensive areas of scientific computation. The next decade will see many of these move into the every day life of the campus. Surely, also, the next decade will see developments in new areas that nobody yet foresees."

The final sentence of the quotation has proved abundantly correct. Nobody predicted the uses that would be made of this new world of computing. Less than five years after this sentence was written, the web began its breakneck expansion. Soon afterwards mobile computing began its equally dizzy growth. Both were predicted in general terms, but nobody could imagine their impact. Fifty years ago computing was for highly skilled specialists; twenty five years ago it was available to the members of well-funded organizations such as universities; today it is universal.

The web

Universities did not create the web though they were significant contributors. In the early 1990s there were several competing systems for distributing information on the Internet. They included Gopher from the University of Minnesota, the World Wide Web from CERN in Geneva, Z39.50 from the library community, and WAIS by Brewster Kahle which used a modified version of Z39.50. In retrospect, the greatest strength of the web was its simplicity. Z39.50 failed because it tried to do too much. It assumed that nobody would place valuable information online unless it was protected by an authorization system. The early web was particularly easy to use. I personally created Carnegie Mellon's first home page in about an hour.

The browser that established the popularity of the web was Mosaic from the University of Illinois, released in 1993. It brought proper fonts, color, and images to the Internet and people loved it. Mosaic was initially developed for Unix workstations but it was rapidly ported to all standard personal computers. The same group at the University of Illinois also developed the web server that is now called Apache.

Most of the web search services, except Altavista, began as university projects or used university search engines, including Infoseek, Lycos, Yahoo (which began as a catalog), and Google, but they all rapidly became start-up companies. More recently, Facebook began at a university but it quickly became a start-up company. MIT was the founder of the World Wide Web Consortium, which has played a major role in standardizing and enhancing the web technology, but overall, universities have been users of the web rather than the creators of the technology.

Open source software

It is hard to exaggerate the importance of open source software in academic computing. Many articles have stressed the benefits of collaboration and the excellence of the best open source software, such as the Linux operating system, Eclipse development environment, Apache web server, Python programming language, Lucene search engine, and the Hadoop distributed file system and map/reduce engine. The zero cost is of course important, but the availability is even more so.

In my software course at Cornell, I never worry that the students do not have access to the software that they need. Students can download the open source packages on to their own computers and be up and running immediately. Many of the best students never buy any software. They rely on what comes with their computer and what they can download for free. We are educating a generation of students who are experts in the open source packages, but have little knowledge of the commercial alternatives.

Rather surprisingly, universities are not very active in creating open source software. All of the packages listed above are maintained by not-for-profit organizations. Individuals from universities may be contributors, but they get no academic reward for such activities, whereas many corporations contribute staff time as a substitute for building their own software. For instance, Hadoop provides companies that lack Google's expertise with an alternative platform on which to build very large Internet services. Facebook could never have grown so fast without it.

University research and spin-off companies

This is a remarkable time for technology transfer from computing research to the outside world. Ideas developed in universities are making their way into the marketplace very rapidly. Computer science and electrical engineering are in a particularly productive period where many long-standing research areas have matured into practical products, but people in all disciplines are finding high powered computing to be a never-ending source of innovation and entrepreneurship.

A dominant theme is the spin-off, a start-up company that individuals create based on their university research. Spin-offs are not new. As early as the 1950s Stanford University was encouraging the growth of Silicon Valley while Route 128 around Boston was synonymous with spin-offs from MIT and Harvard. In recent years, the pace has changed as venture capitalists became ever more willing to fund start-ups early in their life cycle. Sun Microsystems, which combined expertise from Stanford and Berkeley, was a good example. Mosaic was developed at the University of Illinois, but a year later, moved to a start-up, Netscape. Search engines such as Lycos (Carnegie Mellon) and Google (Stanford) began as research projects but rapidly formed spin-offs.

Over-exuberance about the potential of the Internet led to a speculative bubble, the dot.com boom, and its stock-market crash in 2000, but the success of companies such as Microsoft, Amazon, Google, and Facebook are an inspiration for individuals to turn their ideas into products and bring them to market. Each of these four companies continued to be led by its technical founders. In the past there was an attitude that companies should be led by people with business and financial backgrounds, but today's students have realized that founders who come from a technical background can be successful entrepreneurs.