When I wrote the first edition of this book, networks were hardly a VV hot topic. Few nontechnical people had any concept of what networks even were. Yet, in a short period of time, the Information Highway has become sufficiently popular to even figure in national politics. In the '90s, the Internet and the World Wide Web are achieving much of the same notoriety that personal computers achieved during the '80s. As a result, while few people understand what networks are really about, suddenly wide-area networks (WANs), providing the glue that ties computers together all over the world, are hot topics of conversation. Just as it took over a decade for personal computers to mature to the point where they truly were widely usable, global networks, while interesting to talk about, are still far from being ready for serious prime time. This chapter is about those WANs, where they come from, what they are truly useful for today, and where they are going in the future.
By interconnecting computers and local-area networks (LANs) wherever they may be located, WANs will convert the entire world into a global village, make telecommuting possible, change the meaning of the word office, and put information at your fingertips on a scale that today is literally unheard of. In the last chapter, you learned about LANs -- networks that are not really networks. This chapter is about real networks, the WANs.
136
To understand these real networks, you need to consider them from three perspectives. First, you'll look at the underlying network technology itself, both in terms of how networks evolved and in terms of how they work today. Second, you'll learn about electronic mail, the major application that has already caused a great deal of true cultural change. Third, you'll look at networks through the World Wide Web, bulletin boards, and internal applications like Lotus Notes. These networks are providing a form of community memory that builds on electronic mail to make the global village, at least at a corporate level, complete.
Finally, as you consider how all the various elements of the worldwide network fit together, you'll find cooperating components coming into the picture. Chapter 3 discussed the surprising places that cooperating components would show up; this chapter is the first of those places.
To begin with, consider the role of the classical network. The trusty American Heritage Dictionary defines a network like this:
1. An openwork fabric or structure in which rope, thread, or wires cross at regular intervals.
2. Something resembling a net in consisting of a number of parts, passages, lines, or routes that cross, branch out, or interconnect: an espionage network; a network of railways.
3. A chain of interconnected radio or television broadcasting stations, usually sharing a large proportion of their programs.
4. A group or system of electric components and connecting circuitry designed to function in a specific manner.
As the definition points out, networks existed long before computers. For example, a "network of highways" would have made immediate sense to a listener in the '50s -- before computer networks had been invented. Similarly, many people commonly understand the phrase "television network" than understand any computer-related network terms. Drawing again on the definition, the common elements in these usages of the word relate to the concepts of structure, connections and interconnections, and components, and tying them together into a larger whole. Computer networks are based precisely on this set of concepts; the term, of course, is based on the networks that existed in other forms at the time computer networks were invented.
The central concept behind a computer network is connectivity. The network facilitates the interconnection of computers, terminals, printers, and other computers -- without having to worry about the mechanics of how the connection is made. Achieving worry-free, thought-free, any-to-any connectivity is critical to the eventual long-term success of client/server systems -- if they are to fulfill their potential.
137
In the beginning, to borrow a biblical phrase, networks were invented to connect terminals to computers. In the late '60s, as terminals became common, it became very attractive to start locating terminals wherever a company did business. For big companies, this meant spreading terminals out all over the world. As terminals became ubiquitous, and as computer applications that could talk to terminals became common, there was a need to give terminals the capability to talk to more than one computer system. For example, a company originally might have installed a terminal to facilitate order entry , perhaps to enter the orders for a particular product family. How- ever, once the terminal was placed on a desk, the user started wondering why the same terminal couldn't be used to check on the status of orders, schedule deliveries, enter customer complaints, record payments, and so on.
This is why. In that same beginning, each terminal was connected to a single, specific computer by a dedicated, special-purpose communications line. If the terminal was in the same building as the computer, the connection was a physical cable -- a set of wires that literally connected the computer to the terminal. Later, as terminals spread to remote locations, special dedicated phone lines were used for the same purpose. During these early years, these remote terminals operated, literally, by fooling the phone system. Telephone companies in those days provided dedicated telephone lines as a way of facilitating voice communications for large companies; a special leased line enabled branch offices to be connected to head offices. This bypassed the complexity of the long-distance network, and it saved the company some money along the way. So whether they were local or remote, early terminals were once connected to their computers through a dedicated connection.
Naturally, when users began to ask about talking to more than one computer, this raised some eyebrows, and at first no straightforward answer was apparent. As hard as this might be to believe (or picture), the initial answer was simple: put another terminal on the desk. And in fact, through the '70s, and even into the early '80s, many desktops ended up with two, three, four, or as many as seven or eight screens and keyboards on them. This cozy situation was not limited to clerical or entry-level positions, either. Some of the most demanding consumers of information are market traders -individuals who make and lose millions every day trading stocks, bonds, commodities, and currencies. These traders are totally dependent on incoming information: the more sources of information they can receive, the more ways they can analyze that information, and in turn, the more money they can make (and lose) every minute. As a result, until recently, the most cluttered desks -- the ones with the largest numbers of screens and keyboards -- were those of highly paid and highly skilled brokers. Finally, in case you are thinking that such complex conglomerations of equipment are never seen by ordinary humans, think back to your visions of airplane cockpits, rocket launches, and the control centers of nuclear plants -- all seen regularly by millions on television sets around the world. All of these
138
environments involve highly skilled people, working with many screens at one time, drawing information from a large number of separate computer systems, to do their jobs. So the need to exchange information with many different sources is at least moderately common.
Obviously, while the operator of a nuclear submarine may need (and even want) to view many screens at one time, the average person is happy with (or will tolerate) one at most. Consequently, some mechanism is required to enable the user's terminal to talk to many computers. One way to solve this problem is with a simple switch located at the user's desk (this is similar in concept to the switch used for a shared printer, discussed in the preceding chapter). The switch would allow a single terminal to talk to many computers by shifting back and forth across the still-separate dedicated lines running to the remote (and local) machines.
Having all these dedicated lines is a problem, too. In the first place, having individual lines connecting terminals to computers is an expensive proposition. Each line in a building requires wires running throughout much of the office structure. As terminals are added, and as each terminal talks to more and more computers, pretty soon there aren't enough wires to go around. Long-distance connections are even more of a problem because of their expense. Having permanent, dedicated long-distance circuits between every terminal and all the computers it might use becomes prohibitively expensive almost instantly. Finally, all those connections are a physical problem for both the computer and the terminal. At the computer end, supporting thousands of connections, many of which are not in use at any particular time, quickly becomes too expensive to be affordable. And at the terminal end, having lots of wires, while feasible, is hardly practical, let alone neat or compact.
When this problem first presented itself, a straightforward solution quickly appeared based on an older technology: the telephone system. By the '70s, the telephone system had already reached the point where any telephone in the Western world could connect itself to any other telephone in the Western world by simply punching in a series of digits. The connections were usually fast and for the most part transparent, at least for voice conversation. Why not find a way to send data over the same network? If this could be done, giving every computer a phone number and giving every terminal a phone would enable any terminal to connect to any computer. No sooner said than done.
As it turns out, with the invention of the modem, computers and terminals could talk to each other over normal phone lines, and the telephone system couldn't even tell that it was data moving back and forth, not voices. This concept of fooling the phone system was critical because it meant that, without having to invent and build a complete alternative worldwide interconnection network -an undertaking that would take years -- widespread and highly flexible computer-to-terminal connectivity became possible almost instantly.
139
Many of today's simple networks are still based on this simple technology. In fact, the same technology enables any personal computer owner, by dedicating a PC and in- vesting in several extra phone lines, to create a public bulletin board. Think about the power of this technology. A teenager, with no outside assistance and by spending less than $5,000, can establish a database and communications environment that strangers all over the world can tap into and use. Connectivity to the max. Yet, for all of its power, this approach still had some fundamental limitations.
The voice-based telephone system was never designed to handle high volumes of data in an environment where dropping a connection or losing a bit could have catastrophic consequences for the terminal or computer participating in the conversation. Furthermore, even ignoring the issues of reliability and integrity just raised, the voice system is completely inadequate for allowing even terminals to talk to computers very quickly, let alone the even higher speed requirements imposed by computers talking to computers. After this problem became clear, entrepreneurs rushed forward to start inventing, building, and selling new technologies to facilitate data communication.
During the '70s, network design focused on two primary problems: 1) providing inexpensive conduits for transporting large amounts of data quickly and 2) ensuring that the data would be transported reliably and accurately. To understand the first challenge, consider the needs of a terminal user. As I covered in the section about GUIs and UIs, a terminal screen holds about 2,000 characters of information. In the early '70s, the typical voice-grade phone line could carry about 120 characters (bytes) per second. As a result, sending an entire screen of information from the computer to the terminal could take 15 to 20 seconds -- an eon for the human waiting at the other end. With special equipment and carefully conditioned phone lines, this screen transmittal time could be reduced to under a second, but most customers couldn't afford to pay for such expensive facilities between all their terminals and computers.
To understand the second problem -- reliability -- recall again the design goals of the original phone system: transmission of voices. The resulting analog (as opposed to digital) system was never built to provide a noise-free environment; most people don't care and can't even tell if their phone conversation is accompanied by constant or intermittent background noise. In fact, people are particularly good at understanding spoken conversation even in the presence of surrounding noises of all kinds; computers, on the other hand, are prone to just not work in this kind of environment.
The solution to both problems relies on a single observation: even though computers and terminals send large amounts of information in very short periods of time, there are also long periods where no information is being exchanged. The reason this observation is so critical is that all the solutions to the problems of capacity and reliability involve the use of very expensive communications lines and equipment. However, if computer traffic is in fact bursty -- information is sent back and forth in relatively short but intense bursts with long periods of silence in between -- then these expensive facilities can be shared by many users. In that case, these otherwise too expensive facilities become affordable.
140
What's required to facilitate this kind of sharing? A dedicated computer that can allow many terminals and computers to all share a common communication line. You've seen this before; it's a server. The specialized network switching computers of the late '70s, sometimes called network nodes, were in fact the earliest form of specialized servers. Sharing alone solves half the network problem: it allows several computers and terminals to share a communication line, making it affordable to have a much more expensive and capacious link than would otherwise have been possible. How about reliability, though?
Reliability has three aspects: availability, accuracy, and resilience. Availability is simply a measure of how often the network is there when you need it: 99.7 percent avail- ability, for example, would mean that, averaged over some long period, a network was available 23.928 hours (all but 4 minutes and 19 seconds) of every 24-hour day. Accuracy, of course, measures what percent of the information transmitted through the network arrives at its destination unchanged; typically, a network drops as few as one bit in many billions. Resilience -- how well the network can withstand failures of individual components -- is the final aspect of availability. For example, if a single modem or telephone line goes down, can the network somehow patch information so that it flows around the failed components?
Just as terminals were becoming really popular, the armed forces began to realize the tremendous advantages of using terminals tied to computers to coordinate and control military units spread around the world. For the first time, it was possible to imagine having all parts of a distributed military operation constantly being coordinated with each other; headquarters command and control could be located anywhere and still stay up to date all the time. Unfortunately, when such a system is introduced, the organization quickly reaches the point of total dependency -- where it can no longer function when the communications network goes down. This is a case where the issues of availability, accuracy, and resiliency are taken to their limits. For this reason, in 1975, the Defense Advanced Research Projects Agency (DARPA) funded research into the construction of highly reliable networks. In fact, the goal of the research was the design of a computer communications network that could continue to function even in the face of nuclear attack -a network that could tolerate the complete destruction of multiple major components and still work smoothly.
The result of this research, still at the center of most network designs today, is the concept of packet switching. Consider a message that needs to be sent from a terminal to a computer or from one computer to another. Now imagine chopping that message into equal-sized packets. Each packet can be thought of as a little envelope containing a small part of the message. Having divided the message up into chunks, the network now transmits those chunks one at a time. After sending each packet, the network node (the communications server) waits to find out whether the packet was received correctly at the other end; if not, the packet is retransmitted until it is received correctly.
141
Without going any further, such a network is already vastly more accurate and tolerant of failure than before. For example, if a lightning storm causes a phone line to be temporarily noisy, the network simply retransmits packets until the noise dies down. Furthermore, because the packets being sent over and over are relatively small, even a small lull in the storm allows packets to get through so that progress is made, even if a little at a time. This eliminates the situation in which a long message had to be retransmitted over and over for a very long time just because a single bit error some- where in the message caused it to be rejected each time.
Today, even in very noisy environments, error-correcting modems built on these principles provide virtually error-free transmission by implementing packet transmission and error detection and retransmission, as just described. This approach even has an unexpected benefit. At first, it might seem that doing all this work would slow things down: transmissions have to be cut up into packets, each packet has to be checked for accuracy on receipt, some packets have to be retransmitted, and finally, at the other end, the packets have to be reassembled. But as it turns out, this approach leads to more throughput, not less.
The accuracy of a communications line depends on the rate at which information is pumped through it. The faster you shove data through the line, the closer to its limits you will push it, and the more errors you will see. A line that looks totally error free at l00 characters per second may introduce errors into 1 to 5 percent of the characters at 1,000 characters per second. Normally, this would force you to drive the line at the slower speed. However, with error-correcting modems, you can safely run the line at a substantially higher speed, knowing that the occasional error will be detected and corrected. One reason that modems today commonly run at 14,400 or even 28,800 bits per second -- over ten times the speeds common only five years ago -- is that those modems now contain tiny computers that implement packet-based error-tolerance techniques. In fact, having these computers built right into the modems makes them much faster in not just one but two ways. While the computer is busy splitting the data up into packets, checking for errors and retransmitting when necessary, they are also compressing the data at the same time. That is, when the computer sees repeating characters or patterns, it automatically sends the repeated data only once. So packets, when supported by intelligent (computer-based) modems, are actually faster because the phone line can be driven faster and because compression allows more data to be transmitted in less time.
What has been described so far is packet-based transmission; where does the switching come into play? The use of packets enables users to tolerate noisy lines, but what if the line goes away altogether? That's where packet switching comes into play.
The network described until now consists of three components: terminals, computers, and network nodes. The nodes are specialized servers that convert messages into packets, handle error detection and retransmission, and so on. How are the nodes connected to each other? Earlier I discussed the use of the phone system to replace direct connections between terminals and computers. This facilitated a shift from a scenario where every terminal was connected to all computers it talked to by a direct
142
connection, to a scenario where the connection was initiated by placing a phone call. With the arrival of packet switching, direct connections come back into play. But now, I'm talking about direct connections between network nodes instead of between terminals and computers or between computers and computers.
To understand this transition, go back to the very idea of a network -- conceptually, a web of connections that allows components attached to the network to talk to each other. In the very early days, the network consisted entirely of direct connections: fast, but very expensive and inflexible. Every time a new connection was required, a direct link was established between the terminal and the new computer it wanted to access. Soon, the country would have been covered with wires. By using the phone system and placing telephone calls to computers, this direct connection network was replaced with the phone network itself. This new approach was highly flexible, but very limited in throughput; dial-up phone lines simply can't handle large volumes of information. That puts you between a rock and hard place: high throughput with unacceptably high cost and rigidity or low-cost flexible networking with very limited throughput.
What makes the telephone network so flexible? Its switching capability. By entering a telephone number, you enable the telephone exchange to build a connection "on the fly" that can take you to any other phone in the world within seconds. If telephone exchanges can switch calls from any phone to any other, why can't computer-based network nodes do the same thing? They can, of course -- that's the switching in packet switching.
To finish explaining packet switching, I need one more conceptual building block: addressing. How does a terminal get to ask for a connection to a particular computer? In the dedicated-line world, the answer was simple: you didn't get to ask. Going through the telephone network too, the answer is easy: dial a telephone number. The telephone number functions as an address; it specifies the destination you want to be connected to. The word address means, in fact, just what you think it does. Just as every person has a street address, allowing mail delivery, every telephone subscriber has a telephone address that allows the delivery of phone calls. The street address consists of words and numbers; the telephone address, of course, is just a number. The point is that addressing schemes are not new ideas; they solve a big problem in communications. If new types of networks are going to exist, new types of addressing will have to exist along with them.
So an important part of a packet-switched network (and other computer networks, too) is the addressing scheme. Conceptually, this is pretty simple. A terminal is connected to a network node. The terminal may be either directly connected or it may access the network node through a local phone call. Initially, the terminal may communicate with that node, but until some instructions are given, any information goes to the node and no farther.
143
The first thing the terminal does is specify a network address, asking the node to establish a connection to a computer connected to that address. What do addresses look like? They can be names, numbers, or any other construct that can be programmed into a computer; the network node, after all, is just a computer. How does the network node know what connections are available? It stores a directory in its memory; again, it's a computer. Now to the key question.
How are nodes connected to each other? Recall the very first discussion about networks and the cost and rigidity associated with having too many direct connections. The existence of network nodes between the terminals and the computers doesn't make having too many direct connections suddenly affordable. Network nodes do, however, introduce a new possibility: store and forward.
Suppose that you want to connect a terminal in San Francisco to a computer in New York. And suppose that you have a link from San Francisco to Denver, and another from Denver to New York. Wouldn't it be nice if you could somehow join those links together into one big link? That's precisely what a packet-switching network makes possible in this scenario:
1. The user requests a connection to a computer in New York.
2. The San Francisco network node knows that there's no direct link to New York but that there is a route through an intermediate node in Denver, and it permits the connection to be established.
3. Each time information flows back and forth, it moves first from San Francisco to Denver and then on to New York, or vice versa.
What happens when the link from Denver to New York gets very busy? If a packet arrives from San Francisco and the line to New York is too busy to accept it, the Denver node stores the packet, queues it, and when its turn comes, forwards the packet to the New York computer. That's why this is called store and forward. Okay, okay, how does all of this relate to reliability, resiliency, and nuclear attack?
Suppose that the simple network has grown. Now there are connections from San Francisco to Los Angeles, Portland, Seattle, Denver, Chicago, and Dallas. Each of these in turn is connected to New York. Now if the node in Denver goes down, the San Francisco node can still get through by using one of these other intermediate cities.
All of this is pretty simplistic; to appreciate the true resiliency of packet switching, consider what a real network might look like. Imagine building a network of nodes connecting offices across the country .Pictorially, such a network looks like a flight diagram; in many fundamental ways, all large networks are similar. Now look at resiliency in action at two levels: dealing with complete failures and dealing with softer failures, such as busy or exceptionally noisy links.
144
First, pick through all the routes a packet can use to get from San Francisco to New York. For example: San Francisco, Portland, Seattle, Chicago, Minneapolis, Cincinnati, New York is a perfectly fine route, and if enough intermediate nodes or communications links go down, it might be the only route. This theoretical network is not complex compared to many real-world nets, but it offers literally dozens of routes between almost any two points on the network. Even if quite a few nodes and links go down, many connection paths are still open.
Alternative routing is very useful, even when nodes and links haven't gone down. Suppose that one worker is sending a huge report from New York to Denver, just as someone else asks for a screen of information from New York at the San Francisco-based terminal. The New York network node, noticing that transmissions to Denver are taking a long time, picks another route; the user can't tell which route -- all the user sees is continuing immediate response.
The best thing about packet switching is that it is adaptive and dynamic. This means that the network picks the best route for information over and over each time information has to be sent. Moreover, the computer can replan the route continuously, packet by packet, and node by node, as information moves through the network. For instance, if a packet reaches Denver but has trouble getting directly to San Francisco, it can go instead through Portland or perhaps Seattle. Different packets can take different routes as the network continuously adapts itself to the conditions around it.
A packet-switching network practically defines availability and resiliency. Information routes are changed on the fly, even in the middle of messages, adapting not only to failures, but also to heavy traffic and noise on the line. And when conditions improve, links are repaired, traffic dies down, and noise goes away, the network, without human intervention, adapts to the new circumstances, too.
To fully appreciate this self-adaptive behavior, I recall a sales demonstration that manufacturers of network nodes used to present to their prospective customers. The sales representative set up two network nodes with several lines interconnecting them. A high-speed printer was attached to one node and a computer to the other, with a report being sent to the printer. At the beginning of the demonstration, the printer plowed through the pages quickly. The sales rep then began unplugging wires interconnecting the two nodes. As this disconnection progressed, the printer slowed down, printing less and less quickly, until eventually, when the last wire was unplugged, printing ground to a halt. Later, as the representative plugged the wires back in, printing began, moving faster and faster, until, with all the wires finally reconnected, the printer again produced pages at full speed. For all of its naive simplicity, this simple demonstration explains adaptive behavior better than many long explanations do.
145
Packet-switched networks are so strategically important in the client/server world because of the degree of shared infrastructure they facilitate. Because many people can share the nodes in a switched network, it becomes economical to interconnect these nodes with very high-speed links. Understandably, high-speed links become more expensive as they get longer. For example, trans-oceanic communications lines are particularly expensive, as are lines running across continents. However, the switching behavior of packet networks allows a small number of super-high-speed-long-haul lines to service many more network nodes spread across the country .Conceptually, this structure is just like the feeder networks used by airlines to bring passengers in from secondary cities to the gateways they use for long-haul routes. By using switching in a store-and-forward environment, users at nodes in even very small locations can still access super-high-speed links reaching around the world. And the resiliency of the network against failures and bottlenecks makes the construction of large networks both feasible and attractive.
Originally, packet-switched networks were used primarily by the government and very large companies. By the late '70s, however, several public packet-switched networks had become widely available. As a result, any computer or terminal anywhere in the (Western) world could access a local packet-switched network node and reach out through a relatively high-speed and totally noise-free connection to any computer -- no matter where it was located -- as long as that computer had a connection to the network.
These public networks had a fundamental cultural impact as well as a technical one. This impact is based on three factors: distance-independent pricing, the development of electronic mail, and the development of community memory systems. The overall effect of these three factors was to convert the world into a "corporate global village." With the original invention of the telephone and the airplane, the world became a much smaller place, both for organizations and for individuals. The true impact of the wide-area network, while not complete, will magnify this shrinking of the world, eliminating the effect of geographical distance. Given the availability of packet-switched networks, electronic mail, and community memory , teams located in any number of locations can operate almost as efficiently and effectively as teams located all in the same office. Effectively, the world shrinks to feel like a village where all locations feel as though they are within arm's reach. This chapter explores how today's technology is on the verge of making this true for organizations. The end of the book takes the same trend a step further into the home and the personal global village.
In the Wizard of Oz, Dorothy discovers that by clicking her heels three times she can be transported home instantly; even contemplating a form of magic that makes distance disappear fills us all with wonder. In a way, the fundamental transformation that
146
a wide-area network introduces is the idea that distance no longer matters. Long-distance telephone call pricing, for example, is generally based on the distance the call travels. In a packet-switched network, though, distance has very little to do with the true cost of building and maintaining the network. Rather, in a computer network environment, almost all the costs are related to the volume of information being transmitted. So, from the beginning, the cost of using a computer network has been based only on the quantity of information being transmitted and sometimes on the speed at which it is being sent, but not on how far it is going.
Two fundamental applications have developed to take advantage of the distance-eliminating effect of the network: electronic mail and community memory .Each has enormous cultural impact on organizations and users, changing the way people work in fundamental ways. Because mail is the older and more mature technology, it will be examined first.
Early users of computer networks quickly noticed that they could use the computer as a convenient drop-off point to deliver messages to other users in a variety of locations. Why not develop software specifically designed to facilitate message exchange? What I'm talking about, of course, is the birth of electronic mail.
Electronic mail is very simple to understand. A sender types a message, such as a memo or letter, from the keyboard and then specifies a list of recipients for the message or mail. When finished, the sender presses a send key, enters a send command, or in some other way lets the computer know that the message is ready to go. The computer (in a moment I discuss which computer I'm talking about) takes the message (or e-mail, as it's often called) and stores it in a special e-mail database, some- times called a mail store. Later, when a recipient checks for e-mail, the computer tells the recipient about the new message, along with any other messages that may have come in, and the recipient can read the mail at any time.
The power and beauty of e-mail stem from a sort of contradiction: it is instantaneous, yet it doesn't operate in realtime; it is this exact conundrum that makes it wonderful. E-mail is instantaneous because when a piece of mail is sent, it is immediately avail- able worldwide. In other words, the moment I hit the send key, you can immediately read my mail even if you are in another country or continent. At the same time, be- cause mail is stored for later retrieval, if you are not available at the time I send my mail, it doesn't matter; after I hit the send key, I can forget about the mail, knowing that it will be delivered to you the next time you connect to the e-mail database. To understand electronic mail in context, compare it to the two standard alternative communication mechanisms: telephones and memos.
Telephones are instantaneous, but they also work in realtime. If I call you and you pick up the phone, we are connected immediately; however, if you're not there, we start playing phone tag, sometimes forever. The phone tag is based on the realtime
147
nature of the telephone system; if you're not there, telephone conversation simply doesn't happen because the system can operate only in realtime.
Memos are not instantaneous but have the advantage of operating in extended time. The term extended time means simply that a communication happens whether you're available immediately or at some later time without the communicator being responsible for extending the communication through time; the extended time mechanism gets the message through. Memos work this way automatically, of course. If I send you a memo, it sits in your In box until you get to it. I don't have to worry about trying to send the memo to you over and over (as I would using the phone); I put it in the interoffice mail and depend on your In box to store it until you are ready to read it. The problem with memos, of course, is that they are far too slow.
Electronic mail falls right in the middle. It's instantaneous, arriving almost before it's sent, but it's queued as well, so that after it's sent, e-mail waits patiently forever with no extra action required. Electronic mail also is constantly accessible. At any time, from any location, a user can always pick up his or her e-mail. So it is instantaneous, like a phone, but constantly available, like a memo. The one thing e-mail lacks is true interactivity -the capability to hold a literal two-way conversation. However, by pro- viding instant delivery, constant access and availability, and automatic queuing, e-mail changes organizations. E-mail, as it turns out, is a cultural change agent. When it's widely adopted in an organization, it radically changes the way people work.
To begin with, e-mail usually makes the phone stop ringing. The telephone is generally useful for immediate conversation -- questions and requests that can't wait. At the same time, telephone tag is so common that most people find the phone as much of a nuisance as it is a convenience. In addition, for the receiver of a phone call, the telephone is a prime interruption agent. Historically, however, the only alternatives to the telephone were face-to-face discussion, which requires physical travel, or memos, which take a long time to produce and distribute.
Because e-mail arrives as soon as it is sent, often literally, it has the most important spontaneity characteristics of the telephone. It stays queued until the receiver reads it, thus eliminating phone tag. To make the picture complete, an e-mail message can just as easily be sent to 50 people as it can to one person, so it eliminates the need to make multiple phone calls. In this respect, e-mail is better than a memo too, because unlike paper documents, a message sent to 50 people can be sent to all 50 with a single keystroke, with no duplication or distribution required. So sending e-mail brings peace of mind; after it's sent, I know that it will be delivered immediately and read as soon as the recipient gets to his or her mail.
148
For receivers, e-mail provides fast access to communications while still providing control over interruptions. The beauty of e-mail is that you get to decide how often and when in the day you go through your mail. Most systems provide three levels of control:
To see how these three levels fit together, picture a typical scenario. Each time you enter your office -- in the morning, after lunch, and so on -- you scan through your In box to see what new messages are there, who they're from, and what they're about. Unlike voice mail, for instance, the scanning process takes just a few seconds as your eyes flip through the titles. Occasionally, you see an e-mail message that seems important; you quickly open it, scan through the contents, again at warp speed, and decide whether to spend more time on it. At times of peak activity, you may have to
149
flip through several messages in this way; these messages become your initial notification of something at work, followed by responses from all the other people involved with the particular issue.
Even reading through ten messages in sequence often can be done in just ten key-strokes; you read the first couple of paragraphs of each and then move on to the next message. Through the rest of the day, as you work first on a memo and then on your sales forecast, the mail system periodically alerts you to the presence of new incoming mail; sometimes you care, and sometimes you don't, but either way the alert, although immediate, is also unobtrusive (unlike a telephone). Finally, once a day, at a time of personal choice, you spend half an hour going through your In box, methodically dealing with all the information, requests, and questions that have come in that last day.
Does this description of a day centered around e-mail seem quite innocuous -- hardly a major change in working lifestyle? Consider that the electronic messages I described could have originated in countries all over the world. Also, consider the impact of sending an alert on an issue to people all over a building -- and all over the world -- immediately, without having to pick up the phone many times, without waiting for a memo to arrive, and without having to schedule a teleconference. The ability to include many people on a distribution list makes it easy to broaden the scope of a discussion, and in turn, each of those people can easily forward the entire message when he or she receives it. Because the mail always arrives immediately, all those involved are able to participate in several rounds of discussion with the circle of participants steadily broadening. Each new participant, as he or she receives forwarded mail, is able to review the entire context of the discussion without anybody having to take the time to repeat oral descriptions (let alone suffering the information loss implicit in orally passing information around a circle). Finally, all the players are able to interact fluidly without either having to wait for memos or suffer from constant telephone interruptions. For the international participants, e-mail makes it possible to be directly involved in the issue, in realtime, despite time-zone differences.
The cultural impact of electronic mail is best described and summarized by one extensive, carefully instrumented military experiment. In this experiment, an electronic mail system was successful in the following ways:
 | Decreasing the volume of memos and telephone calls by more
than four times. |
| Facilitating faster decision-making and reporting newly
made decisions to more people sooner. |
| Changing patterns of communication. Before the implementation of e-mail, almost all conversation was vertical -- between adjacent levels in the chain of command. In the electronic mail environment, organizational members were |
150
free to send mail to anyone else on the system, including high-ranking officers and people in completely different parts of the organization. This mail was not only condoned organization wide, it was also encouraged.
Reaching beyond efficiency, any communications technology that eliminates phone calls and memos, accelerates decision making, results in increased consensus, and significantly democratizes the pattern of communication itself can truly be described as a cultural change agent.
All of this aside, probably the most telling measure of the importance of electronic mail is its popularity. For many users, e-mail -- not word processing, spreadsheet, or database -- is the application they run most every day. E-mail is the first application they turn on at the beginning of the day, the first tool they turn to on returning to the office, and the software they spend the largest number of hours interacting with most days. Speaking of mission critical, in most companies, electronic mail, even ahead of the network itself, is the component whose absence is noticed the soonest and whose failure will result in the largest number of complaints, the most quickly, when problems occur.
When networks were first invented, the original goal was to connect terminals to computers. Similarly, when electronic mail first became popular, the communications path always involved a terminal talking to a computer through a network. Thus, first generation networks provided terminal to computer communication. With the growth of packet switching, users could quickly and easily access their home computer from any location in the world, and any given computer could service users no matter where they were located. What happens, though, when users on separate electronic mail systems want to start communicating with each other?
The original answer to this question was, "Each person should use several electronic mail systems." Just as originally users had several terminals on their desks, the e-mail equivalent is having accounts on multiple e-mail systems. This is in fact a very modem problem. It has become routine for me to receive business cards that include a CompuServe, MCImail, and perhaps an Internet e-mail address all on the same card.
While having several e-mail accounts may be modem, it is hardly convenient and quickly becomes unworkable. Imagine, for example, a research scientist who routinely communicates with other scientists at 15 or 20 universities and corporate research labs. Each lab has its own e-mail system; does this mean that the scientist has to have 20 e-mail accounts and, to stay current, has to check each account regularly to receive mail? Even if this were somehow workable, how does the scientist send a given piece of mail to colleagues on these foreign e-mail systems? By copying mail 15 or 20 times? Doesn't this negate many of the original benefits that e-mail was supposed to provide in the first place?
151
The problem of connecting multiple e-mail systems is very similar to the problem that phone companies faced in the 1920s. By then, individual telephone exchanges had become relatively common, but no mechanism existed for connecting exchanges to each other. Once that problem was solved, telephones became vastly more interesting because suddenly, communication became possible across the country instead of just within a single community. The second generation of networking had a similar impact on the computer community.
While the first generation of networks connected terminals to computers, the second generation connected computers to computers. Once computers could talk to each other across high-speed communications links, three major applications became possible: computers could share information with each other, electronic mail systems could be interconnected, and distributed databases became possible. The first application is discussed later in this chapter; distributed databases are covered in other chapters. It is the impact on e-mail that needs to be considered now.
The original implementations of electronic mail all ran on large computers: mainframes and minicomputers. So the original impetus for using networks to interconnect e-mail systems was focused on connecting these large computers to each other. How- ever, these large computers were (and are) relatively few in number. So while interconnecting them offers strong benefits, it is also true that users might have been willing to live with separated e-mail systems on big computers for some considerable period of time. The fact that so many business cards have multiple e-mail addresses on them even today is silent witness to this fact. However, as e-mail grew in popularity, an entire second e-mail community began growing rapidly, one which made e-mail interconnection a virtual requirement.
Even before servers and LANs became available, the trend towards running e-mail systems on smaller and smaller computers had already begun. Digital Equipment Corporation (DEC) is one company that took advantage of the growing popularity of electronic mail in this way. DEC used the popularity of e-mail to sell its VAX minicomputers, which ran corporate mail systems more cost-effectively than mainframes could. A key selling point for DEC was that smaller departments and remote divisions of a company could expect better and more reliable e-mail service from their own minicomputer than they could from the central shared mainframe. For small workgroups and little companies, however, even minicomputer-based electronic mail was far too expensive. The development of LANs and personal-computer-based servers in the mid '80s gave even the smallest company or workgroup the benefits of e-mail. Consequently, today's largest electronic mail community runs on local-area networks, often tying into central mainframe-based systems that link all the LANs in a large company to form a single organization-wide mail system.
Having e-mail accounts on a small number of large e-mail systems is inconvenient but perhaps barely tolerable. What happens when there are literally thousands of e-mail systems, most running off servers connected to LANs? Nobody could possibly have or keep up with accounts on dozens or hundreds of separate systems. All of a
152
sudden, having a way of connecting all these e-mail systems together into one big e-mail system becomes not only convenient but essential -- and that requires networks that support computer to computer communication; that's what second generation networking is about.
Just as DARPA had funded the original research that led to packet-switched networks in the first place, it also funded the subsequent research which resulted in second generation networks connecting computers to each other. What DARPA found was that while packet switching was useful for failsafe missile command and control systems (the original goal), it was even more useful for general purpose communications on a daily basis. The question was, "What would happen if a large community of computers, used by researchers and military organizations, was connected together by a high-speed network on a permanent basis?"
To answer this question, DARPA agreed to fund the installation of a nationwide packet-switched network, called ARPAnet, the foundation from which the modem Internet grew. DARPA installed high-speed leased lines connecting about a dozen large research labs in the United States. At first, ARPAnet was used only by its developers, the software engineers working on the packet-switching technology and underlying network node engines. Soon after, however, ARPAnet's limited audience grew.
About the time that DARPA put ARPAnet into place, e-mail systems were gaining popularity at many of the same research locations being selected to be on the net- work. In retrospect, the result was predictable. E-mail developers and network developers discovered that combined, the two technologies were much more powerful than either one alone. To understand the resulting explosion, picture the situation existing just before. Researchers, whether in universities, corporations, or government labs, tend to work in closely related areas, even though they are widely distributed geographically. Before the advent of e-mail, these researchers had very limited means for working together, unless they were at the same institution. Publication of papers is too slow, and so is writing letters, for that matter. Telephone suffers all the disadvantages I've already discussed plus even more in an international environment. In fact, the very existence of conferences is based on the idea of providing researchers with a convenient forum to exchange notes and collaborate, but conferences don't occur frequently enough to enable people to work together. E-mail suddenly changed the whole picture.
As scientists, programmers, and researchers of all kinds quickly discovered, e-mail made collaboration across the continent almost as convenient as collaboration across the hall. Ideas could be sent to any number of people with instant delivery guaranteed. Recipients of those ideas could forward the ideas to others, and responses could return the same day. Because an e-mail message can easily include digital attachments:, such as spreadsheets, word processing documents, and even small databases,
153
researchers could send lab results, diagrams, and pieces of code along with their messages. The recipients received everything -- the mail and all the attachments -- with little additional work required by the sender.
Like the convenience of instant communication, the cost structure of e-mail and WANs also played a major role in fueling the e-mail revolution. Unlike the phone system's costs, packet-switched network costs have always been largely distance-independent. Instead, charges are based on the amount of information sent. As a result, for the first time, people all over the United States, and later the world, could communicate freely with each other without having to think about how far away their communications partners might be. Adding a name in France to an e-mail address list or forwarding a message to a colleague in Canada became a mere afterthought, freely entertained and put into action.
The immediate result of the creation of the ARPAnet was twofold:
| E-mail became such a compelling application that the
ARPAnet quickly grew in size and scope. |
| Several startup organizations grew out of DARPA'.s electronic mail research. |
By 1980, several parallel startups were all hard at work building toward their respective dreams for a changed world, and they were backed up by a growing set of inter- national standards. In the United States, Telenet and Tymnet embarked on ambitious plans to build worldwide packet-switched networks. In the beginning, the goal of these networks was to connect users all over the earth to both public and private databases: the global library concept. As Telenet and Tymnet built their systems and signed up customers, governments in the rest of the world decided that this opportunity must not pass them by. As a result, phone companies in countries around the world built domestic packet-switched networks at whatever rate seemed appropriate to them. Canada, for instance, was an early leader in this technology and today has a packet-based environment that is second to none in terms of geographic coverage.
Picture the situation that existed in the late '70s. In the United States, two major network vendors (Telenet and Tymnet) along with other less prevalent competitors had developed public packet-switched networks. Abroad, where communications was more tightly controlled by the government, each major country had its own public packet-switched network: Datapac in Canada, BTnet in the United Kingdom, and so on. Subscribers to anyone of these networks could talk to other subscribers on the same network. As soon as a customer on one network wanted to talk to a user of a different network, however, the whole system broke down. Each network used its own private language -- its own protocol, as networking people say -- and none of the protocols used on anyone network could connect with the protocols found on any other network.
154
As these networks grew, connectivity became a major issue. Given access to Telenet, how do I connect to a database attached to Canada's Datapac? Or, while traveling in the UK, even if I have the local BTnet phone number, how do I get from BTnet to my server at home, which is attached to Tymnet? Finally, in the early '80s an organization called CCITT (Consultative Committee for International Telephony and Telegraphy) came to the rescue. The CCITT is a quasigovemmental organization that functions as the United Nations of telephone companies. For several decades, this organization has facilitated the development of common protocols that allow telephone and telegraph systems built by competing vendors and installed in many countries worldwide to talk to each other. These protocols, called standards, make the world a connected place. The CCITT publishes standards that allow worldwide telephone systems to interoperate, hopefully transparently. Transparent interoperation enables a telephone subscriber in Australia to dial a series of digits and immediately talk to his or her relative in Switzerland, even if the call goes through intermediate phone systems in seven other countries.
Working with networking vendors around the world, CCITT applied that same expertise and coordination to computer networking and developed a standard protocol for packet-switched networks called X.25 (pronounced x dot twenty-five). X.25 prescribes how network nodes can talk to each other in a fashion that provides universal understanding at the packet-switching level. Just as telephone networks all interconnect on the basis of CCITT standards, X.25 allows all packet-switched networks to interconnect transparently.
The X.25 standard (along with some associated standards with equally riveting names such as X.3, X.75, and so on) prescribes the mechanisms with which computers talk to networks, terminals talk to networks, and networks talk to each other. As companies and network vendors worldwide adopted X.25 over a ten-year period, it became possible to count on worldwide access to computers through networks. As it turned out, however, access was not enough.
By the early '80s, Telenet and Tymnet were attracting more and more users, X.25 networks were sprouting up in other countries around the world and were becoming the industry standard, and in the public sector, dozens of universities and government sites were using ARPAnet. Everyone assumed that these networks would be used primarily for database access. In this same time frame, a number of other startups began peddling their databases, believing that the widespread availability of distance-insensitive networks would spark a demand for accessible information. The problem was (and is) that although compelling, the concept of a global library is still a distant vision.
The developers of ARPAnet had discovered a surprise hit with electronic mail, and the builders of the other networks soon made the same discovery .Their first reaction was to turn electronic mail into a product. The results were electronic mail offerings such as TeleMail, OnTyme, BTGold, Envoy, and a host of other services.
155
Electronic mail requires critical mass in order to be successful. Critical mass is a concept from nuclear engineering. An atomic explosion requires a critical mass, or amount, of uranium. If less than the critical mass is present, no explosion occurs, no matter what. With the critical mass and the right engineering, an entirely different outcome results. In the same way, given a group of people who routinely communicate, electronic mail becomes effective only when the majority of the people in the group use the system. Access to mail is not enough; the majority have to use the mail system routinely. And like the bomb, the situation is very bimodal: below the critical mass, electronic mail is just a nuisance, and above critical mass, it suddenly becomes the only way to work. To see why e-mail is so use-sensitive, compare the two cases.
Below critical mass, sending e-mail is an exercise in frustration. Even after you send the message, you can't assume that everyone has read it, so you have to follow up in some other way. Similarly, reading electronic mail, while not as frustrating, still isn't very productive because important communications still arrive on paper and the telephone still rings. E-mail is just one more place to look for information, one more potential interruption cropping up through the day.
When critical mass is achieved, however, it becomes a social requirement to use e-mail all the time. All your co-workers are starting to luxuriate in immediate delivery to distribution lists without the constant annoyance of that ringing phone. If you missed a meeting or didn't participate in a discussion, they (and you) are rightly annoyed that you're not going with the flow. E-mail has suddenly become the norm! And when this happens, the cultural forces acting on individuals to stay on top of their e-mail are automatic, insidious, and totally effective. When the right level of participation is reached, e-mail becomes a self-fulfilling prophecy, but only if critical mass can be achieved.
The number of regular users required to achieve critical mass depends on the population of the group. In a small company of about 15 employees, critical mass might be 10 people. Having just those 10 using the system all the time makes it totally worthwhile. In a company of 600 employees, the same critical mass easily could be 400 people. In society at large, critical mass requires not only that millions of people use electronic mail all the time, but also that those people use the same electronic mail system all the time. After all, what's the point of sending mail to somebody on another electronic mail system when that person won't receive your message? This is the problem that network and electronic mail providers have been struggling with for over ten years.
By 1985, Telenet, Tymnet, and all the other network providers realized e-mail's critical role in making networks attractive. ARPAnet had spread to Europe, Canada, and dozens of institutions in the United States. Thousands of individuals depended on
156
ARPAnet for routine daily communication on multitudes of interwoven issues. In fact, electronic mail had become so important that new classes of startups and standards appeared, with plans to capitalize on the opportunity.
This "opportunity" is nothing less than the replacement of mail, as it is known today, by e-mail, as it will be known tomorrow. Think about what the term mail means. Every day, mail carriers in every country deliver hundreds of millions of pieces of mail -- love letters, contracts, bills, and all the other documents that define so much of our lives. By addressing an envelope and putting a few cents' worth of postage on it, your mail is virtually guaranteed to be delivered anywhere on earth in a matter of days (at most, weeks). Mail is so integral to society that it is one of the few services provided by almost every modern government in the world. And now I'm talking about replacing it?
For all of its advantages and conveniences, electronic mail is even more convenient and powerful than conventional mail. Why be limited to receiving mail once a day? Why not on Sundays? Why wait for weeks to hear from Africa? Is it really necessary to make ten copies of a letter, address ten envelopes, and pay for postage ten times, just to include all the right people in a conversation? And when they receive it, why should they have to copy the mail again to forward it? If mail takes three days to arrive, and a letter is forwarded three times, the last recipient is 12 days out of date! The list of potential reasons to use electronic mail instead of real mail goes on and on. Most compelling of all, because the real mail is increasingly likely to have been generated on a computer in the first place, why not then just use that computer to send the mail instantly when you're done?
That's the vision that TeleMail, MCI Mail, CompuServe, and a host of other competing services were built to serve. The market as pictured in 1985 was more than huge: the corporate market alone was indeed immense, but extrapolating forward to a day when every home, first in the U.S. and then in the Western hemisphere, and then everywhere, would send and receive mail every day, the potential clearly goes far beyond huge! But is society any closer to realizing that ideal today than it was ten years ago?
Led by the CCITT, the standards community realized that X.25 and pure networking for connecting computers and terminals was just the beginning. So, actually beginning in 1982, the CCITT chartered a working group to develop a standard for interconnecting electronic mail systems. The result, called X.400, prescribes mechanisms whereby two or more e-mail systems can exchange mail, as well as other mechanisms to allow personal computers to talk to e-mail providers in a fashion that takes advantage of the intelligence of those PCs. By 1984, when the X.400 standard was finally published, many prognosticators were excitedly predicting that every mail vendor in the world would be X.400 compliant in two years. As a result, several efforts were initiated in 1985, many of which continue to this day, to start making the e-mail-linked global village a reality.
While the CCITT was laying the groundwork for all mail systems to speak the same language, startups and established companies were busy building the competing mail systems that the CCITT was planning to connect. For example, in 1985 MCI had just
157
established the first domestic rival to A T& T for long distance services. MCI looked at electronic mail, deduced that it threatened not only postal mail but also telephone long distance, and decided that e-mail protected and complemented its core business. Similarly, around the same time General Electric, which operated one of the largest public networks and computer service bureaus in the world, saw e-mail's potential and launched an effort to establish a worldwide e-mail system.
In the mid '80s, just as public e-mail was starting to take off, two different kinds of private e-mail were starting to become popular, as well. One kind was based on mainframes and minicomputers installed in large corporations. IBM's Profs and DEC's All-In-One systems each built worldwide communities with several million users each. The other kind, which is even more popular today, is LAN-based electronic mail systems such as Lotus cc:Mail, Novell Groupwise, and Microsoft Mail. While these private mail systems became quite successful in their own right, the availability of public networks to extend their reach worldwide provided the final link to large scale acceptance. With the widespread popularity of public and private mail systems interconnected by global packet-switched networks, surely it should be possible for any PC or terminal user to send mail to any other PC or terminal user. And because electronic mail is always so much more popular than regular mail (when it reaches critical mass), why hasn't the postal service disappeared? Given that so many organizations are so committed to the use of mail internally, why are normal letters still the routine norm for intercompany communication? If e-mail is so great, why are postal carriers still walking their rounds?
Postal carriers still walk the rounds for three reasons:
| Cultural change is a slow process, even (some would say particularly)
in corporate environments. |
| Interconnecting mail systems has proven stubbornly
difficult to do. |
| Beyond just connecting the networks and mail systems, providing directory services is the problem of the moment, standing in the way of the electronic mail dream. In theory if you are using MCI Mail and I'm using CompuServe, nothing should be easier than exchanging mail. In reality, unless one of us knows the other's mail identifiers, the task may be impossible. The lack of a universal directory service -- analogous to the white pages taken for granted in the telephone environment -- makes what should be simple often impossible in e-mailland. |
So, will electronic mail really become ubiquitous, potentially replacing the post office, or not? The answer is that that ubiquity is much closer than most people imagine. The Army Corps of Engineers has a saying: "The difficult we do immediately; the
158
impossible takes a little longer." The "difficult" has been achieved; now society is living through the slow process of making what was once thought to be "impossible" come true.
To begin with, many large organizations routinely depend on electronic mail. It is the single most popular application for users in those organizations, and as I pointed out earlier, those users notice and complain immediately when their electronic mail becomes unavailable. It has taken a long time for this reaction to be normal, however, and e-mail is still far from ubiquitous even in the corporate environment. It has taken so long precisely because it is a social and cultural shift.
Of course, the organization that is making the shift can be less than a whole company. It would be impossible for e-mail to take hold if entire Fortune 500 companies had to adopt it all at once. Nonetheless, because mail is only moderately interesting in small groups, it requires commitment from large divisions, departments, or workgroups to really take root. The result is a paradox of the most bittersweet kind. The good news is that when e-mail catches on, it catches on quickly and on a large scale. The bad news is that for it to catch on, it must catch on quickly and on a large scale.
Fortunately, the positive experience of the ARPAnet-centered research community provided some early examples of the huge gains offered by e-mail. Because of the military nature of DARPA, these gains were quickly noticed by the army, which in turn decided to experiment very deliberately with e-mail. Being centrally controlled, military organizations are in a position to achieve critical mass literally overnight, if they decide they want to. The result was the documentation of the experimental gains noted a few pages earlier. Catching the eye of the business community, e-mail quickly gained a strong following with an impressive roster of companies worldwide.
After corporate customers caught on to the productivity benefits of electronic mail, hardware vendors such as DEC and IBM realized that a potentially enormous software opportunity lay in front of them. Not long after the big vendors, dozens of other companies followed with mail products for machines of all sizes. Today, All-In-One, PROFS, cc:Mail, Microsoft Mail, and many public systems like America Online, CompuServe, and MCI provide regular service to millions of everyday users. E-mail is on the verge of becoming pervasive in the business world. Yet somehow, even with its widespread usage, e-mail is still far from replacing real mail. How can this be?
Although there are many mail systems, each with large numbers of users, these mail systems cannot talk to each other. As in the biblical parable, the resulting tower of Babel -mail systems with incompatible protocols cannot be combined into a larger compatible whole. The solution to this problem is far from complete but is beginning to take shape in the form of the Internet.
Today, it is safe to say that electronic mail is here to stay and is well on its way to eventually becoming at least as popular and dominant as the regular postal system. Even the problems associated with the tower of Babel are beginning to find solutions.
159
For example, at work I can routinely send mail messages to users on CompuServe, America Online, and any number of private e-mail systems, all in the same message; I can depend on the fact that my mail will be delivered transparently and quickly. As it turns out, though, even e-mail represents just the beginning of the cultural revolution that networks really represent. In fact, e-mail is the first step towards a broader phenomenon that I call community memory. And the Internet, the third generation network, is the enabling technology that really takes you beyond e-mail and into the world of community memory .
By the late '80s ARPAnet had outgrown DARPA's interest as well as DARPA's desire and ability to fund it. By then, however, ARPAnet's underlying protocols had been implemented so widely that ARPAnet was ready and able to become the Intemet. The very idea of the Internet represents a step forward in thinking about wide-area networks, or WANs. First generation networks connected terminals to computers; in the second generation, computers were connected to each other. The third generation of WANs, connect networks to other networks. In fact, internets, as these nets are called, are a step up from second generation networks -- they still connect computers to each other, but they also glue networks together .
Internets are important because they build on the idea that, in the world of LANs, the network is the computer. If personal computers tied to a LAN are to be a dominant form of computing in the '90s, then WANs need to explicitly support this style of computing. So, in a world where big computers were at the center of most large organizations, computer to computer communication was key. In a world where LANs are central, LAN to LAN communication becomes the key. Best of all, since a big computer can be just as easily connected to a LAN as can a personal computer, internets can serve the needs of both second and third generation networking users.
In the internetworked world, everybody either has a local computer or talks to a local computer. That local computer may be a large computer, such as a minicomputer or mainframe, or it may be a small computer, such as a server. In either case, the local computer provides the user with all of his or her immediate computing needs. The WAN now serves another purpose altogether: it moves electronic mail and other forms of data flexibly between computers. The computers interconnected by the W AN may be individual computers, perhaps even mainframes or minicomputers. The interconnected computers may also be the client/server computing clusters created by a LAN. As a result, WANs can connect computers with local networks of computers, and they also can connect local networks with other local networks. As LANs become more common, the interconnection of these local networks is the norm rather than the exception. Thus the W AN is often a network between networks, or an internetwork. Intemetting is a generic term used when networks connect networks (and computers); the Internet is the granddaddy of large intemetworks.
160
The Internet is important to e-mail systems because it was the first successful attempt at interconnecting a wide variety of geographically distributed mail systems on a large scale. Starting from the original ARPAnet hub, the Internet has spread to include thousands of private- and public-sector organizations. Each of these organizations has its own e-mail systems. Often there are different types of systems within the organization. The Internet itself has no e-mail system; it has something much more important. The Internet gives subscribers access to any of the e-mail systems connected to it so that the subscribers can send mail to any other subscribers no matter which e-mail systerns they are connected to. As a result, to many of the organizations connected to it, the Internet is second in importance only to the telephone network (sometimes more important).
If internetworking can work so well, why isn't it more common? Until the beginning of this decade, the primary obstacle to widespread e-mail was cultural. Today, although cultural barriers still exist (particularly in small businesses and homes), the major block is e-mail interconnection. The Internet proves the concept, but it leaves many important details to be worked out.
Even though its scale is already awe-inspiring, the Internet works particularly well because by global standards, it is still small. Moreover, the Internet is still used most heavily by research organizations, where competitive concerns are less of an issue than they are in the business world. Competition prevents internetting -- particularly in the e-mail context -- from catching on more widely in the commercial arena.
Particularly in this decade, e-mail has found a surprisingly broad base of users. Forums such as CompuServe, Prodigy, and MCImail are attractive to hackers, hobbyists, and small business users. Having captured these customers, the e-mail providers see them as potential lifetime customers. Looking beyond e-mail and bulletin board services, every one of these providers dreams of eventually selling all kinds of products and services in an electronic shopping world of the future. So when I talk about the dream of connecting all the mail systems together, the nightmare that immediately counters the dream for all e-mail vendors is one of previously captive consumers, now free to roam across competitive servers at will.
Superficially connecting mail systems is a straightforward and simple process. A variety of standards, including the famous X.400, already exist for connecting such systems. The Internet works because most of the computers on the Internet use the UNIX operating system, and most of the UNIX mail systems in turn have adopted a standard similar in concept to X.400. This demonstrates that as with X.25, it is possible to get all the mail systems of the world to talk together. The question is, if they did talk together, how would you look up another subscriber's e-mail address?
Well, how do you look up phone numbers? In the white pages, of course! What if you need a phone number in another city or country? Everybody knows that you call 555-1212 domestically, and the operator abroad! So simple, yet it works because
161
provision of phone service is a regulated monopoly in every country. Even after the breakup of AT&T, local phone service (including allocation of phone numbers) is still a regulated monopoly, now with six providers, one in each region, instead of one for the whole country .And in every other country in the world, phone service is a monopoly, as well. So phone books are readily available, 555-1212 works like a charm, and even abroad the operator is always available to help.
Provision of e-mail services is the opposite of a monopoly. It's a highly competitive market with huge stakes. Some countries initially tried to regulate e-mail service provision, but it's so easy to start up an e-mail service (set up a server and publish some numbers) that practically nobody tries anymore. E-mail is not competitive only in the public arena. Competition exists at many levels, with corporate systems existing side by side with public systems, all of which compete with private specialized systems.
Competitively, this comes to a head in the directory services arena. White pages are a form of directory service, and so are the yellow pages; so is directory assistance (555-1212). In fact, even in the closely regulated phone industry, directory services is a huge industry .So where do you get directory services for e-mail? Can the directory services be made to talk to each other?
As you might expect, the industry's answer is that it's time for another standard. And, of course, the CCITT has one: X.500, which is X.400's sibling. X.500, first discussed in 1986, prescribes a standard for directory services to exchange information with each other and also for computers to ask questions of those directory services. Why computers? Because e-mail is driven by computers, so if you type a name into e-mail, you want your computer to be able to look up the address for that name automatically.
The problem with X.500 is that it implies giving out customer lists. Imagine approaching your biggest corporate competitor and asking for his or her entire customer list, complete with up-to-date addresses. A problem? You bet.
Perhaps this sounds like an impossible situation. If you keep in mind how far this industry has come, though, there's every reason to be hopeful. E-mail is now a major cultural component. Stories about couples meeting and even becoming engaged strictly via e-mail are now commonplace in myth, if less so in reality. E-mail is on its way to becoming commonplace, and huge barriers still need to be knocked down before mail carriers stop knocking on doors. Keeping a sense of perspective, however , it is now quite likely that by the end of the decade, e-mail, like the telephone, will have become a standard mechanism for personal and organizational communication over distance and time.
Cultural education is still required, particularly among individuals and smaller businesses. The F AX revolution is helping in that effort. It is only a small step from F AX to e-mail: the step from transmission of printed output to transmission of the underlying information itself in a form that can still be edited. E-mail systems need to become better connected, but as the ubiquity of e-mail increases, the pressure for interconnection will make this inevitable. Users will get to the point where they just won't tolerate not being able to communicate with other users anymore. Finally, as mail systems
162
become increasingly interconnected, competition ( even though it blocks directory services today) will ensure that e-mail providers confront the directory issue them- selves; otherwise, external providers will do the job for them, leaving them with even less of the customer control they are so carefully trying to safeguard.
In many ways, it is ironic that electronic mail was the first application that virtually defined what networks are about. How is that ironic? Because although networks are about communication, at the time they were being invented, technologists were convinced they would be used for everything but communication. Yet, as it turns out, the one application that has propelled the growth of large networks more than any other has been e-mail, which is not primarily about communication, it is only about communication. Yet for networks to achieve their true potential, you must move beyond communication -- and that means creating the first electronic community memory.
Community memory involves the merger of two technologies: servers and electronic mail. Servers provide the means to share information so that users located anywhere can access the most recent version, while electronic mail allows those users to communicate with each other essentially instantly. What does it mean to merge these two technologies into one? Consider what happens when e-mail becomes too popular.
Microsoft is one company where e-mail is ubiquitous, and as a result, most employees have become file clerks. In fact, Microsoft, if thought of this way, has over ten thousand file clerks.
Suppose I send an e-mail to several other people within Microsoft. Before sending it, I think carefully about the contents, write the mail, and file it away for later retrieval. Then, each person who receives it reads it, perhaps forwards it to others, and files it away for himself or herself. Now suppose a colleague, at a later date, wants to come up to speed on the issue being addressed by the e-mail; is there any easy way for the colleague to do that? Only with the help of somebody who received that mail in the first place, and only if that original recipient took the time to file the e-mail. Electronic mail has a fundamental deficiency; it doesn't provide any form of shared memory.
Memory I whether shared or individual, is one of the most fundamental aspects of hu- man thinking. In fact, the idea of literally improving the way people think by improving the way they remember things dates back to 1945. Writing in the Atlantic Monthly, Vannevar Bush in "As We May Think" set forth the basic revolution driving not only the Internet as we know it today, but most of the personal computer revolution, too.
Vannevar Bush was the individual responsible for directing the applied research efforts for all of the United States government during World War II, including such critical efforts as the Manhattan Project. With the war behind him, in the Atlantic article Bush
163
reflected on the type of automated aides he imagined helping someone like him -- a knowledge worker -- in the not too distant future. The article, still in many ways fresh and relevant today, is particularly amazing because it predates the availability of modem computers. Nonetheless, it describes a device, called the Memex, that sounds like the kind of computer most of us would call a personal computer, but with facilities most of us would still like to have and can't. Particularly, Bush focuses on the ability to have the Memex store every piece of information -- printed, heard, communicated -- for subsequent retrieval on command.
Bush's paper is particularly significant because of the research and development that it subsequently spawned. Most important of all, from the perspective of community memory was the work of Doug Engelbart, and his Augmentation Research Center (ARC).
Starting in the early '60s, Engelbart began thinking about how a Memex could really be built, and how it could support teams as well as individuals in being more productive. It is hard to overestimate the impact that Engelbart had on the entire history of computers, even though his name is hardly ever mentioned any more today. At the time that Engelbart began his work, most of the facilities that we take for granted -- computer screens, keyboards, personal computers, networks, mice, and more -- had yet to be invented. Even Xerox's PARC (Palo Alto Research Center), often thought to be the source of these ideas, was still more than a decade away from being created. (In fact, Xerox itself was just taking off.)
In a period of about ten years, then, Engelbart and his team literally invented many of the ideas and technologies that drive our entire computer world. The mouse? Invented at the ARC (yes, invented from scratch). Windows, bulletin boards, hypertext -- all invented by Engelbart and his team. Overall, the entire effort was driven by two common elements.
The first element was the idea that computers should augment humans, not replace them. Recall that this work was taking place at a time when many still believed that as computers became "artificially intelligent," they would eventually replace people. On the contrary, Engelbart believed, computers would augment and not replace people by providing community memory.
The second element was that Engelbart's team created the first successful form of community memory, which is discussed next.
Community memory is the shared records of all the thoughts, concepts, conversations, and decisions that drive an organization. A community memory system can be thought of as a form of database, but really it isn't. Colloquially, any set of shared data is a database; in that sense, so is a community memory system. The term database has practically come to take on a very specialized meaning, which will be explored in more detail later in this book. Therefore, community memory is not a database in the normal sense of the word.
164
In many ways, in fact, a community memory is actually the exact opposite of a data- base system. Databases are typically highly structured, centrally located, and devoted to a single application. Community memory systems, on the other hand, are purposely structured only very loosely, highly distributed, and cross a wide variety of applications. Technically , the underpinnings of the community memory systems of the future is a technology called Hypertext, which is also the underpinnings of the World Wide Web. To understand what this all means, go back to the problem of the file clerks and see how Engelbart's system solved that problem.
Electronic mail is based on the metaphor of "sending a message." By definition, that implies that once sent, the message is carried from the sender to the receiver. Community memory changes that fundamental metaphor from "sending a message" to "pointing to a shared item."
Suppose that I am about to write an e-mail to you about a marketing experiment. This e-mail is just the most recent of a long series of messages about a marketing problem, various proposed solutions, and this one particular experimental approach to solving the problem. Now suppose that instead of sending that e-mail, I store it as a note in a shared folder on a server you can access, and then somehow bring your attention to the new item in that shared folder.
In this new world, quite a few things are different from a conventional e-mail system:
| There is no reason for you to file the note; it's already
filed in the right folder. It arrived "prefiled." |
| As soon as you look at this note, you have access to all
the other notes in the folder, providing you with a
constant context. |
| Any new colleague entering the conversation has access to the same folder and can immediately bring himself or herself up to date without asking anyone else for help. |
If all of this sounds very similar to the servers discussed in the last chapter, it is because the similarity is very real. In fact, the whole point of community memory is to combine the concept of the server as a shared filing system with the concept of the network as a communication medium. So what's missing to make the solution complete? Just convenience.
Most servers are designed to support users working with relatively large documents. Word processors are great for writing memos, reports, and books; a server acting as a file cabinet is designed to organize large documents of this type. A database works well with individual and smaller records, but as already mentioned, it is optimized for highly structured applications in which all the records are similar. What about e-mail? How would the server have to be designed to do a good job organizing e-mail?
165
Mail messages are typically quite short. The whole point of the In box is that it allows hundreds of these short messages to be scanned quickly and frequently. E-mail messages are meant to be read in context. A memo or other large document is generally written to be understandable by itself; usually considerable space is taken up at the beginning of many documents laying out background material so the reader can use the document in isolation. E-mail messages, on the other hand, are often not understandable by themselves. The operating assumption is that the reader is part of a longer conversation. In fact, there's even a name for that conversation: a thread. The whole point is that once an e-mail discussion is started, it often travels in several different directions, each of which becomes a thread in its own right. The entire discussion can then be thought of as a kind of tapestry built up out of many interwoven but separate threads.
What Doug Engelbart invented back in the '60s was the world's first bulletin board, called NLS (oNLine System). As you'll see later in this chapter, the system was really much more than just a bulletin board; but for now, even considering the bulletin board aspect of NLS gives you a valuable view of the community memory concept. So what is a bulletin board, and why is it such a compelling alternative to electronic mail?
What NLS provides is a structured environment for carrying out threaded conversations among large communities of users. The central organizing concept is the idea of the outline. Most people were first introduced to outlines in high school. Later in college, many people started using outlines as more than just a form of organizing information; instead, outlines became a great way to record all the information about a topic when taking notes or writing a paper.
NLS revolves around outlines, all of which are shared among all users of the system. Suppose that I want to start a conversation about marketing problems. I type in a few notes, perhaps as a single level at the top of a new outline. At this level, the NLS system looks a lot like a word processor of today operating in outline mode. However, there's a novel twist.
My note about marketing problems is actually part of a larger NLS conversational forum about marketing in general. This forum is shared by many users. In fact, like a conversation at a party, interested observers can add and remove themselves from the conversational forum at will. Every interested participant in a conversation is told about new conversational entries automatically. In this way, the NLS system functions just like an e-mail system. The e-mail system tells readers about new arriving messages; the NLS system tells conversationalists about new forum entries. There is one crucial difference, though.
In the e-mail system, each reader is working with his or her own private copy of the mail message; the mail system's job is to transmit copies of the message to all recipients. In the bulletin board system, on the other hand, each reader is working with a shared copy of each conversational item; the bulletin board's job is to make readers aware of new shared items without ever making copies of those items. In a very real sense, the bulletin board functions as a form of community memory.
166
In the NLS system, the file clerks are gone. Each reader sees new items, but they are always seen in the conversational forum where they were originally filed. Only the originator needs to decide which conversational thread an item belongs in; filing is done only once. Whenever a new participant joins a conversation, the entire contents of the bulletin board is available to him or her to catch up on history and become fully aware of the overall context.
If NLS was so great, why haven't more people heard about it; why aren't bulletin boards in general more successful? The surprising answer is that while NLS itself never became a large commercial success, it's successors -- Lotus Notes, CompuServe, and America Online -- have indeed become very popular. Looking further, bulletin boards have become a fundamental staple of underground communications worldwide.
While NLS, later called Augment and marketed by Tymshare, never really took off, Engelbart's original vision finally found popularity in the form of a system called Lotus Notes. Today, Notes is the first commercially successful form of corporate community memory .And because community memory is still such a new concept, people, even today, have a great deal of trouble understanding what Notes is really about.
Lotus Notes was first introduced in the early '90s, about the same time that Windows and LANs were first becoming really popular. At the time, tools for building graphical applications in the Windows environments had not become commonly available. Large organizations, in particular, were looking for tools to build graphical, multi-user database applications quickly and easily. Superficially, Lotus Notes looks like a tool that meets this need. Moving beyond NLS, it makes it easy for developers to design forms that can be used to participate in conversations. To support these conversations, it provides conversational forums, called databases. As early users quickly discovered, Notes is not well suited at all to building true database applications. The kinds of tools that support building this type of application are discussed in Chapter 15. Instead, what Notes represented was a quick and easy way of building friendly community memory applications.
Why does Notes succeed where NLS failed? NLS was built on large, shared, time-sharing systems that could be accessed only from terminals connected to high-speed lines. While NLS could be used over low-speed, dial-up lines, it lost most of its friendly graphical appeal. So NLS could be used only at locations where terminals and high-speed access were available. Furthermore, since NLS depended on both expensive networks and expensive central machines, it in turn was expensive to use. Notes solved all these problems.
Notes was one of the first true distributed client/server applications. Notes databases (recall, these are really conversational forums) run on servers based on ordinary per- sonal computers. Users in turn access Notes using ordinary personal computers running Windows. Finally, to make access really universal and convenient, Notes pioneered a technology called replication.
167
Sales and service organizations are the heaviest users of community applications like Notes because of their need to track customer contacts in a shared fashion. Suppose that a sales rep is about to call on a customer after several weeks away from the office. The first thing the rep wants to know is what interactions other people through- out the company have had with the customer. Logging into Notes, the rep enters the customer conversational forum, finds the topic for that customer, and looks for unread posted items. Notes flags these items for the rep's attention and by reading them, the rep is immediately updated about all outstanding contacts, all problems, and all proposed resolutions.
Watching the system in operation, the resemblance to NLS is easy to spot. The entire Notes system is organized around the concept of outlines. Each conversational area is one large outline. Drilling into outline topics reveals a hierarchy that can dive as far as the participants in the conversation care to go. By tracking multiple conversational areas, and multiple outline items, readers can participate in any number of conversational threads at one time. However, the real difference between NLS and Notes stems from the replication.
Even with WANs becoming widespread, nobody can be connected to the net all the time. Even when connected, quite often the connection is either slow or noisy, particularly while on the road. Replication solves all these problems by allowing users to keep copies of all the data they access most often on their personal machines. The trick is that the replication service keeps this data synchronized with the other copies, no matter where they might be. Consider an example.
After visiting the customer discussed in the previous scenario, the sales rep adds three notes to his Notes system. One note is a little trip report, one is a request for some new marketing material, and one is a note about a problem with a current promotional campaign. The notes, added to three different Notes forums, are entered offline while the rep is sitting on an airplane. On arriving back at the office, the rep connects his notebook computer to the corporate network. Immediately, his local copy of Notes contacts the copy of Notes in the local server, and the two copies exchange their respective updates. But the process does not stop there. That local server is only one of dozens, and each of those servers, in turn, supports many other individual users, each with his or her own notebooks and personal computers. Notes replication takes changes and ensures that they percolate all through the entire network, eventually reaching every server and every Notes machine organization wide. It is this pervasiveness that really makes Notes so useful. Now a user can have the advantages of the bulletin board's community memory and still retain the mobility associated with operating on his or her own personal machine.
What Lotus Notes has proven beyond a doubt is that community memory , in the form of a rich, customizable bulletin board system, particularly with high quality replication, is an effective next step beyond e-mail for large organizations. Notes works well within the boundaries of a single company. With its sophisticated security model and simple facilities for building custom forms, an organization can quickly and easily adapt Notes to specialized requirements. It's important to also recognize Notes' limitations.
168
First, it's not currently a complete replacement for electronic mail. For one thing, the population of Notes users is still relatively small, while the population of mail users is huge; therefore, the cost of cutting yourself off from the rest of the world by living only in Notes is just too high. Furthermore, Notes is best suited for facilitating somewhat structured conversations within the boundaries of a known organization, leaving e-mail still a better tool for the many random, unstructured, communications required in daily life. The fact that Notes is not a flat-out replacement for e-mail is really a non-issue because Notes and e-mail can very easily work together.
The second and more important limitation of Notes is that it is neither a database nor a tool for developing applications that drive core business processes. This observation often comes as a surprise to many Notes fans, at least initially. On the surface, one of the attractions of Notes is that it does such a good job of making shared information available throughout an organization. It's replication facilities are particularly appealing in this context because they make remote offices and laptop users first class citizens for the first time. At the same time, Notes also makes it pretty simple to develop graphical forms based applications that use this shared data. In fact, when Notes was introduced, it was one of the very few, perhaps even the only, widely available system that provided all of this in a Windows based, client/server environment. Little wonder then that eager developers saw Notes as their complete path into the client/server future.
Saying that Notes is not an appropriate tool for developing true database centered applications takes very little away from it. Notes is still a trend-setting, high-quality product with important benefits. However, when it comes to developing serious applications to run core business processes, it is not the right tool. Why? At its center, Notes it not a database. It has no concept of records. Transactions, record locking, and other key facilities required in the world of business processes are just not there. Notes was never designed to be a database in the first place, so this limitation is not surprising; yet it is still sometimes a surprise to users just becoming familiar with it. What this all means is simple. Notes is just one more part of a complete, client/server information system. It supplies bulletin board facilities, something databases don't do, and it does that job very well. No, it is not all things to all people, but then no tool ever is.
While Notes is a very strong community memory system for supporting individual organizations, what about bulletin boards for people at home, bulletin boards that cross companies, and generally, bulletin boards for the world at large? Do these make sense?
At the same time that Notes was starting to take off in popularity, an entirely different class of bulletin board systems was finding broad spread acceptance, as well. These bulletin boards take two similar but different flavors: one institutional and one homegrown.
169
On the institutional front, CompuServe, America Online, Prodigy, and several other providers have successfully built services reaching several million people allover the world. At first glance, it is a little hard knowing exactly how to classify these services. Each of them offers at least four core facilities to subscribers:
| Electronic Mail |
| Bulletin Boards |
| Public Databases |
| Electronic Shopping |
Reading an America Online ad clearly leaves the impression that all four of these facilities are equally real and equally attractive. Not so. First and foremost, in terms of both utility and utilization, is electronic mail. We've already talked about it quite a lot, but suffice it to say that for many subscribers, electronic mail is the main, and often the only, service they use -- and worth it, by far.
The bulletin board services offered by public systems like CompuServe represent a type of truly global community memory. While Notes allows individual organizations to pool thoughts, these public bulletin boards provide the same service literally to the world. A public bulletin board, like Notes, is organized around conversational forums and threaded conversations. However, unlike Notes, the range of topics is truly global. At one level, public bulletin boards often provide the best single place to go for technical advice of all kinds. By posting a question about even the most obscure technical problem, a user gains the advice of literally thousands of potential helpers worldwide. Furthermore, before even posting the question, the reader is more than likely to find the solution to his or her problem lurking somewhere in the thousands of notes and messages posted under most interesting topics on the board. But technical discussion is just the beginning.
Bulletin boards today have forums dealing with the weather, travel, entertainment, cars, sex, education -- in short, the world. It is precisely the broad nature of the boards, and the worldwide community that inhabits them, that can make them so addictive. Addictive, like a drug in many ways. Often users find themselves starting a bulletin board session early in the evening and terminating it only to find half the night gone by. So while e-mail may be the most heavily used service overall, bulletin boards are certainly the most addictive; and with the wide range of topics covered, bulletin boards are the most controversial.
What about public databases and electronic shopping? In theory, these services should be highly attractive. It is actually hard to imagine that electronic shopping, banking, and so on, will not one day be commonplace and heavily used (this concept is explored further in the last chapter in the book). Today, though, both electronic shopping and all forms of personal electronic commerce are talked about a lot and
170
used very little. Public databases are used slightly more. Accessing information about stock prices, airline flights, and so on, is useful, and some users use such services routinely. In fact, specialized "public" databases catering to stock brokers, travel agents, and so on, are highly successful businesses, but they serve niche markets. When it comes to serving the general public, though, public databases are used little more than electronic shopping.
So for all the discussion about public services, electronic shopping, and millions of users, what we have is a much simpler picture today, but with a twist. CompuServe, America Online, Prodigy, and their competitors do have several million users altogether. This is but a small fraction of the total personal computer user population, let alone the world population. Some people believe that the introduction of the Microsoft Network, with its close connection to Windows 95, will be the catalyst that finally brings public networks to the masses. Perhaps -but before leaping to conclusions, it is important to remember that today the two primary uses of these systems revolves around electronic mail and bulletin boards. The question then is, are these two services compelling enough today to really appeal to a wide population, or is more evolution required before the leap really happens?
No discussion of bulletin boards would be complete without touching on all the private bulletin boards located in homes and offices all over the world. As hackers came into contact with bulletin boards, many of them became addicted. It was only a matter of time until the first hacker wrote private bulletin board software. Today, by dedicating a single personal computer and a phone line to the task, any individual can set up a bulletin board in his or her house in just a few hours. These bulletin boards have become a true staple of the underground culture.
In its most innocuous form, teenagers in every city of any size use bulletin boards to exchange notes about their favorite music, chat casually, and meet other teenagers of both sexes. Bulletin board operators sponsor "GTs" or "Get Togethers," at which subscribers get to meet their conversational partners physically for the first time. All of this represents a new culture often unknown, and almost always totally alien to parents, even if those parents are computer literate products of the '70s or '80s. But bulletin boards go farther.
Of course, specialized groups -- private clubs or special interest groups -- can use bulletin boards to exchange information widely, but so can subversives and terrorists. What better way to exchange notes and disseminate information about targets and techniques. Of course, this is a two-edged sword. In times of repression, a single PC can quickly and easily be set up in hours to be a secret middleman keeping information flowing. Yet the same tool can just as easily be used by those whose cause is not as easily justified.
Starting with electronic mail as a cultural change agent, this discussion moved on to bulletin boards and community memory, in many ways an even larger change agent. Mail changes the work patterns of individuals, while bulletin boards affect groups of people. As you continue to consider the impact of networks, there is one more step that has potentially even larger effects. That step involves thinking even more deeply
171
about how people communicate and about the fundamental nature of the words they use to do it. The result is the creation of a form of memory that is not just group or community oriented, but actually global in nature.
Looking back at Engelbart's original NLS, you discover that he thought about text and information in two fundamentally different (but related) ways. Most of the time, he believed that users wanted to think about information in a hierarchical, outline oriented fashion. Outlines provide convenient viewing frameworks, make choices obvious, and provide simple ways to expand and contract the conceptual scope of the information being worked with. At the core, though, hierarchical views of information depend on the fact that the information is basically linear in nature.
Books, music, movies, television programs, and conversations are all linear in nature. One thought follows another, and there is only one way to progress through the sequence. Our thinking, however, is far from linear. And it is from that observation that the concept of hypertext springs. A hypertext is collection of information that attempts to capture all (or many) of the complex possible connections that can exist between pieces of information.
Imagine an author researching a book. Over time, the author collects hundreds of three-by-five index cards representing bits of data, thoughts about content, references, and so on. Eventually, carefully filtering and sorting all the time, the author arranges the cards into a single linear sequence that becomes the basis for the book. But suppose that the author didn't have to pick a single sequence?
Think of the book as put together out of thousands of little chunks or passages of text, each representing a single thought or concept. Now also think of threads running from each such chunk or passage to all the potentially related other chunks. To make the picture complete, imagine that the reader has a mechanism that allows him or her to decide, while reading each passage, which related passage to read next.
Encountering a book on programming, a beginner takes one path through the book, and an advanced reader takes another. Putting together a dinner menu, 25 different users of a recipe book may each have a completely different path through the exact same book. But hypertext doesn't stop with single books.
Suppose that you are reading a book about building design, and you encounter a reference about architecture. Following the link, you are now reading a completely different book which, in turn, points to an article about engineering stress calculation; that link takes you to a journal. The journal article briefly mentions a famous engineer and artist; you follow the link, and you are now reading an encyclopedia article. Following a few links in the encyclopedia, each of which instantly takes you to the relevant article (no pulling books off the shelf and no finding the page), you land on a reference about color design, which takes you to an obscure set of lecture notes located on a university server...
172
This is hypertext in action. There are many ways of thinking about hypertext, but the concept is indeed revolutionary enough to warrant having the many approaches for reaching it. Narrowly, hypertext presents a way of organizing information in a book or manual so that users can quickly follow references. Most modern help systems today use this approach to provide quick ways of jumping from topic to topic. The same system makes multimedia encyclopedias like Encarta flexible and fun, too. Looked at from another narrow perspective hypertext presents a way of organizing mail systems, filing systems, and bulletin boards more effectively. A pure hierarchical approach forces all items to be placed in only one folder. However, most memos and messages can easily relate to several topics. By providing facilities for placing these items in several folders, and for creating links between items, hypertext makes the whole system more flexible. However, Hypertext is broader than either of these two approaches.
Samuel Taylor Coleridge, a famous poet of the last century, dreamed of a fabulous pleasure palace called Xanadu. Upon awakening, he began writing down the verses describing this palace but was interrupted before he could finish. Later, he was never able to fully recapture either the vision or the words to describe it. Laboring through the '60s, '70s, and the '80s, Ted Nelson, an early hypertext visionary, had a similar experience.
Around the same time that Engelbart was building NLS, Ted Nelson was developing early text editing systems based on many of the same concepts. While Engelbart built a broader and more complete system, Nelson took the specific idea of hypertext much further.
Internal bulletin board systems create perceptions and memories that span organizations. Public bulletin board systems allow people from a wide variety of backgrounds and organizations to participate in discussions across wide distances. Still both private and public bulletin board systems tend to deal with conversations and topics that are somewhat limited in nature. This limitation stems for the hierarchical nature of the medium.
By definition, an outline deals with a single broad topic. Granted, the topic is divided and subdivided in turn, which allows for many twists and turns; still overall, the parts of the outline tend to be related to the broader outline topic area. Introducing a single new element, hypertext, changes the focus of the conversation tremendously. That new element is the pointer.
A pointer allows any conversational item or passage to be linked to virtually any other conversational item or passage. Creating links in this way allows arbitrarily sudden leaps of thought or obscure references to be created. More importantly, particular passages can now be included in any number of conversations. And that is what got Ted Nelson so excited.
173
Like many evangelists before him, Nelson wanted to solve problems on a truly global scale. In his case, Nelson's goal was nothing less than world understanding leading to world peace. How can this possibly be related to hypertext?
Nelson was fascinated with the problem of how ideas develop, spread, and become popular. The written and spoken word is part of this process. One problem with all the normal ways of transmitting information over time is the loss of attribution (attribution is the action of giving credit to the original author or the original source from which a piece of information is derived).
Say that I have an original idea that I publish in a paper or article. Somebody else sees this paper, borrows the idea, but fails or forgets to give me credit. At the center, this is the problem that Nelson's Xanadu system was designed to solve.
Xanadu was intended to be a worldwide hypertext system. It would be capable of being used by every living person on the planet, and would be able to store every written and spoken word, with all this information interlinked in the rich ways that only hypertext makes possible.
The thing that would really set Xanadu apart, though, aside from its sheer scale, was the implementation of transclusion. Transclusion is best thought of as a special or fancy kind of inclusion. The idea was to completely eliminate the need to ever copy text. Any time that anybody wanted to include a phrase, sentence, or paragraph from an original cite, transclusion would allow that inclusion to take place without copying. Using hypertext links instead, the included text would consist of an invisible pointer to the original. So if you wanted to use my idea, instead of copying the appropriate text from my paper, you would simply transclude it, incorporating it by reference. Now comes the special part.
Links of all kinds, including transclusive links, can always be viewed, followed, and traced. Thus a reader, starting with an idea, can always trace it back, farther and farther, to find the roots from which it stemmed. Thus Xanadu would provide a mechanism to support seeing where all ideas came from, and how all ideas are related to all other ideas.
Practically speaking, it's hard to even know how to think about Xanadu. Of course the system was never built; even today nobody knows how to even think about building such an all-encompassing system. The concept is obviously attractive. The idea of having all the information in the world stored in a single place; a place where every idea is linked to every related idea, with simple facilities to trace these links quickly and easily. Like the original Xanadu, this is all a dream, hard to even describe let alone build. As a real system, Xanadu today is just an interesting footnote in the history of computer science. As a concept, Ted Nelson and his Xanadu project had an impact that continues to this very day. That impact excited hundreds of key developers about hypertext and led directly to the World Wide Web and other related technologies that have very real life today and in the future.
174
Hypertext has seen several real implementations in the last 25 years. The first was found in the NLS system. While Engelbart's system focused primarily on outlines, NLS also contained rich facilities for creating pointers that reached across outlines. A particularly interesting aspect of these pointers, not replicated again until the World Wide Web, was the capability to point from one computer system to another completely different computer system, although both had to be running NLS.
In the '70s and '80s, a number of experimental systems at universities, particularly Brown University, explored the medium of hypertext in some depth. One of the interesting outcomes of this work was the realization that after a while, a new link between two concepts could actually have as much value as a new piece of content. The idea that a relationship could be as valuable as primary content itself takes some getting used to. In one example of this concept at work, students in a course were assigned the project of working with a large body of hypertext-based content with the goal of finding and creating a single creative and meaningful new hypertext link.
The first widespread commercial implementation of hypertext was the Macintosh's HyperCard system. Along with event-based visual programming, HyperCard pioneered the idea of allowing even users to create links between any two parts of a stack of cards. The mechanism used was simple. First the user would select a sequence of characters (usually a word or phrase) to be the beginning of the link. After highlighting that text, the user would activate the Create Link command. HyperCard would ask the user to then navigate to the links destination point and click on a completion button. At that point, the link was complete. A reader encountering this card later would see the link text highlighted, as a kind of hot spot. Moving the mouse pointer over the hot spot changes its shape to a hand or some other evocative icon; clicking the mouse takes the reader to the destination. The primary limitation of HyperCard was that these simple links were confined to single stacks of cards.
After HyperCard, thousands of other software systems used the same basic technology to build hypertext-based help, hypertext-based encyclopedias, movie guides, and so on. In a short period of time, the idea of having pervasive and ubiquitous links all through a body of information had become commonplace. The primary limitation of all these systems was that each was self-contained, dealing with a single body of material at one time.
As hypertext was becoming common, the Internet was growing, and with it servers by the thousands were attaching themselves all over the world. Finding information within a server was often quite easy with many different types of search engines helping the user through the process. But finding a server in the first place was hard. That's where the next generation of hypertext enters the stage.
The first World Wide Web development took place at the CERN research lab in Switzerland. Programmers working there realized that the concepts of hypertext could be used to tie servers together so that users could leap from server to server with just a few keystrokes. In one fell swoop, sets of previously separate servers could now be
175
thought of and worked with as though all those servers were part of a single, large global hypertext. Hypertext itself is often best thought of as a large web tying together otherwise disparate thoughts. This new web, unlike any before it, as it jumped from server to server, broke new ground by being world wide; hence, why not name it the World Wide Web (or WWW).
The initial WWW system was text oriented like much of the original Internet itself. The most important single invention that set the WWW apart from its predecessors was HTML, short for Hypertext Markup Language. HTML, in turn, is a specialized form of SGML, Standard General Markup Language. A markup language is a notation for describing the formatting of a document. Markup languages were first used by book and newspaper editors who would scrawl quick notations onto a story or book to indicate paragraph breaks, missing words, phrases to be printed in larger fonts, and so on. More recently, markup languages have become associated with high-end word processors and desktop publishing programs used to produce particularly complex documents. A markup language allows an editor or author to describe very powerful formatting operations to be applied to parts of a document. The author's instructions are identified by special lead-in characters to distinguish them from normal text. Once the lead-in characters are identified, the markup language can then contain arbitrarily complex sets of instructions.
In a way, the return of markup languages is somewhat ironic. The first PC-based word processors, like WordStar, used simple markup languages as their primary formatting mechanism. Millions of early PC users learned to love, or at least tolerate, these "dot" commands. One of the primary benefits of graphical word processors was the complete elimination of visible formatting commands; markup languages appeared to be gone forever. Instead, the user formatted text, the formatting was directly visible on the screen, and that was that. In fact, the word processor stored the formatting directives, in hidden form, in the document; the user could even ask to see them if desired.
HTML is a particular form of markup language designed with two purposes in mind. First, HTML provides formatting primitives particularly suited to the kinds of simple two-dimensional pages so common on the Web. Second, HTML provides navigational primitives. Essentially, these primitives allow one WWW page to point to another. All of these HTML primitives operate in design mode and in user mode. In design mode, each primitive is a visible piece of text, a sequence of characters that can be directly typed by a person. In user mode, the HTML directives all disappear. Instead, the user sees a formatted page. In place of navigational directives, the user sees highlighted text; when the user points to that text and clicks, the user jumps to the page being pointed to by the underlying HTML directive.
Why focus on an actual HTML language at all? To begin with, the HTML makes WWW pages independent of any particular workstation. Formatting directives can be interpreted one way under Windows, another way on a Macintosh, and yet another way on a character-oriented terminal. Secondly, and even more importantly, because HTML consists entirely of text strings, it can easily be generated by a computer. Thus HTML makes it quite easy to generate WWW pages programmatically.
176
The basic WWW, even including HTML, laid the foundation but still was not quite enough to kick the whole effort into high gear. The last kicker was the development of a graphical front end called Mosaic, written by Marc Andreessen, a programmer work- ing at the supercomputer lab located at the University of Illinois. Mosaic was an application capable of running on a wide variety of workstations that made navigating the Web a point-and-click affair. Suddenly, users could traverse pages all over the world with hardly any typing. The rest, as they say, is history.
What makes the Web so different than everything that came before? The first question, of course, is what did come before? The two answers have to be bulletin board systems and the Internet.
Compared to the Web, a bulletin board system, for all its variety, is far more insular. Bulletin board systems basically run on single servers. Yes, Notes has a sophisticated replication scheme, but the point of that scheme is actually to make a network of computers all look like a single large computer. Another way of thinking about it is that Notes' replication scheme creates homogeneity: any change made anywhere is then made to appear everywhere. Bulletin board systems also tend not to be highly graphical; this is a direct consequence of the fact that they tend to run computers shared by very large numbers of people. It is also related to the fact that BBSs tend to revolve around conversational threads and topics, which in turn tend to favor text-based interaction. So the Web, beyond a limited resemblance, is not very much like a bulletin board -- at least not today.
The Web fundamentally transformed the Internet by making it, for the first time, truly easy to use. The Internet, until very recently, showed a clear family resemblance to the UNIX operating systems that so many of its servers ran. And before UNIX, Tenex, Tops 10, and other hacker-oriented operating systems were dominant. As a result, the design of FTP, Gophers, and all the other utilities that made the pre-WWW Internet work favored powerful but cryptic commands that were anything but friendly. While HTML itself is anything but friendly, the Web itself, and the browsers that run on it, hide that complexity almost totally.
While the Web was different than either the Internet or BBS, it borrowed key features from each of them. The fascinating, even addictive, idea of being able to browse through a wide variety of topics just by following links from place to place came directly from the Bulletin Board. In fact, even the idea of having a simple point-and-click interface was really driven by Notes, PCs, and simple mass market systems. On the other hand, it was the Internet, with its globe-spanning connections, that made the heterogeneous and cosmopolitan nature of the WWW imaginable and possible.
The Web is unique in a way that no predecessor can really claim to copy. On any given day, a Web crawler (a person "crawling" around the Web) really has no idea at all what he or she is going to find. Leaping from server to server literally can mean
177
hopping from country to country and continent to continent. But the variety reaches much deeper than that. Until the advent of the Web, any computer-based information system had a certain built in degree of uniformity. Even if programmers were quite creative, essentially all of them worked with the same tools, in the same system environment, and built relatively similar applications as a result. The Web, however, consists of completely separate computers, each playing the role of server, and each owned by completely different people and institutions. A developer wanting to connect to the Web has almost total freedom to decide what computer to use, which operating system to run, what tools to use, and so on. It's true that certain standard protocols -- TCP/IP, HTML, GCI, and so on -- are required to make the connection work; however, once those are in place, anything goes.
The variety of material found on the Web is literally, already today, beyond imagination. One server, connected to a camera, offers a view of a college dorm; another offers a picture, updated daily, of an individual's lunch bag. Stock prices, flowers for sale, personal companion ads, up-to-date news -- the list goes on and on. If a developer can imagine it, and a computer can be programmed to do it, that computer can then offer its presentation over the Web. And as quickly as other users discover interesting new pages, they can add links to them, and those pages become part of the Web and are easy to find. So how do we understand this tremendous variety and what it really means -- why it makes the Web so compelling, interesting, and different?
In one step, the Web has brought a major new element to community memory systems: the concept of cooperating components. A bulletin board system is essentially monolithic. For all of its sophisticated replication and its capability to run across thousands of machines, Notes also is basically monolithic; the whole point is that it presents a single system image to the user. But the cost is that developers can build only what Notes lets them build. The Web is different. Every Web server can be totally different than any other Web server, yet they all communicate with each other using standard Web protocols. The result is the fascinating, distributed, heterogeneous environment seen today. As an example of the power of cooperating components, the Web is outstanding. That example is not enough yet for us to stand back and see the whole picture. What it does offer is the first example of the power of this new paradigm.
Starting with the simple idea of a network and servers, you now have several very different systems, all of which provide ways of bringing people together. How do all these different systems fit together, and what does the future hold? Begin by reviewing the systems:
178
| Electronic mail allows individuals to communicate with
each other, competing with the telephone, memos, and
regular mail. In its simplest form, e-mail offers
one-to-one communication. |
| Bulletin boards make it easy for large groups to exchange
information with each other. A note posted in a
conversational forum lives on indefinitely and can be read
by a number of people. BBSs offer many to many
communication. |
| Finally, the Web provides a mechanism to disseminate information from a single person or group to a very large number of people. The Web is different in that the communication is entirely one way. If I create a Web page, only I can change that page. Anybody can read it, but nobody except me can write to it. So the Web essentially offers one to many communication. |
What this model for electronic mail, BBSs, and the Web really shows is how different these systems are from each other. It also shows how immature networking technology still is. That maturity shows up most acutely when you start to think about how to build a system that combines the best of electronic mail, bulletin boards, and the WWW; it's not at all clear how to do that.
The true promise of wide-area networks is to eliminate the effect of distance -- to make the world into a global workgroup. This change is a fundamental cultural change, one that will affect workstyles, personal relationships, and eventually even the very meaning of work. How close is all of this to happening? There are two ways to think about that question.
[Tom Note: remember, this was written in 1995.]
At a macroscopic level, it's pretty amazing to compare the claims for today's networks with the reality. By reading popular articles about the Web and the popular bulletin board services, it's easy to believe that most homes and offices are connected, shopping is about to disappear, and most commerce takes place over the net. The reality is that fewer than ten million individuals, out of a world population of billions, use the Internet on any kind of a truly regular basis. And when you consider the true usage patterns, it is electronic mail -- not shopping, not banking, not even the Web itself- that is by far the dominant application. Again and again, technical pioneers report about their experiences as they start to explore the Internet. First, they quickly discover how very limited the services of the future are; shopping, banking, and commerce over the net are still safely in the future. Then they do find the Web itself to be interesting and even addictive. Finally, when time has passed and the dust has settled, the reports trickle back in, and electronic mail is it.
At a more microscopic level, why are e-mail, bulletin boards, and the Web so separated from each other? For example, why can't a bulletin board system be geographically distributed, just like the Web is? Doesn't true hypertext linking make just as much sense for bulletin board style conversations as for relatively static information? And what about e-mail? If bulletin boards are such an effective means for carrying on
179
threaded conversations, why haven't systems like Notes merged with systems like cc:Mail and Microsoft Mail? The clear separations between all these different applications is a more detailed sign of the development and continuing change still ahead of us.
The global village is almost at hand, and the impact of the WAN on our lives is hard to over estimate. WANs will change the way people communicate and relate to each other within and across organizations. The next thing to look at is the role the computer will play in running the organizations themselves. This is the role in which the computer functions as a computing and management engine, alongside the role just explored in which it functions as a communication agent.
As you think about all the implications of networking, go back to the idea of cooperating components for a moment. Recall that the thing that makes the Web stand apart is that it is an early implementation of cooperating components. Yet the Web today provides functionality that is too limited to really fulfill the promise explored in Chapter 3. In fact, the Web is too limited to even compete with, let alone replace, the other dominant systems on the WANs of the world: e-mail and BBSs. Yet from a broader perspective, the beginning of a bigger answer is clear.
Imagine a Web-like environment. Users can hop from server to server quickly and easily. Now, suppose that some of those servers have grown up to the point where they are running part of the business. Suppose that sometimes when hopping from server to server, there's no person involved; it's one organization, one server, talking to another. That sounds like where you need to get, right? The question is how do you build the servers that run the business? That's where the next chapter comes in. In fact, the next chapter gets to a key question that's been looming since Chapter 1.
All these distributed servers are really great, but can these systems really replace today's mainframes, and if so, why hasn't it already happened? To answer this question, you need to take a look at the mainframe itself and at what -- if anything -- makes it special.