Online Information Retrieval – A New Business Tool

During the early 1980s, a radical new toolset was exploding on the scene… a sort of paleo-Google that was primitive by modern standards, but was life-changing for those who could adapt and learn to incorporate online information retrieval into their work. I covered this industry for Online Today and a few other magazines, and the article below was an overview of the entire field… slanted to the business community. From the perspective of a few decades later, it is fascinating to recall a time when online databases were rigidly structured, expensive, and very intimidating to the non-technical user. Here is some history of that early age of online-information systems…

by Steven K. Roberts
Online Today 
October, 1983

Not long ago, a couple of basement experimenters with an exciting new product idea approached an investor in Columbus, Ohio. “We’ve figured out a way,” they said, “to completely revolutionize the car stereo industry.”

The investor, though unfamiliar with high technology, has heard enough of these classic success stories to be intrigued. His eyes lit up. “How?”

The inventors explained their concept for a digital audio disk machine that would use a laser to play prerecorded media, very much like the videodisks that were appearing at the time (mid-1982). Four hours of music could be stored on a small and rugged disk, they claimed, and the sound reproduction would be of unprecedented quality. The units could even be made programmable for selection of desired musical “cuts,” eliminating one of the traditional complaints about magnetic tape.

Fascinated, the investor called a few of his friends. He discovered that the potential market was huge and saw no reason why the idea had to be limited to car stereos. Here was his chance to get in on the ground floor of a big one! They began negotiating.

Later, as he was discussing the matter with his attorney, the investor was advised to have the technology independently researched before risking $50-75,000 in the proposed startup. The attorney knew of a small on-line searching firm that might be able to help.

“Sure, why not?” The investor and his newfound associates appeared at the door of the Information Institute — and during the following hour suddenly found themselves face-to-face with reality.

They discovered that many companies (among them Philips, Mitsubishi, and Sony) had already announced working prototypes of digital audio disk players that had been under development — and in the news — for four or five years. In less than 30 minutes of on-line searching via Lockheed’s DIALOG system, the dismayed trio was treated to a succession of technical article abstracts, news stories, conference papers, and patents having to do with their “hot new idea.” The investor was billed for $32.60 in on-line time plus a consulting fee; the designers walked away crestfallen, and it was business as usual for the information broker.

This sort of thing goes on daily. Have you ever scooped rotten leaves out of your gutters, for example, and dreamed up an expandable-arm gutter cleaner that could simplify the job and make you a fortune? Forget it. There are over 20 patents on variations of that idea already.

Have you ever moved in the wrong direction, discarded a good idea, settled for a “shotgun” marketing approach, or failed to accomplish a critical business objective — simply because you were unable to locate a few particularly critical facts? If so, you’re not alone: there’s hardly a company in existence that could honestly deny it.

The business world abounds with unanswered questions. How many typesetters are in Massachusetts? What is the U.S. market for medical razors? Are there any lawsuits pending against Tandon Corporation? Is there a correlation between chiropractic cervical manipulation and stroke? What are the latest rulings by the Commissioner of Patents and Trademarks affecting prior use of an unregistered logo? Is there a book about organo-tin compounds that explains barnacle-proofing? How has Chrysler been advertising lately? Is there a trade association for maraschino cherry producers? What does it take to weld bismuth? Are nitinol heat engines viable? Where can one find an expert in robot vision systems? Should you lease or buy a punch press? Is your product marketable?

Traditionally, such questions have been a pain (to put it gently). Many of them still are, but every one is directly approachable through on-line searching.

How? This article will acquaint you with a technology so powerful that you might even want to change the way you do business… before your competition does.

A new business tool

Our subject is on-line information retrieval. Before going any further, it would be wise to agree upon a preliminary definition of that term.

The words “on line” have been applied to a lot of computer-related phenomena, including everything from communication with timesharing services to the current condition of a computer terminal. None of those traditional meanings adequately express the kinds of capabilities we will be discussing.

We will be talking instead about a new business resource of staggering significance: an “information utility” that can give you access to more collected human knowledge than any library in the world. You’ve been hearing about the “information explosion” for years, but only recently has it become practical enough to be truly relevant to the individual.

Kinds of information

It has become common knowledge in recent years that one of the powerful trends affecting the American business community is an unmistakable shift from an industrial to an information economy. This has manifested itself in many ways: the personal computer explosion, the effect that Megatrends author John Naisbitt called the “collapse of the information float,” the relocation of the workforce away from manufacturing, and more. All of it affects your business in a variety of ways.

When we talk about business information, two broad categories come to mind. First, there’s the “internal” information associated with your company: financial records, payroll data, word processing files, production reporting, and so on. Some of this may reside outside your physical premises in a timesharing system, but it is nevertheless your information — and definitely not for public consumption.

The second kind of information that affects your business concerns the rest of the world: the latest publications about robotics, recent legislation on leaseback arrangements, what your competition is up to, and so on. Without access to this, you operate in a vacuum and risk unpleasant surprises from your market or your competition.

The traditional “ear to the ground” has involved careful scrutiny of everything from trade journals to the rumor mill. It has never really been possible (without substantial effort and expense) to track industry activity in a timely and accurate fashion until relatively recently.

The information that interests us here is almost entirely of this second variety. We won’t be discussing ways to get extra mileage out of your accounting system or build your own in-house databases of corporate information. Instead we will be giving you the tools necessary to tap a resource that has grown within a decade from obscure academic roots into a $2-billion-plus industry. Your “ear to the ground” can now be exquisitely sensitive, global in scope, and remarkably affordable.

Information: Where it comes from

Imagine, if you will, an organization of many people with various specialties, each sitting at a computer terminal. All day, every day, they work through thousands of magazines and journals, keying bibliographic references and abstracts for every article into the system. On a regular basis, these records are integrated into an expanding computer resident database of similar information — perhaps already consisting of well over a million items going back 10 to 15 years.

Imagine further that this database is housed in a computer system that has been equipped with the software to search through it very efficiently, allowing someone to request, for example, “all articles since 1975 that mention both Subchapter S corporations and profit-sharing plans.” Or, perhaps, “all articles in TODAY Magazine that have something to do with French videotex systems.”

When such requests can be made easily, and if the results can be had in a few seconds, then this stored database of bibliographic references and abstracts becomes a powerful research tool. No longer is it necessary to go to the library and painstakingly scan the table of contents or annual index of every magazine (a process that is not only notably inefficient but also incapable of yielding good results). Suppose that the search for information about profit-sharing plans in Subchapter S corporations yields 44 articles, and that one of the more interesting ones is entitled. “New Tax Legislation Closes Loophole.” Any manual search of titles that picks that up is a very careful one indeed.

Now suppose that this imagined database is one of a thousand or so, and that some of the others cover such diverse sources as:

• The National Library of Medicine
• All U.S. patents since 1950
• The Congressional Record
• Commerce Business Daily
• Various directories of manufacturers
• 10,000 reports on all SEC filing corporations
• Who’s Who
• Economic and marketing forecasts
• The national Yellow Pages

and hundreds of other collections of information totaling over 100-million records. Imagine them all in the same system, readily accessible by phone from your office, and you have a reasonable image of on-line information retrieval.

The implications of this capability tor the business community are profound. Some of the sample questions in the introduction would be almost unapproachable without either an online terminal or a huge research budget.

The implications

The opening scenario in which the investor saved himself a bundle by exposing the naïveté of the audio disk “inventors” is not fiction. During the on-line session, some 2,600 technical journals abstracted in the INSPEC database were scanned all the way back to 1969, as were U.S. patents and other sources of information.

Clearly, this kind of capability implies a remarkable resource for the business community. Actual execution of the INSPEC search, including printout of promising “hits,” took a little over seven minutes. The cost?

$10.38 connect time
$.73 Telenet communications
$3.00 typing 12 records
————————–
$14.11 Total

That’s not bad. Compare it with any other kind of business research — keeping in mind human time, transportation, and overall effectiveness — and you will see why the on-line phenomenon is causing a stir. Of course, it hasn’t been much of a stir as far as the world at large is concerned, because the information industry has traditionally been one of those “incestuous,” academically flavored subcultures. It has been growing in sophistication at a remarkable rate, but has lacked good public relations. Publications have been mostly internal, and the conferences, though spirited, have only recently started attracting outside business and industry.

Yet the on-line world has spawned a set of well debugged resources that have staggering market potential. As one information industry executive noted recently, “You can assume that more than 95 percent of all information published in the last 12 to 15 years is accessible online.” Now that it has “come out of the closet,” online is proving to be indispensable — and the implications for those who ignore it are not pleasant.

Let’s look at some of the reasons why the field is suddenly growing so rapidly:

Scope. We have already hinted at the sheer quantities of information to be found in on-line databases; Nearly two million records on agriculture. Over a million conference papers. Over a million in the Department of Energy file. Over 10 million records in the Electronic Yellow Pages. Over two million in the Funk & Scott index of companies. Four and a half million in just one of the many scientific files. This adds up to a wealth of multidisciplinary information that is readily available to anyone with a terminal — and it is updated daily.

Interactiveness. Much of the effectiveness of on-line searching is directly attributable to the dynamic and interactive nature of the system. In the Old Days, one had to operate in a batch mode, wherein requests for information were submitted, run by the system operator (often overnight), and answered many hours later. This was frustrating not only because of the time delay, but also because much of the art of good research comes from the ability to continually refine one’s inquiry based upon intermediate results.

You could, for example, find over 2,500 articles on equipment leasing in the ABI/INFORM database. But this is far too much information for the question that inspired the search, so you can restrict the choices by date, language, type of leased equipment, and even article slant. If you are interested in the lease/purchase trade-offs in a hospital’s acquisition of an X-ray machine, for example, you can quickly restrict the choices to a single article:

81004686 Buying Diagnostic Imaging Equipment: Points to Ponder, Pitfalls to Avoid
Robb. Walter L.
Hospital Materiel Mgmt Qtrly v2n3 PP: 31-37 Feb 1981 ISSN: 0192-2262
JRNL CODE: HMM
DOC TYPE: Journal Paper
LANGUAGE: English
AVAILABILITY: ABI/INFORM

In the past, hospital radiologists have enjoyed almost complete authority for selecting diagnostic imaging equipment because this equipment has been considered too complex and specialized for the normal purchasing routine. However, in the last few years, as a direct result of increasing pressures from government and consumer agencies to control health care costs, hospitals are now more likely to handle capital equipment acquisition. To participate effectively in the decision making for this type of purchase, there are certain things that a materiel manager must know. Diagnostic imaging equipment may be classified into 4 categories:

1. X-ray 
2. computerized axial tomography 
3. nuclear 
4. ultrasound

The simplest way to acquire capital equipment is the straight purchase in which the manufacturer agrees to deliver and install a system at a fixed price. Managers need to consider preparation of a room, delivery and installation, and clarification of purchase terms. Leases, warranties, and equipment service are other matters that materiel managers must analyze in the purchase of diagnostic imaging equipment.

DESCRIPTORS: Hospitals; Medical; Equipment; Purchasing; Equipment acquisition planning; Materials management: Leasing; Warranties; Services
CLASSIFICATION CODES: 8320 (CN=Health care industry): 5300 (CN=Production management); 7500 (CN=Product planning & development)

Speed. Swiftness of information retrieval is a major boon in a business context where questions are seldom the result of idle curiosity. It is possible to be on the phone with a potential corporate client and quietly call forth on your CRT screen his latest balance sheets and income statements, pending litigation, news about the company, a description of its products, plant information, recent acquisitions, names and salaries of officers, and so on. This can be very disquieting to the person on the other end of the line, subtly shifting the balance of a negotiation.

Cost. On-line searching seems expensive at first glance. Idle perusal of current database catalogs will reveal hourly connect time figures ranging from $35 to over $300 an hour. Searching chemical patents can cost you as much as $5 per minute — although the average database runs only a little over $1 per minute.

But you can get a lot done in a short time. It is doubtful that even an expert manual researcher in the most complete library in the world could have found all those articles and audio disk patents — and certainly not in less than half an hour!

Extra Features. There are a number of interesting things you can do to extend the system’s usefulness beyond “straight” searching. One of these deserves particular mention — the SDI service (Selective Dissemination of Information) offered by both DIALOG and ORBIT.

Suppose you are in the toothpaste business and are concerned with ongoing work in amorphous precipitated siliceous pigments. (They make great clear toothpaste abrasives.)

You would certainly kick off your research with substantial on-line time, but it might become too expensive if you signed on every week to check for new patents or publications in the field.

No problem. Execute the search once, but instead of simply logging off the system, type END/SDI (if you are on DIALOG). Henceforth, you will automatically receive a monthly mailing containing abstracts of the latest information that meets the criteria of your search. The cost is minimal, and it keeps you up-to-date in a way that no other kind of information resource can.

Capabilities such as these suggest that the long promised “Information Revolution” is becoming enough of a reality to take seriously. It is no longer necessary to be a computer wizard to become constructively involved in system use. although in all fairness, it must be admitted that the available services do require the development of some expertise before they can be used effectively. It is for this reason that database “intermediaries” have sprung up all over the country to give you the unmistakable business advantages of on-line use without having to pay for a learning curve.

But that learning curve is not particularly long or steep, at least for the average business’ requirements, and there is enough information available to help you take off on your own if you see no need to support the overhead of a consultant, Take your pick.

All that knowledge…

This talk about huge storehouses of readily accessible information calls to mind the attitude with which many people still approach computers. “Have you got anything in there on me?” “Can I ask it something?” “Gosh, how does it know?”

The fact is, of course, that it doesn’t know. It is tempting when dealing with computers that contain vast amounts of human knowledge to expect a modicum of human wit, but this is unfortunately not the case. When you type in a search strategy with the intent of extracting a desired piece of knowledge, you will be taken literally. An example will demonstrate this:

One of my first experiences with the DIALOG system took place during the research phase of an article about on-line searching. Armed with a password and amazed by what I had already seen, I decided to have a look at the U.S. patent files. Being of a somewhat lascivious frame of mind, I inquired about sex-related inventions by simply searching for the word “sex.” There were hundreds, of course. Trembling with anticipation, I ordered the first one to be typed…

It was a method for inducing the early flowering of young deciduous trees.

As you learn on-line searching technique, it becomes apparent that much of your success depends upon constant awareness of the fact that the machine is not intelligent. It won’t automatically provide plural forms, overlook misspellings, give you articles about GE when you ask for General Electric, look for “paraffin” if you are trying to find “kerosene” in England, or understand that Steven K. Roberts is the same person as Steven K Roberts. It can be annoying. All that knowledge — and so little sense.

The information industry

From difficult and obscure beginnings in the late 1960s, there has evolved a complex and fiercely competitive industry. The annual National On-line Meeting this year attracted some 3,000 registrants and hosted over 100 exhibitors — each actively purveying databases, on-line services, private file support, consulting, and more. Let’s look at the classes of information service and the entities that have appeared to market and support them.

There are four major categories of on-line information vendors:

• Database Supermarkets. Typified by DIALOG, ORBIT, and BRS, these systems offer dozens, sometimes hundreds, of different databases. These are perhaps the most visible entities in the on-line world because of their diversity and size.
• Specialized Services. These systems, such as Mead Data Central and Data Resources Inc., host their own files rather than those from a variety of producers. Many offer unique high-quality services unavailable elsewhere, such as Dunsprint from Dun & Bradstreet.
• Timesharing Services. These would not even be mentioned here but for the fact that they overlap the on-line industry in many places. A number of companies that have traditionally been in the business of marketing computer time to business customers now offer a number of database services as well.
• Personal and Videotex Services. These are a relatively recent addition to the on-line world. With less sophisticated search capabilities but a much wider range of services, these systems are significantly less expensive and address a broader market — yet they overlap the “hard core” database services in many areas.

These four classes of on-line information resources represent the logical divisions within the industry (the major types of “product lines,” if you will), but they do not, alone, tell the whole story. As in any industry, there are manufacturers, wholesalers, retailers, consultants, trade organizations, and various classes of end-users.

Database producers are the original publishers of the information that is intended for marketing via an online system. One of the premier general business bibliographic databases, for example, is ABI/INFORM. Although you access this on a “supermarket” system such as DIALOG, the database is produced by Data Courier Inc. of Louisville, Ky. Other databases are simply machine readable versions of printed publications. Database producers may be thought of as the manufacturers.

On-line vendors are the systems with which you actually interact — whether they are supermarkets, such as DIALOG, or specialized services, such as the International Monetary Fund database. These services can be loosely compared to wholesalers in the traditional marketing chain.

Intermediaries are something of a cross between retailers and consultants, and are becoming an increasingly significant part of the on line industry. One of the options that you have as a consumer of information is the use of a trained intermediary to do the searching for you. In this category we might include document delivery vendors who will locate and provide copies of the original documents discovered during an on-line search.

End users, of course, are the ultimate consumers of the information. Not much of the foregoing would be significant without a market, and it is the rapidly growing population of end users that provides it.

The on-line industry is young and still suffers from many of the standardization and philosophical arguments that affect other new businesses. One issue not likely to be so easily resolved is the cost of searching. For a variety of reasons, the traditional method of charging for information — hourly connect time — is becoming less and less useful.

What price information?

If you ever want to get a spirited conversation going, just wander into a large library and start talking about how patrons should be charged for on-line searching. Perhaps because libraries are traditionally free, this has become a major issue. In 1977, in fact, the American Library Association asserted that “the charging of fees and levies for information services, including those services utilizing the latest information technology, is discriminatory in publicly supported institutions providing library and information services.”

While this may not be a critical issue to the average business user of on-line services, it is indicative of the seriousness with which this new technology is being received among the traditional purveyors of information. Libraries are, in effect, being asked to subsidize the high costs of on-line searching if they wish to offer the best possible service to patrons.

Many openly defy this policy, of course, and have begun adopting alternative approaches. Some split the costs between library and patron, others have established flat fees based on the cost of an average search, and still others offer a limited amount of free on-line time and then pass through all vendor charges after that. The ones staying with subsidized on-line services have begun to notice that they are losing money at an alarming rate: Not only are vendor charges regularly increasing, but free searching encourages sloppy techniques and allows inefficient methods to proliferate.

Outside the library, the question of free on-line searching seldom arises. But the cost of information is every bit as difficult an issue in the business world.

One of the reasons for this is that people are generally accustomed to making only a single payment for a printed volume of information — a book — which may be used for years and passed around without further cost. This renders $1/minute information retrieval costs a bit startling at first glance.

It turns out, however, that the efficiency of on-line searching coupled with the human cost of manual alternatives renders on-line charges quite reasonable for most applications. But suppose you need frequent access to the same database? A directory, for example, such as Books in Print, which would otherwise take the form of a set of bound volumes? This is available on-line for $65/hour, which might seem a questionable alternative to the printed version until you consider the host of extra searching features.

This discrepancy between traditional and new information costs has led to some interesting problems. In the case of the directory, there is nothing to stop you from doing a very general but well-tailored search once, storing all the results on disk in your own computer. With easily-available database software, you can subsequently search this to your heart’s content without any additional charges, perhaps signing on to the big system once a month to update your copy of the file.

Database producers look unkindly upon such copyright violations, but are apparently powerless to do anything about it. This is a situation comparable to that of software piracy that has surfaced in the personal computer world — there is ample economic incentive to cheat, and the chances of getting caught are slim (see “Database Downloading,” August TODAY, pages 27-31).

Additionally, on-line charges in systems such as DIALOG are primarily based on connect time. If you buy a 1200 baud modem to replace your 300 baud unit, you have improved your communication speed by a factor of four and substantially cut your searching costs. A local microcomputer can slash them further by retransmitting previously prepared search strategies in a fraction of the time it would take you to key them in online.

None of this is particularly pleasing to the on-line vendors, who have moved out of academia into the business world and are correspondingly more attentive to the bottom line. Ideally they would like to charge users up front for the privilege of using their system (still at a stiff hourly rate, of course), thus recouping production costs immediately and acquiring tangible data about the size of their user population. Some systems — notably the more specialized ones such as DRI’s econometric databases — manage to get away with this, charging front-end subscription fees of $20,000 and up.

But users prefer the “pay as you go” approach, although the straight connect time basis is becoming undesirable from even their standpoint. The problem here is that the more users a system has, the slower it gets. It is, in fact, contrary to my own best interests to write about the field: Every time someone else gets enthusiastic and signs on with a database service, my average searching costs increase slightly! The vendors try to control this with ever-faster computers and communications processors, but system behavior at peak times is still noticeably sluggish.

The net effect is a trend away from straight connect-time charging toward a scheme that is based more upon the actual amount of information delivered. This will likely evolve into a basic charge for the time spent on the system, plus a fee based upon the number of records or abstracts actually displayed or printed.

Whatever happens, you can expect considerable flux in the coming years; this industry, though already sophisticated and immensely useful, is still suffering growing pains.

The future

It is entertaining in any new and fast-growing technology to speculate about the future. Where is the information industry going?

In general, we can observe that the systems will become more responsive as problems of user overloading are solved, and that more data will continue to become available online every month. Further, it is safe to predict that there will be more full text databases appearing — those that provide not only a bibliographic record and abstract of an article, but also the article itself. Mead Data Central’s Nexis (news) and Lexis (legal) databases have always been this way, and others will surely follow as storage costs continue to decrease.

The technological changes are perhaps the most obvious. Information handling systems are continuing to develop at the same dizzying rate that has characterized the last decade, and this will affect on-line information retrieval in a number of satisfying ways.

First, as noted a moment ago, we can expect ongoing increases in the amount of data available. With the average annual 30 percent decrease in the per-bit cost of mass storage, it is only reasonable to assume that more information sources will be brought online.

Second, we can expect an extremely sharp drop over and above this 30 percent soon — laser disk technology is promising mass storage costs of .00000001 cents/bit within the next five years. Already it is possible to store 108,000 video frames on a single low-cost disk, and this is being used in the on-line industry as a delivery vehicle for patent drawings (Video Patsearch from Pergamon). In this system, standard methods are used to search the remote patent database, whereupon a local processor notes the patent number, accesses the videodisk, and displays the applicable drawings on a TV screen. The drawings for the nearly 850,000 U.S. patents issued since January 1971—along with over 10,000 foreign patents — occupy only nine videodisks.

But this is only one of a host of new techniques for information delivery. At present, most on-line searching takes place at either 30 or 120 characters/second — hardly sufficient for cost-effective retrieval of full text documents. New transmission methods involving satellite links, coaxial cable, and fiber optics will be tried, yielding long overdue alternatives to the present slow data transmission schemes.

Meanwhile, terminal costs are finally decreasing, making those traditional slow methods easier on the pocketbook. There are already a number of portable terminals on the market which, while not easy on the eyes, do allow on-line access no matter where you are. Briefcase computers such as the Radio Shack Model 100 are softening the financial burden of terminal ownership.

But some of the most exciting changes occurring in the information industry involve new software techniques. More than simple extensions of old methods, these involve such provocative research areas as artificial intelligence (AI), natural language understanding, and knowledge engineering.

In particular, AI is making some meaningful contributions to that over advertised but under achieved phenomenon called “user friendliness.” Present on-line command protocols, while certainly understandable by any reasonably intelligent person, are nevertheless counter-intuitive and somewhat arbitrary. Improvements are being made in on-line “help” facilities (you can become reasonably proficient on DIALOG’S “Knowledge Index” system without a manual). But it would still be nice to enter information requests in relatively unstructured English and expect to be understood.

Intelligent software will also be useful at the terminal end. Customized through experience with a given user and the appropriate subset of all possible applications, it will accept high level commands, translate them, and interact with the remote database system as necessary to satisfy the user’s stated need. Encouraging developments along these lines are already appearing in the form of micro based “intelligent database access machines” such as SDC’s SearchMaster system.

Another trend to watch for is equally desirable: the replacement of the current profusion of command protocols on different systems with one universally applicable style. As with any standardization issue, there are approximately as many points of view as there are participants, and you can be sure that few online vendors will be in any hurry to make their systems look like someone else’s. But the demands of the market are rigorous.

Some other new developments are likely to affect the data communications part of the on-line picture. Not only will the networks begin to offer features of their own, but they will be come larger and more pervasive, reducing on-line costs for those unfortunate folks who do not presently live within a local telephone call of a network node. Data communication charges via Telenet average only $8/hour, but long distance rates on top of that can be painful.

Along with the enlargement of data communication networks will come a proliferation of “gateways.” At present, you access most databases by communicating directly with whatever on-line system hosts them. If you are signed on to the DIALOG system and want to change from Management Contents to Standard & Poor’s News Daily, you simply enter the command “B 133” to begin searching in database number 133. That is possible because both of them happen to reside in DIALOG.

But if the file you want to switch to is Accountants, which exists on ORBIT, then you have to log off of DIALOG, log on to ORBIT and take it from there. It’s not particularly difficult — but it would be nice to issue a simple command and let the network worry about what computer you happen to be talking to.

The logical extrapolation, and one that makes sense for a lot of reasons, is a move toward more and more distributed databases and away from massive systems hosting hundreds of different files. With large size comes a bewildering array of problems, not the least of which is the impact upon users when the system is down. If gateways between intelligent networks and small database systems can be easily managed, then every mom ‘n’ pop operation with information to sell could make their database available without having to meet the profitability criteria of a supermarket service.

All of this raises some interesting business issues. It has been estimated that on-line information retrieval will represent a 1990 market in excess of $10 billion, and major vendors such as ITT, IBM, Xerox, AT&T, and GTE are currently positioning themselves to participate. In addition, large traditional publishers such as McGraw-Hill and Prentice-Hall are frantically trying to align themselves with this field, since the potential for erosion of their print market is significant. Electronic publishing will begin to appear less and less strange — especially when the costs go down and computer facilities exist to deal with it efficiently.

All this growth implies trouble, of course, since well-established industries usually don’t give up without a fight. You can expect some extended legal battles, including lawsuits about restraint of trade, monopolistic practices, and labor disputes. Also, sticky copyright issues are beginning to surface, with no one in the industry sure of the best way to control ownership and reproduction of information once it is in machine-readable form.

With these events occurring, something that is still a rather obscure phenomenon for the average citizen is likely to become a major social issue. Who controls information? When on line services penetrate the home, how will they compete with newspapers and magazines? How will the existing network of authors, publishers, wholesalers, distributors, and readers deal with an unprecedented reduction in the volume of print media?

One thing is certain: On-line information retrieval is on the rise. Everything is increasing — the number of databases, the number of terminals, the number of users, the number of searches performed, the revenues, and even the number of mergers and acquisitions in the on-line field. Information is easily available from sources once obscure; the Bangladesh Journal of Scientific and Industrial Research can appear in your search results right next to Scientific American. The small business operator no longer struggles under the information disadvantage that has handicapped him in the past — if anything, he is more likely than his Fortune 500 competition to stay current with the applicable trends and technologies. National borders mean less and less, and it is more dangerous than ever before to assume that your current technological advantage in the market is likely to continue without ongoing effort. The number of secrets you can keep about your business is decreasing — but at the same time, you can keep your eye on the competition better than ever before. These trends will continue and should be woven into an intelligent business plan in the 1980s.

Next month, we consider the smorgasbord of services that are available in the on-line world — from ABI/INFORM to the Zoological Record.

Steve Roberts is a Columbus-based free-lance writer and consultant who has written extensively about on-line information retrieval, microcomputer design, and artificial intelligence. He is currently working on his fourth book (Garland Publishing Co.) Roberts’ CompuServe User ID is [redacted]

The Rise of the Intermediary

by Steven K. Roberts
Online Today — October, 1983 (same issue as above feature)

The recent emergence of on-line information retrieval services, such as Lockheed’s DIALOG and SDC’s ORBIT, has spawned a new breed of entrepreneur. Bridging the technical gap between commercial databases and end users, these information brokers are carving out a necessary (and profitable) niche in the information business.

Demand for “on-line intermediaries” is increasing rapidly. On-line information retrieval systems are complex, rapidly changing, and somewhat intimidating to the inexperienced user; nevertheless, they are becoming ever more essential to almost every type of business as the only efficient means of tracking the world’s prodigious output of specialized literature. This has created an interesting problem: A growing number of people are starting to realize that they need to use databases, yet they are reluctant to take on the learning curve that is required to do so.

Enter the information broker. Trained in the fine points of on-line searching and conversant with a wide range of subject areas, these quintessential generalists will find out — for a price — just about anything you want to know. Intermediaries have researched subjects as diverse as the tastes of cocktail waitresses, the viability of nitinol heat engines, the molecular structure of a paint sample, and the marketing of kerosene heaters in Scotland. This sort of flexibility would not be possible were it not for the databases themselves, which now contain, according to some estimates, references to roughly 95 percent of all the world’s literature published in the last 10-12 years.

The information brokerage business is a growing and healthy one. but it has its share of problems. One of the most frustrating ones is trying to explain to the public just what, exactly, information brokers can offer. Explanations of the capabilities offered by services such as DIALOG are often met with incredulity, and most business people have the conviction that they are already keeping up with their fields quite well.

Anji Brenner, an information broker in Minneapolis, doesn’t seem to have that problem. “People call me with their information needs,” she said. “I don’t have to try to convince them any more.” A former librarian, Brenner is quite familiar with the “head-in-the-sand” attitude many people have when it comes to information. But her acceptance by the business community has changed much of that; now she finds that a much greater problem is simply keeping up with the demand for her services.

The rise of the intermediary is evidence that the information industry is maturing. Originally catering only to academic, library, and research clientele, on-line services are now reaching out to the business community at large. The difference is dramatic, since only rarely does this new class of information-consumer have the expertise to access the services directly. Stockbrokers and real-estate brokers evolved long ago for similar reasons; now the information broker has appeared as an essential part of the emerging information society.