[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Tracking Internet Infrastructure
From: IN%"[email protected]" 14-APR-1996 07:00:37.86
From: Phil Agre <[email protected]>
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
This message was forwarded through the Red Rock Eater News Service (RRE).
Send any replies to the original author, listed in the From: field below.
You are welcome to send the message along to others but please do not use
the "redirect" command. For information on RRE, including instructions
for (un)subscribing, send an empty message to [email protected]
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Date: Fri, 12 Apr 1996 22:35:54 -0400 (EDT)
From: Gordon Cook <[email protected]>
Tracking Internet Infrastructure:
A Handbook on Business, Technology & Structural Issues Reshaping the
Landscape of the Commercial Internet
An Anthology of Recent Articles from The COOK Report
The Internet has undergone huge changes in the year since the NSFnet
backbone service was turned off. It has become a much larger, more
stratified, and more expensive entity within which to operate. Since
last September we have published a series of articles examining these
changes in depth. We have concluded that it might be useful to gather
them together into a single volume which is titled Tracking Internet
Infrastructure: an organized, indexed Handbook on Internet Infrastructure
Issues.
This Handbook covers the following critical range of issues:
* interviews with key industry players about viable Internet business
models,
* the hierarchical organization of ISPs through CIDR and routing
pressures,
* to the viability of the NAPS;
* to renumbering and ownership of IP addresses;
* to strains on backbone routers and backbone network redesign with
switched cores;
* to industry views on quality of service issues; to issues of
settlements and route charging;
* to issues of bandwidth availability in network design; and to ATM as
technology savior or dinosaur.
The rest of this message contains: (1) Our introduction to the Handbook
(2) The Handbook table of contents
(3) Description of the Handbook's audience
(4) Price and ordering information
1. Introduction
A Summary of the
Operational
Environment
Power Consolidates at the Top of a
Hierarchical Internet
Less than three years ago, at about one twentieth its current size, the
topology of the Internet was relatively flat. Service providers could
attach to each other via the NSFnet backbone or CIX router and, for the
most part, they could consider themselves plugged directly into the
Internet with no one up stream of them. This meant that no one was in a
position to dictate to them a multiplicity of rules, regulations and
costs as part of providing Internet service.
One paid the leased line costs for one's own backbone and, in the case of
research and education, was connected to the NSFnet backbone for free.
The R&E networks paid ANS a transit fee for their commercial customers.
If the service provider were commercial, it joined the Commercial
Internet Exchange and interconnected at the CIX router for the princely
sum of $45,000 a year ($10,000 membership fee, $5,000 port fee and about
$20,000 for T-1 line to the CIX. Or one was a downstream customer of
Sprint and relied on Sprint to deliver one's packets to the CIX router
without having to pay the CIX membership fee.
Over the period of about a year Sprint, by allowing ISPs to resell
connectivity, and by giving all ISPs downstream of it connectivity to the
rest of the Internet Universe, created a situation where customers of
Sprint received most of the benefits given those ISPs that connected
directly to the CIX router. Some ISPs were shocked in mid summer of 1994
when the CIX proposed that those packets of SprintUs resellers who didn't
pay the CIX fee would be blocked.
Since then, as the Internet has grown by more than an order of magnitude
in size, the importance of the NSFnet and CIX interconnects has either
disappeared or faded. A very hierarchical Internet has emerged. One can
have several levels of upstream service providers. Such service
provision ends at a traffic exchange point known either as a NAP or a
MAE. Address space in the form of IP numbers is no longer handed out the
interNIC to all ISPs.
To get address space direct from the interNIC, you have to do one of two
things: (1) show that you have no one up stream. The only way to do that
is to directly connect to a NAP or MAE and preferably to more than one.
This level of the hierarchy is reached by perhaps 40 of about 2,400 ISPs
nation wide. Or (2) an ISP can multi-home (take a connection from two
different backbone providers). Perhaps four to five hundred ISPs are
multi-homed. But even a multi-homed ISP is unlikely to get interNIC
address space, unless it can demonstrate a rapid rate of growth.
Connecting at a NAP or MAE, in an effort to put one's own operating
environment under one's direct control, is very expensive. Unlike the
$45,000 CIX fee the minimum annual cost is $100,000 and up. Once there,
providers have to pay additional sums of money to those still higher in
the hierarchy to see that their packets are delivered. These sums of
money are known as transit fees. Multi-homing is obviously much less
expensive.
But even at traffic exchange points (NAPs or MAEs) there are additional
hierarchies. Some find others who will peer with them. That is to say,
they will engage in cost-free transit for a certain percentage of those
attached. Those at the peak of the hierarchy are the six service
providers who are believed to engage in cost free peering and transit
with each other - MCI, SPRINT, UUNET, ANS, PSI and AGIS. These six
operate the default free core of the American Internet. Exchange of
routing information among them is supposed to be complete, so that none
need say: if your routers don't have address XYZ, send packets by default
to the next large central player, in the hope that his routers will
know. Virtually everyone else is, in one sense or another, a customer of
these top six. BBN is a special case. MCI carries all of BBN's routes.
Consequently in this sense BBN which is as large as UUNET and certainly
much larger than AGIS, is a customer of MCI. We expect this to change by
year's end when AT&T fully deploys the network for its partnership with
BBN. [Editor's Note: when we fact checked this assertion with sources
at BBN we were told that BBN is putting its own national T-3 backbone in
place, that significant parts are now operational, that it is at MAE
East, MAE West and the Sprint NAP, that it peers with PSI, UUNET, ANS and
others, and that its transit relationship with MCI is to reach MCI
customers and any Internet sites not directly reachable via its peers.]
Those who are able to buy transit at a NAP gain by this action gain the
ability to ensure it to their downstream (ISP) customers. In other words
transit rights are generally resellable. In this sense, those who
purchase transit at the NAPs or MAEs are as fully connected to the core
Internet as the big six - with one critical exception. They are renting
their core connectivity for a hefty monthly price and without the
protection of a long term lease. Readers however should avoid
generalizations. Transit agreements are very private, never talked about
openly and vary widely on a case-by-case basis.
IP Numbers and Other Indicators of Hierarchy
Another way of describing the top of the hierarchy is to point out that
all six get their IP numbers directly from the interNIC and hand out
numbers lower on down in the IP hierarchy to those connected to them.
But this hierarchy is not uniformly rigid. The rule of thumb of your
upstream provider as a source of IP numbers has some notable exceptions.
For example: of the remaining approximately 35 providers which are
directly connected to one or more of the major NAPs or MAEs, most get
some of their IP numbers direct from the interNIC while others are
derived from being attached at some point or in some way to one or more
of the central seven. Finally, as we indicated above, the several
hundred ISPs who are multi-homed also get their numbers from the
interNIC, as may ISPs who can demonstrate extremely rapid downstream growth.
An ISP without IP addresses is worthless, since the only thing making it
possible to connect an individual to the Internet is the IP number that
tells others how to find him. The hierarchy of the Internet is now such
that about 80% of service providers must get their addresses directly
from their backbone vendors who are often also their competitors. These
address blocks are referred to as CIDR blocks. We have recently written
extensively about them.
Until the fall of 1995 large ISPs were able to connect to one or two NAPs
and sometimes negotiate cost free peering. As an option they could then
buy transit to the other five from one of the big six. During the past
90 days this has changed. Reports reaching us indicate that cost free
peering is available only to those connecting at T-3 speed to three or
more NAPS a task that will cost an ISP well over $300,000 a year. But
even those who do this will find that there is nothing that will force
everyone at each NAP to which they connect to peer with them. If one or
more of the majors refuses to peer with a newcomer, that newcomer will
have to buy transit to that major from one of those with which it peers
or be in the awkward position of being unable to reach a significant part
of the Internet.
For most of 1995 an ISP connecting to a NAP could by transit for $5 to 15
thousand dollars a month from one of the seven. But reports reaching us
now indicate that the six have effectively eliminated the purchase of
transit from them as an option for domestic ISPs. These developments
effectively shut off connection to a major NAP as a means for an ISP to
operate from the top of the Internet hierarchy. The only ISPs that may
even attempt to do so now are those with upwards of $500,000 a year to
spend on the adventure. In the meantime the NAPs themselves are
developing a hierarchy. MAEs either are opening or have opened in Los
Angeles, Texas, Chicago and New York. Without the presence of at least
the big six at these NAPs, all they are good for is exchanging local
traffic among local ISPs and keeping loads on long haul backbones down.
The Issue of IP
Portability
When a customer changes phone service from AT&T to MCI, or visa versa,
that customer does not have to change phone numbers. Although the
portability of IP address assignments from CIDR blocks has been
discouraged it has never been prohibited. But last month a new internet
draft (draft-ietf-cidrd-addr-ownership-07.txt) by Yakov Rekhter and Tony
Li of Cisco was published. This draft created much debate in mid
February when it was put forth by the CIDRD Working Group for elevation
to best current practice. Such elevation would put IETF approval behind
the practice of a service provider insisting on the return of address
space when a customer left. If such a customer were to go to a different
service provider for a new connection, every one attached to that
customer's network would be forced to renumber their own networks. For a
network of any size renumbering would be expensive and, if that network
were involved in anything approaching a mission critical application,
would become unthinkable.
This would very likely mean that any customers buying leased lines to
connect a network larger than a few dozen hosts in size will find
themselves well-advised to purchase only from an ISP directly attached to
one or more of the MAEs/NAPs -- and, therefore, in direct control of its
address space. Furthermore, the safest and most conservative action with
be to connect to the six providers who are part of the default free
backbone. Certainly we suspect that the auto industry would tell its
suppliers not to connect outside the direct NAP connected top tier.
While there are technical reasons for this policy (fear of the collapse
of internet routing if it is not carried out), it is ironic that such
policy would greatly accelerate the Internet's stratification into a
business service and a consumer service, for those who are there to
explore, just for the fun of it. There is also an anti-competitive
aspect to implementing such policy, in that large organization customers,
which embed non-transferable IP addresses into their network hosts are
really locking themselves to a single provider. Should a provider's
service becomes "less than optimal," we are sure that providers are aware
that the cost associated with renumbering in order to change vendors,
limits their customer's options. While network address translation
devices (NATs) do exist and will give some customers an alternative, they
are by no means regarded as a perfect answer to these difficult problems.
It is beginning to appear that, the more the Internet increases in size,
the faster that power flows upwards into the hands of a few who, since
they are both operators and rule makers for the commercial Internet,
would find themselves singled out for accusations of blatant conflict of
interest in most other situations. Under these conditions where the fox
is essentially in charge of the hen house. Given the nature of a large
portion of the customer base (ie large industry and educational
networks), we wonder how long customers will suffer these burdens without
demanding regulatory relief.
2. Contents:
Tracking Internet Infrastructure:
Editor's Introduction
A Summary of the Operational Environment p. 1
Part One: Internet Business Models
Some Large Providers Seek Forum to Push for Internet
Service Model Change (Sept. 95) p. 8
Interview with Vint Cerf: Discussion Needed of Benefits
Derived from Backbone Resources (Sept.95) p. 12
PSI Satisfied with Cooperative Best Effort Internet Business
Model Interview with Bill Schrader(Oct. 95) p. 17
Thoughts on Internet Business Models by Sean Doran (Oct. 95)
p. 20
Zero Sum Internet Business Models Vie with Internet
Cooperative Culture (Oct. 95) p. 24
Routing Arbiter & Charging for Routing Announcements: Potential
Operational and Financial Impacts Assessed (Jan. 96) p. 33
Part Two: Internet Architecture Change & Network Stratification
Evolution in CIDR Rules In 1995 Makes Most IP Numbers non
Transportable (Sept. 95) p. 43
Constraints of Growth: Provider Based CIDR Likely to Impede
Smaller Players Interview with Dave Crocker and Noel
Chiappa (Nov. 95) p. 46
Pace of Internet Stratification Increases -- IETF Internet Draft
Suggests That Customer Network Renumbering Be Accepted As
"Best Current Practice" (Mar. 96) p. 54
Part Three: Backbone Routing Versus Switching
Continued Exponential Growth Stresses Internet Backbone Routing
Infrastructure (Dec. 95) p. 62
Part Four: Institutions - Sprint; IETF, ISOC, and the NAPs
SprintLink Experiencing Employee Attrition - Executives Slow to
Provide Staffing Resources Needed for Continued Major Growth
(Sept. 95) p. 81
National Science Foundation Domain Name Charges Financial
Implications for Network Solutions NSF Rationale Behind Actions
Interview with Don Mitchell (Oct. 95) p. 85
Internet Society: Role of Charter Members a Contentious Issue
(Nov. 95) p. 87
Transition Pains at the Internet Society (Feb. 96) p. 88
Interview with Paul Mockapetris Who Considers Future of IETF &
Finds Software Patents a Growing Obstacle (Jan. 96) p. 89
Interview with Tony Rutkowski Who Finds Internet International
Coordinating Group Desirable (Mar. 96) p. 93
No Room at Sprint's Pennsauken NAP (Jan. 96) p. 96
Part Five: Quality of Service
Automotive Industry Will Seek Internet Service Provider
Certification (Feb. 96) p. 102
Steve Wolff Sees Convergence Between Internet and Telephony
(Feb. 96) p. 107
Part Six: ATM and the Technology of Bandwidth on Demand
Can Bandwidth Supply Keep Pace With Demand? ATM to the Rescue?
An Introduction to a Series of Articles on Role of ATM in the Internet
(Mar. 96) p. 111
ATM: Grand Unifying Technology or Brain-Damaged Transport
Product? (April 96) p. 114
Bandwidth & Resource Reservation as Factors in Ones Network
Provisioning Philosophy -- Can Bandwidth Ever Be Too Cheap to
Meter? (April 96) p. 117
Interview with Bellcore's Dave Sincoskie Who Discusses the Internet
Future of ATM & Outlines BellcoreUs Interest in Building Network of
Interconnected ATM NAPs (March 96) p. 120
InternetMCI Bets its Future on ATM Data Services Marketing & Data
Services Engineering Vice Presidents Explain MCI Strategy.
Interview with Stephen von Rump and Steve Tabaska
(April 96) p. 124
Interview with BBN's John Curran: Has the Internet Derailed ATM?
(May 96) p. 132
Index p. 140
3. The Audience for the Handbook
Within the national Internet service provider community, Tracking
Internet Infrastructure is intended to educate strategists with the
complexities facing their engineering and operations staff. Among
smaller ISPs it should serve as a tool to bring owner-operators, who are
busy 18 hours a day ordering lines, installing them and servicing their
customers, up to speed on the changes going on in the environment in
which they must operate. LECs and other phone companies will find it
useful. Finally familiarity with the issues discussed within the
Handbook will provide corporate MIS people with a valuable knowledge base
from which to negotiate with their present or future internet service
providers.
However, since these infrastructure issues are also critical to the
continued growth and success of the industry, this Handbook is expected
to be a tool for use by those in the banking and investment community.
If those in the financial community understand the changing technical and
power relationships in the industry, they will be able to improve the
quality of their investment decision making. It should also be useful to
corporate strategic planners who will be advising their companies'
decision making in vertical industry applications.
4. This handbook may be purchased in several ways:
A. Single Copy GBC Bound, double sided xeroxed. $275.00
B. Site license: Set of single sided original 600 dpi laser written
pages suitable for purchasing organization to reproduce as many copies as
it wishes for its employees only. $750
C. A current subscriber without a site license may upgrade to a site
license and pay an additional price of $275 to receive the report with
full site license privileges.
D. A current subscriber with a $650 site license or higher may purchase
the report with full site license privileges for $275.
To order contact Gordon Cook by phone (609) 882-2572 or email:
[email protected].
*********************************************************************
Gordon Cook, Editor & Publisher Subscriptions: Individ-ascii $85
The COOK Report on Internet Individ. hard copy $150
431 Greenway Ave, Ewing, NJ 08618 USA Small Corp & Gov't $200
(609) 882-2572 Corporate $350
Internet: [email protected] Corporate Site Lic. $650
http://pobox.com/cook/ for new COOK Report Glossary of Internet terms
*********************************************************************