Abstract
The
Microsoft Exchange development team performed major surgery when it created
Exchange 2000. A huge degree of change has taken place at the heart of
messaging interconnectivity, so it’s important to understand the factors that
have driven the change, how the transport core works, and how the new routing
mechanism affects designs. That’s what this chapter is all about — exploring
the heart surgery that Microsoft has performed on message routing.
THREE TASKS OF MESSAGE ROUTING
When
you think about it, message routing is concerned with three tasks:
Server
to server message delivery
Location
to location message delivery, where a location is a collection of servers
External
message delivery
Exchange
has always been good at routing. All Exchange 5.5 servers know about every
other server in the organization and about every connector or available route
that messages can take. This information is held in the DS and replicated along
with all of the other configuration data. The MTA (Message Transfer Agent) took
this information and used it to route messages quickly and effectively, and
most of the time things worked out very well. The wide range of connectors has
helped to link Exchange to all of the other major messaging systems, including
its main competitor, Lotus Notes, and many of the connectors handled directory
synchronization. Exchange 4.0 through 5.5 certainly knew how to route messages
in all of the situations described above.
The goals for message routing in Exchange
2000 are:
Move
to an SMTP-centric model
Exploit
AD, DNS, and LDAP
Process
messages at least twice as fast as the Exchange 5.5 Internet Mail Service, and
be at least on a par with the SMTP service in MCIS (Microsoft Commercial
Internet Server).
The old adage of “when something works,
don’t fix it” must have crossed the minds of the Exchange development team when
the time came to decide how message routing should work in Exchange 2000. The
temptation to leave well enough alone and focus on other areas of functionality
must have been present, but it was ignored. The developers decided to go ahead
and make changes. Instead of tweaking and improving the existing code, they
decided on a radical makeover and changed everything. The net effect is that
the world of routing is far different in Exchange 2000 to anything that’s gone
before. The role of the MTA has been taken over by a new SMTP-based transport
core. The GWART (Gateway Routing Table) used to determine how messages flow
between servers has been discarded in favor of dynamic routing based on
information reported back to servers about the current state of network links.
The set of old connectors used with Exchange 5.5 are still there, but some have
been replaced with new versions and others have been left connected to the MTA.
A huge degree of change has taken place at the heart of messaging
interconnectivity, so it’s important to understand the factors that have driven
the change, how the transport core works, and how the new routing mechanism
affects designs. That’s what this chapter is all about — exploring the heart
surgery that Microsoft has performed on message routing.
THE BATTLE BETWEEN SMTP AND X.400
SMTP
(Simple Mail Transport Protocol) is the primary mechanism used in Exchange 2000
for inter-server messaging. The protocol was originally defined in RFC 821 in
1982, and since then it has been widely used as the de facto standard for
e-mail exchange within the Internet and between different corporate bodies. At
the beginning of the 1990s, SMTP battled with X.400 to become the generally
accepted protocol for e-mail exchange. X.400 had many champions, mostly in
Europe and industry bodies such as the Electronic Mail Association (EMA), but
the overwhelming force applied by the Internet-led juggernaut catapulted SMTP
into the lead position.
Exchange 4.0, 5.0, and 5.5 are built
around an X.400-based MTA. The presence of X.400 at the heart of these versions
of Exchange reflects the view that corporate messaging systems could really
only be built around a solid, well-defined and comprehensive protocol. When
Exchange 4.0 was in development, the case could not be argued that SMTP had
reached the same level of development as X.400, which at that stage had evolved
through the 1984 and 1992 set of recommendations and had been deployed in many
corporations around the world. In comparison, to the maturity of X.400, SMTP
had not yet evolved to accommodate non-text bodyparts (in essence, binary
attachments). In real terms, this meant that X.400 was the only way that people
were able to send each other attachments such as Word documents or Excel
spreadsheets without resorting to complex encoding schemes that required manual
intervention. Forcing people to encode a document before it was attached to a
message is OK within the technical community, but few office-type users have
the patience to go through such a laborious procedure.
Microsoft purchased the code that the
Exchange MTA is based on from a small UK-based company called Data
Communications Limited (DCL). The original code has been extensively upgraded
and modified to meet the needs of Exchange since it was taken in-house. For
example, the DCL code needed to be integrated with the Information Store so
that it could fetch and deliver messages and it needed to be able to access the
DS in order to validate e-mail addresses. A single executable (EMSMTA.EXE) and
Windows NT service (the Microsoft Exchange Message Transfer Agent) took care of
message routing, the RPC-based site connector, and the X.400 connector. These
three components linked systems together within and between sites and provided
the real glue for pre-Exchange 2000 organizations. It’s also the reason why you
cannot delete the X.400 addresses that are allocated by default to each
Exchange mailbox. Without the X.400 address, messages cannot be routed to an
Exchange 5.5 mailbox.
While some system administrators may have
viewed it as a part of Exchange that was complex and unapproachable, there is
no doubt that the MTA delivered real speed in message processing. Even the
earliest versions of Exchange running on system configurations that are now
deemed puny were able to accept messages from the Store, assess their
destination addresses, and route them on their way very quickly. In fact, the
MTA is so fast that it makes the Outlook client “Recall Message” feature
worthless as messages are processed very quickly. Recall message can only work
if the Store is able to retrieve the message before a recipient has read their copy.
This is possible if the message is delivered to recipients on the same server
as the originator, but once the message comes under the control of the MTA, it
is transferred off the server and to another MTA for local delivery, and often
cannot be recalled.
The MTA is only slowed down when it comes
to expand large distribution lists (more than 500 recipients), but Exchange 5.5
improved matters dramatically and can now expand even the largest list in a few
seconds. Of course, the added power of the computers and use of symmetric CPUs
made their contribution, but the underlying speed was delivered by the MTA.
Over three releases, multiple service packs, and many hot fixes the MTA grew in
reliability and robustness. However, the rapid evolution of the Internet
protocols and the general acceptance of SMTP as the way to accomplish global
messaging marked the beginning of the end for the “X” protocols. The MTA has
now been overtaken in importance by the combination of the SMTP service and the
new routing and transport engine, which now perform the vast majority of
routing work for Exchange 2000.
The importance and use of the MTA is at
its height during the initial deployment of Exchange 2000 servers and begins to
subside afterwards as routing activities generate towards SMTP. Each new
Exchange 2000 server that’s added to the network reduces the load on the MTA
and increases the amount of SMTP routing activity until the stage is reached
when the entire Exchange network is composed of Exchange 2000 servers. At this
point all inter-server traffic will be carried by SMTP and the only X.400
messages seen on the network will be those routed back to the legacy
connectors.
THE EVOLUTION OF SMTP
In its
early days, SMTP was a very rudimentary means of transferring messages from one
system to another. RFC-821 (and the closely associated RFC-822) only allows for
the transfer of 7-bit ASCII text messages. If you monitor the traffic going to
port 25 on an Exchange server, you’ll see that the interaction between SMTP
servers is very straightforward. The protocol defines how to contact a server,
how to transmit data to set up the connection and establish the link necessary
to transfer messages, how to transfer the message content, and then how to sign
off.
Simplicity has its virtue in that nearly
every computer system could understand 7-bit text, and SMTP lived up to its
name in its ability to send simple text messages between people. Success in the
commercial e-mail world depends on meeting user requirements and the success of
Windows as the desktop platform of choice introduced many new file types to the
equation. Clearly, something had to be done to enable SMTP to transport the
huge array of PC files that people wanted to send to each other. Equally
important, while 7 bits are acceptable in a US-English environment, there are
many characters in daily use outside the US, which aren’t included in 7-bit
ASCII.
A new protocol was developed to overcome
the serious limitations in SMTP. MIME, or Multipurpose Internet Mail Extensions,
supports binary attachments and guarantees message fidelity across SMTP
gateways. Although specified in RFC 1341 and 1342, MIME does not replace
RFC-821/822. Instead, MIME is fully backward compatible and is designed to
extend the basic messaging functionality defined by SMTP. UUENCODE can also be
used to transmit attachments.
MIME is designed to define a mechanism for
declaring the different parts that might be included in a message such as the
header, body text, and attachments. At the same time, the methods required to
transform content so that it can be transported as if it was simple text are
also defined. MIME meets its goals by introducing a number of new message
header fields, including fields to specify the content type (text, multi-part,
application, message, video, and so on) and the content encoding scheme. Most
MIME-encoded messages seen in an Exchange environment use the Base64 encoding
scheme, which allows binary data to be sent across transports originally
designed for 7-bit transmission. The basic MIME protocol has since been
extended to accommodate the requirements of secure e-mail through a set of
secure extensions (S/MIME). S/MIME enables encrypted messages and the keys
necessary to decrypt the messages to be reliably transmitted between SMTP
gateways. Exchange has supported S/MIME since Exchange 5.5 SP1.
The collective evolution of SMTP and MIME
provided the Exchange designers with a viable option to X.400 when the time
came to select a routing engine for Exchange 2000. The X.400 MTA had been
proven in deployments around the world, but the protocol lends itself to
complexity and transmits this feeling to system administrators. Indeed, the
mere mention of “X.400” is enough to generate an almost visceral reaction when
any reference is made to the protocol. Many system administrators who run
Exchange 4.* or 5.* servers don’t realize quite what a critical role X.400
plays at the heart of their messaging environment, but if the truth were told,
few really want to know. Everything associated with an “X.” prefix (X.500 falls
into the same category) is deemed complex, expensive, and only suitable for
people who like to twiddle bits and bytes.
MIME is a great way to code message
contents for transmission, but SMTP needed some fundamental upgrades before it
could deliver all the functionality that’s in X.400. The key development here
is the definition of ESMTP (extended SMTP) in RFC 1869. ESMTP allows the basic
protocol to be extended with new commands, either those defined in another RFC
and so generally supported within the SMTP community, as well as commands
intended for use by specific applications. Exchange 2000 uses ESMTP to support
a wide array of the industry-standard extensions as well an extension of its
own to support the propagation of link state routing information between
servers.
Ever since Exchange 4.0 was first
released, I have listened to many arguments between the supporters of both
camps who liked to discuss the virtues of connecting Exchange sites using the
site, X.400, and SMTP (Internet Mail Service) connectors. Everyone agrees that
the site connector should be used whenever available network bandwidth permits,
but the discussion over the choice X.400 and SMTP connectors really gets people
going.
For Exchange 5.5, the facts of the matter
are that the X.400 connector is better integrated into the MTA (naturally),
more robust, and uses less overhead and bandwidth to send messages between
sites. It’s also reasonably easy to set up an X.400 connector after you’ve been
through the exercise a couple of times, but these data points never seemed to
register with the SMTP community. The normal responses were comments like “look
how many property pages there are to complete to set up the X.400 connector” or
“it doesn’t matter if the SMTP connector uses more CPU cycles — that’s why I
have multiple CPUs.” In reality, it’s easy for a computer program to hide
complexity from the end user and Exchange does an excellent job of masking all
the things you can adjust with X.400 from the average system administrator.
Another criticism of SMTP is the amount of
overhead added to a message by MIME encoding. While it’s true that MIME
increases the size of a message (in some cases by up to 30%) and that X.400
adds relatively little overhead, the contest between the two protocols is
largely offset by the simplicity of SMTP communications versus the call
set-ups, handshaking, and acknowledgements that intersperse X.400
communications. In other words, the overhead required in establishing and maintaining
an X.400 connection between two servers is heavier than that required by an
SMTP link. The improvements made to SMTP as implemented in Exchange 2000 (see
below) further improve matters, which should mean that SMTP connectivity is at
least as good across a low bandwidth link as X.400. Only time will tell.
Internet protocols are much more amenable
to extension than their “X” counterparts. We’ve already seen that SMTP is
defined in one RFC and then extended through different commands (like chunking)
as defined in subsequent RFCs. The new commands are added to the base SMTP
protocol through ESMTP, or “extended SMTP,” and the same approach can be taken
to extend SMTP further to meet the needs of specific applications. Exchange 5.0
SP1 and 5.5 added support for a number of ESMTP extensions, in particular ETRN,
to allow sophisticated dequeuing of messages, usually when a server has to make
scheduled connections to an ISP to pick up messages held there for a specific
domain1. Exchange 2000 takes even more advantage of extensions to carry
information about link states and build a dynamic picture of the routing
network. The notion of extensibility is carried on through transport and
protocol events that can be used to process messages as they pass through Exchange.
Given the choice and an equal playing
field it’s debatable whether the switch from X.400 to SMTP will bring Exchange
any real advantages, but this case can only be argued from a pure technology
perspective. The real value is seen in how the protocols are implemented and
whether servers are able to easy and seamlessly connect to other servers to
pass messages. Given that the Internet protocol2 juggernaut continues to roll,
it seems that the Exchange developers have made the right choice to move towards
an SMTP-centric model. Experience in production will tell whether the
implementation proves the point.
Critical Challenges of ESI & Email Retention Are you storing too much electronic information? Get expert legal advice and better understanding of what you are required to do as an IT professional.
Rev Up Your IT Know-How with Our Recharged Magazine! The improved Windows IT Pro provides trusted IT content with an enhanced new look and functionality! Get comprehensive coverage of industry topics, expert advice, and real-world solutions—PLUS access to over 10,000 articles online. Order today!
Get It All with Windows IT Pro VIP Stock your IT toolbox with every solution ever printed in Windows IT Pro and SQL Server Magazine plus bonus Web-exclusive content on hot topics. Subscribe to receive the VIP CD and a subscription to your choice of Windows IT Pro or SQL Server Magazine!
Order Your Fundamentals CD Today! Gain an introduction to Exchange, learn server security requirements, and understand how unified communications can play a role in your messaging strategies with this free Exchange CD.
Windows IT Pro
Register
