Sending Messages -- Routing and Transport

Windows IT Pro

- Advertise

Sending Messages -- Routing and Transport
View the book table of contents

Author:	Tony Redmond
Published:	October 2000
Copyright:	2001
Publisher:	Digital Press

    Terminate the message with a carriage return, full stop, and another carriage return. If the message is accepted, you’ve just proved that your system is open to relaying, but at least you now have a way to prove that Bill Gates approves of your plans to deploy Exchange. The message isn’t very pretty, and probably needs to be tidied up by adding a message subject and formatting the text properly before anyone would be convinced that it came from Bill Gates.⁶ A server that responds with a status code of 550 means that it has been secured against relaying, while a 551 code means that the recipient is not local and the server won’t allow you to relay to them.

      Exchange 2000 servers do not use the settings that control mail relays. This is because Exchange 2000 servers authenticate together when they begin a connection with the ESMTP “X-EXPS” Kerberos authentication verb. If an SMTP server advertises this verb, Exchange will attempt to issue an X-EXPS command. Authentication will succeed if both servers are members of the domain “ExServers” group. All servers in an organization are added to this group by default. This means that you only need to allow anonymous connections on servers that are specifically designated to act as the inbound target for Internet or intranet SMTP connections from non-Exchange 2000 servers. Allow anonymous is enabled by default on all Exchange servers.

      There’s no doubt that messaging administrators are aware of the danger that illegal relaying and spamming pose, but even with all the publicity and comment, the most recent report from the Internet Mail Consortium⁷ found that just over 17% of servers tested still allowed mail relays in July 1999. Every one of these servers is an open target to spammers, who are all too quick to take advantage of the open door policy afforded to them. Considerable effort has been expended to protect Exchange 5.5 servers and a number of changes and enhancements have been provided in service packs and hot fixes in a continual effort to frustrate the would-be abusers. Exchange 2000 poses a new challenge, and although Exchange will prevent message relays from servers outside its organization, the possibility exists that an administrator could make a mistake and open up a server to the great unwashed. For this reason, it’s a good idea to check servers on a regular basis and close off any loopholes that might have opened up.

MOVING AWAY FROM RPCS

Why go through all the bother that results from changing the default transport from RPCs? The obvious reason is that the combination of MAPI and RPC results in a Microsoft proprietary mechanism for inter-server communication. SMTP is a standard, so there’s an immediate prospect of better interoperability between Exchange and other messaging systems.

      Every designer who ever had to deal with Exchange in a low-bandwidth environment rapidly became painfully aware that RPCs require high-quality networks. The definition of an Exchange 5.5 site is a collection of servers connected together with LAN-quality bandwidth. All the servers connect together to form an automatic full-mesh network. Generally, RPCs work well within such an environment but demand substantial network resources as the number of servers in a site grows. A lot of the overhead incurred by RPCs is due to the acknowledgement model that is used. Sending multiple acknowledgements over a high-speed connection is not an issue because the data is “lost” in all the other traffic, but acknowledgements quickly become a burden on low-capacity links. Even on extended links that apparently have high bandwidth, such as transatlantic or transpacific connections, RPCs often fail because they are sensitive to latency. The history of Internet protocols is influenced by the original notion of loose connectivity between systems, which means that protocols must be capable of working over very low speed connections. SMTP is much better at transporting messages over low-bandwidth or low-quality connections, so some of the issues that Exchange designers have grappled with over the years, such as site design and choice of connectors to link sites together are now easier.

      The RPC-based site connector is the easiest and quickest connectivity option to link Exchange 5.5 sites together, as long as a high-capacity (> 64 K/bps) network link exists. Many people began a design by connecting sites with site connectors only to revert to an X.400 or SMTP connector when the message load increased or they discovered that the network simply couldn’t transport the RPCs in a reliable manner. Routing everything through SMTP, which is what happens inside routing groups and across the routing group connector, eliminates all the problems associated with RPCs and moves Exchange to a situation where it should be easier to deploy in low-bandwidth environments. Even if the cost of installing high-capacity links is dropping all the time, people seldom buy the argument that it makes sense to increase capacity all round just to ensure that e-mail gets through. The argument is biased towards the US where competition and deregulation have combined to drive network costs down to a level not seen elsewhere. Costs are higher in many parts of Europe and Asia where low-bandwidth connections (less than 32 K/bps available bandwidth) remain the norm. As the effects of privatization and global competition are felt, network costs will reduce around the world, but for now the prospect of being able to achieve reliable messaging transport across low-bandwidth links is very attractive.

      MAPI RPCs are encrypted between servers within a site, and the RPC-based Exchange 5.5 Site Connector protects data with encryption. This is a major, albeit often unrecognized advantage of the Site Connector. However, X.400 connections are not encrypted and the IMS (Internet Mail Service) could not be protected either until SSL support was introduced in Exchange 5.5. Exchange 2000 supports encryption and authentication, and authentication can be accomplished using any of the standard Windows 2000 mechanisms – NTLM (Windows NT challenge/response), Kerberos, or plain-text. Naturally, if you plan to relay traffic across a public network, the last option is the worst.

The role of the X.400 MTA
The MTA has not gone away, but its role is much different in Exchange 2000, as the Transport Core has taken much of its responsibility away. Table 3 lists the major differences.

      By default, an X.400 address is allocated to every user with an Exchange mailbox. The X.400 address is critical within a mixed mode organization because older Exchange servers use the X.400 address to route messages to mailboxes. You should not delete the X.400 addresses from mailboxes unless you are positive that they will never be required again. Given that the addresses take up little space in the AD, it’s best to leave them alone.

      After four years of use, the MTA has attracted a range of connectors, ranging from a variety of fax connectors to those that support other messaging systems. As part of the Exchange 2000 development program, Microsoft is upgrading its own set of connectors to work with Exchange 2000. There are two exceptions to this rule as neither the IBM PROFS nor IBM SNADS connectors have been upgraded. These connectors were purchased by Microsoft as part of the Linkage acquisition in 1997 and have never been a core part of the product. PROFS and SNADS are antique messaging systems at this stage, and the connectors can only be viewed as a migration device to help customers bridge a short period when a backwards link is required to help users move off either system to Exchange. Not many PROFS or SNADS systems are still operated, so there is limited interest in seeing either connector feature with Exchange 2000, but if you need this functionality, you will have to keep an Exchange 5.5 server operational until the requirement ceases. The implication here is that the Exchange organization will remain in native mode until the PROFS or SNADS connectors can be dispensed with and the Exchange 5.5 server removed.

THE TRANSPORT CORE

The transport core is the heart of message routing in Exchange 2000. It is broken down into four major components:

Advanced Queuing: This component manages messages as they enter and exit the different queues within the Transport Core. The queuing engine supports domain-level queues, where messages are grouped according to their final destination (such as any message sent to recipients at compaq.com), and link queues, where messages are grouped according to the next hop that they will be routed across to get to their final destination. For example, messages sent to compaq.com might have to travel across a link to an ISP. We already know that Exchange 2000 maintains a destination queue for all messages to compaq.com, and now we have a specific link queue for messages waiting to travel to the ISP en route to compaq.com and any other address reached through the ISP. A message can pass from server to server and move from link queue to link queue, but it will stay on the same domain-level queue. The queuing engine is supported by a new QueueAdmin API, which is used by the Exchange System Manager snap-in to view queue status. The API allows queues to be stopped and started as well as enumeration of queue contents. The Advanced Queuing component is also responsible for generating delivery service notifications.
Categorizer: The categorizer analyzes message headers and properties to decide how they should be processed. Senders and recipients are resolved against the AD to ensure that any restrictions are respected and that the addresses on messages can be routed. A limited version of the categorizer is provided in Windows 2000 to enable basic functionality such as the expansion of mail-enabled groups. The version provided with Exchange 2000 adds support for features such as checking for mailbox quotas, whether a mailbox can receive mail from a specific sender, what mailbox store a mailbox is located in, whether a user can send messages across a specific connector, and so on.
Exchange Store Driver: This is the interface between the Advanced Queuing component and the NTFS file system. Windows 2000 ships with a basic Store driver that allows it to save incoming SMTP messages as formatted files, which are later processed to accomplish tasks such as AD replication. The Exchange Store driver is an upgraded version that includes ExIFS support, which enables Advanced Queuing to also have direct access to messages in the Store.
Routing Engine: This component replaces the RID master and GWART used to determine routing paths within an Exchange 5.5 organization. It is a much more sophisticated routing mechanism that is capable of assessing incoming updates about the state of the network and adjusting the path messages will take. Messages can be routed according to their size, the sender of a message, and its priority. This information is taken into account with the cost associated with the various links defined between servers in the organization and the current state of the links to decide how a message is eventually routed.

The Routing algorithm used by Exchange 2000 is a modified version of Dijkstra’s algorithm. The method used to route messages is explained in the section on Link State Routing.

Figure 7 illustrates how the different parts of the Transport Core work together. While the diagram looks complicated at first appearance, it’s really not too bad. To explore how the parts of the routing engine work together, let’s follow the path of a message as it is processed within the Transport Core:

Messages can enter the Transport Core from the SMTP service, the MTA, or the Store. Messages coming from the SMTP service originate from other Exchange 2000 servers and arrive as NTFS files that are then processed via ExIFS. MAPI mailboxes submit messages through the Store, and messages arriving from Exchange 5.5 servers come in through the MTA. IMAP and POP3 clients submit messages through the SMTP protocol stack in IIS, which in turn uses ExIFS to create the message bodies and insert them into the streaming file. WebDAV clients can access ExIFS directly without going through the SMTP protocol stack. Message properties are promoted into the appropriate mailbox store. In all cases, the messages are placed into the Inbound Queue.
The Queuing engine manages processing of the inbound messages and passes them to the Message Categorization code. Message Categorization performs checks such as mailbox quota limits for recipients and verifies that the message can be routed onwards. Distribution lists are expanded at this time.
The message is placed into the Categorized Message Queue. The Transport Core now checks to see whether any routing event should be triggered for the message. For example, if an event implements a check to ensure that outgoing messages should not contain profanities, it would be possible to scan the text of the message and its attachment. The message could then be routed onwards or returned to the originator if some problem words were found. Routing events are intended to facilitate the fastest possible delivery of messages to the final destination, so checking for profanities isn’t a particularly good example of how a routing event is used. Nevertheless, it does demonstrate that events open up all sorts of new possibilities to developers.
A lookup is performed against the Domain Mapping and Configuration Table. This table tells the Transport Core to route the message for local delivery or to one of the SMTP virtual servers that are active on the system.
Messages intended for local recipients are placed on a local delivery queue. “Local” means anything that isn’t another SMTP server (including remote Exchange 2000 servers). Remember that the MTA is still tightly integrated with the Store so any message that must be dealt with by the MTA has to be routed through the Store. At this point, another event could be triggered before the message is actually delivered to a mailbox. For example, a copy of the message could be sent to a special recipient if it came from a user in a particular department. The message is then passed to the Exchange Store driver, which either delivers the message to a local mailbox or passes it to the MTA for onward transmission to an Exchange 5.* site or legacy connector.
Messages for other SMTP servers are allocated to the destination queues for the SMTP virtual servers allocated to handle messages for the different domains. The Connection Manager monitors the links to other SMTP servers for the destination message queues and adjusts routing as required. For instance, if the link state table was updated to reflect the fact that a hub server was currently not available, the Connection Manager might reroute messages that pass through the affected server via another hub.
All outgoing messages are dispatched to their final destination using the standard Windows 2000 SMTP service.

      Two different queues are used in the final stages of routing SMTP messages. Domain queues are maintained on a per domain basis. In other words, all the messages for a specific SMTP domain (such as *.com) are held together on one queue. Messages are taken off domain queues and placed on link queues, which are maintained on a per hop basis. This means that the Connection Manager moves a message to the most suitable link queue when it is finally routed. Queues can be equated to links in some respects, but it’s not 100% accurate to think that both are the same. A link is an object that is associated with multiple queues, each of which can provide messages that will use the link. Another way of thinking of this is to define a link as an outbound connection to a single host that can be used by multiple queues.

      Per-destination queues represent an important new concept for Exchange. The alternate approach is implemented in the IMS in Exchange 5.5 where messages are held in a single queue until they are dispatched. If problems are encountered transmitting messages to a domain then all messages are blocked until the problem is resolved. For example, let’s assume that the IMS is used to connect Exchange 5.5 sites in Dublin, London, Dallas, and New York. Messages are normally routed from Dublin to Dallas via New York. If the network link goes down between Dublin and New York, messages are rerouted if another route exists. If not, they remain on the queue until the link is fixed. This underscores the importance of factoring in multiple available routes between routing groups, when possible, when you consider the design of an Exchange 2000 organization. Note too that only the messages on a single link are ever blocked by a failure that closes off communication to a specific destination. Messages going to all other destinations continue to flow unless they have to pass along the link affected by the failure.

      The Exchange 2000 routing engine will know that the link between Dublin and New York is down and attempt to discover alternate routes. If links are available between Dublin to London, and London to Dallas then messages can be moved to the per-link queue for London and be routed via that location. Rerouting happens automatically and doesn’t have to be manually reconfigured when problems occur.

      Outlook users experience an interesting side effect of the new routing engine. In Exchange 5.5, immediately messages are sent, Exchange copies the message from the user’s Outbox into a folder in the Store, and further processing is then performed to decide whether local delivery can be made or the message needs to be handled by the MTA. From a user perspective, the message moves very quickly from the Outbox to the Sent Items folder, and if messages stay in the Outbox, it’s an indication that something bad has happened. Exchange 2000 keeps a message in the Outbox until its addresses have been completely resolved and the message can be moved directly onto a queue for onward delivery. The change was made for performance reasons to avoid a potentially unnecessary item copy within the Store. However, users will often see messages “sticking” in the Outbox and may be concerned that something isn’t quite right. In fact, everything is proceeding to plan and it’s just another hint that Exchange 2000 is very different in many ways to previous versions.

⁶ Those interested in knowing how to create SMTP messages from programs, including the addition of subject lines and adding attachments and text, might like to browse a copy of John Rhoton’s book “Programmer’s Guide to Internet Mail” (ISBN 1-55558-212-5).

⁷ See http://www.imc.org/ube-relay.html. The report is dated July 5, 1999 and may have been revised by the time you read this.

Page: 1, 2, 3, 4, 5, 6, 7, 8

ADS BY GOOGLE