Which of the following protocols is used for retrieving messages?

SMTP Relay

SMTP relays are basically servers that are used to forward SMTP messages. SMTP relays can help you protect your internal e-mail servers. An SMTP relay server can be used to forward messages that are destined for or originating from the Internet. This way your internal e-mail servers are not exposed directly to the Internet. The problem occurs when there are what’s called “open” SMTP relay servers. Open SMTP relay servers are those that are not secure; they allow anyone to send a message through them. Spammers use open STMP relay servers to forward e-mails through. If your SMTP relay service is open, your system may be spamming without your knowledge. Eventually, your server may end up on an e-mail blacklist. This could greatly hinder your company’s capability to send legitimate e-mails.

If you are not using SMTP relay, you should make sure that your e-mail servers have the service turned off. If you are using the service, you should make sure the service is secure. You should limit who can send e-mails through your SMTP relay server. If your SMTP relay is only for internal forwarding, you should also limit where your SMTP relay server can send e-mails.

Simple Mail Transport Protocol

SMTP is a protocol used to transfer e-mail messages and attachments. SMTP is used to transmit e-mail between e-mail servers and from e-mail clients (such as Microsoft Outlook) to e-mail servers (such as Microsoft Exchange). However, most e-mail clients use other protocols, POP3 or IMAP, to retrieve e-mail from the server. These two server applications (SMTP and POP or IMAP) may exist on the same physical server machine.

As with the other protocols and services discussed in this section, SMTP operates at the Application layer and relies on the services of the underlying layers of the TCP/IP suite to provide the actual data transfer services.

IV.A. Simple Mail Transfer Protocol (SMTP)

The SMTP describes how an Internet e-mail client transfers e-mail to an Internet e-mail server, and how Internet e-mail servers transfer e-mail between themselves. SMTP is a text-based standard.

SMTP is defined by a number of IETF standards documents. RFC821, RFC1869, and RFC1870 define the transport of messages over a network. RFC822 describes the format of a text-based e-mail message (RFC's are available at www.ietf.org/rfc/rfcXXX.txt, where the number of the RFCs is substituted for the XXX). An SMTP message is composed of two parts:

Message header. Includes such fields as the message sender, the intended recipient, and the subject of the message.

Message body. An SMTP message can carry a payload of one text-based message.

The original SMTP standards have been extended greatly to handle additional functions. The negotiation of service extensions (RFC1869), the declaration of message sizes (RFC1870), delivery status notifications (RFC1894), and the return of error codes (RFC2034) are a sampling of such extensions.

Simple Mail Transfer Protocol

Many enterprise applications use SMTP for sending various emails from the application. In some cases the enterprise application will have a remote SMTP server set up and in others it will rely on a local SMTP server to accept the message and relay it to the next SMTP server in line. Regardless of the SMTP server location, it is important to be able to verify that these services are working and are accepting messages.

Testing SMTP can be done in a manner similar to that used for testing HTTP above. A TCP connection using a telnet client will enable you to connect to the SMTP server on the appropriate port and send specific commands that simulate what a SMTP client would send to create and send an email. To perform this test, use a telnet client to connect to port 25 (assuming that the default port is used) of the SMTP server in question. After establishing the connection, send the following commands:

HELO mail.fairclothsec.com

MAIL FROM:[email protected]

RCPT TO:[email protected]

DATA

Subject: Backdoor

.

QUIT

Please note that similar to the HTTP test, an extra “enter” is required after the final “.” in the SMTP test message. If everything is working correctly, you’ll receive a response similar to that shown in Figure 2.13. Keep in mind that the syntax for SMTP may vary slightly depending on the server type and version. For example, “HELO” on some systems is “EHLO” on others if Extended SMTP (ESMTP) is supported. ESMTP supports authentication, the use of TLS, and other features in addition to the core SMTP command set.

In addition to differences in command syntax, different servers respond in different ways to each command. In the example shown in Figure 2.13, you see that the server returned a message of “250 OK” when sent the “MAIL FROM” command. Other servers may respond with a more descriptive message such as “250 2.1.0 Sender OK”. Regardless of the messaging, the response code of “250” should be consistent.

When manually testing SMTP, there are a few different tests that you may want to run to help diagnose where a particular problem lies. First, send a message from a local (within the same server) user to another local server user. This will tell you whether or not the local SMTP server is at least handling basic message transfers correctly. The next test is to send a message from a local (within the same domain) user to another user within the domain. This test allows you to pinpoint whether or not the problem is a domain-specific issue. And finally, test sending a message from a local user to a user in a different domain. A failure in this test indicates that there is a problem routing mail externally.

SMTP

The SMTP is a protocol used to transfer e-mail messages and attachments. SMTP is used to transmit e-mail between e-mail servers and from e-mail clients (such as Microsoft Outlook or UNIX and Linux's sendmail) to e-mail servers (such as Microsoft Exchange). Most e-mail clients, however, use other protocols, such as POP3 or IMAP4, to retrieve e-mail from the server. These two server applications (SMTP and POP or IMAP) may exist on the same physical server machine.

As with the other protocols and services discussed in this section, SMTP operates at the application layer and relies on the services of the underlying layers of the TCP/IP suite to provide the actual data transfer services.

Recipient Filtering

When a message has been processed by the Sender Filtering agent and hasn't been rejected, it will be handed over to the Recipient Filtering agent. (Well, this isn't exactly true; the Connection Filtering agent will run once more, before doing so.) This will check the recipient of a given e-mail message against the Recipient Block list. As you can see in Figure 7.36, you can block recipients based on their e-mail addresses (that is, the SMTP address in the RCPT TO: field) as well as messages sent to recipients not listed in the Global Address List (GAL).The Edge Transport server can only check whether a recipient is in the GAL if you use EdgeSync subscription; otherwise, recipient data will not be replicated from Active Directory to ADAM.

NOTE

Any SMTP addresses entered on the Blocked Recipients list will only be blocked for senders located on the Internet. Internal users will still be able to send messages to these recipients.

If an external sender sends an e-mail message to a recipient that is either listed on the Blocked Recipient list or not present in the GAL, a “550 5.1.1 User unknown SMTP” session error will be returned to the sending server.

It worth noting that the Recipient Filtering agent works for only domains for which the Edge Transport server is authoritative. This means that any domains for which the Edge Transport server is configured as a relay server won't be able to take advantage of Recipient Filtering. Diagrams of the Edge Transport Server with the Recipient Filtering Agent disabled and enabled are shown in Figures 7.37 and 7.38, respectively.

SOME INDEPENDENT ADVICE

As mentioned earlier in this chapter, the EdgeSync service will replicate recipient data from Active Directory to ADAM every fourth hour. With this in mind, be aware that any new recipients created on your mailbox server on the internal network won't be able to receive e-mail messages from external senders before the EdgeSync service has taken place hereafter.

The Recipient Lookup feature also includes a SMTP Tarpitting feature that helps combat directory harvest attacks (DHAs). A DHA is a technique spammers use in an attempt to find valid SMTP addresses within an organization. This is typically done with the help of a special program that is capable of generating random SMTP addresses for one or more domains. For each generated SMTP address, the program also sends out a spam message to the specific address. Because the program will try to deliver a message to each generated SMTP address, an SMTP session is, of course, also established to the respective Edge Transport server (or whatever SMTP gateway is used in the organization). The program can therefore collect a list of valid SMTP addresses, since the SMTP session will either respond with “250 2.1.5 Recipient ok” or “550 5.1.1 User unknown,” depending on whether the SMTP address is valid or not.

This is where the SMTP Tarpitting feature comes into the picture. This feature basically delays the “250 2.1.5 Recipient ok” or “550 5.1.1 User unknown” SMTP response codes during an SMTP session. By default, the SMTP Tarpitting feature on an Edge Transport server is configured to a delay of 5 seconds (but the value can be changed for each Receive connector), which should help make it more difficult for a spammer to harvest valid SMTP addresses from your domain.

SOME INDEPENDENT ADVICE

The SMTP Tarpitting feature was originally introduced in Exchange Server 2003. In Exchange 2003 the administrator had the option of specifying a tarpit value in which he or she could define the number of seconds to delay a response to the RCPT TO command during an SMTP session. The problem in Exchange 2003 was that this value was fixed, which enabled spammers to detect this behavior so they could work around it. A common practice was to have the spam application establish a new SMTP session, if it detected it was being tarpitted. To solve this problem, the Edge Transport server uses a random number of seconds, making predictions much harder. Even if the spam application reconnects, it won't be in better shape; the Edge Transport server will know it's the same sending server, so it will retain the tarpit state.

Email or the SMTP

SMTP is the protocol for transferring electronic mail. RFC 5321 describes the protocol for the use of sending mail between mail servers also referred to as mail transfer agents. SMTP has a dedicated well-known port number 25. It is not the protocol for collecting mail by a user. There are two typical protocols users employ to download their email. They are Post Office Protocol (POP) and Internet Message Access Protocol (IMAP). Both allow the users to collect their email from a mail server and view it locally, but do it from a slightly different manner (Table 3.7).

Table 3.7. Basic SMTP Commands

Command Line Use of SMTP Protocol

SMTP is such a simple protocol. Using Telnet5 to connect to port 25 on a remote host you can type an email from the command line using the SMTP commands. This technique is usually blocked today due to hacking/phishing misuse but in the past it use to be a common way to illustrate the use of SMTP commands. The example below shows an email sent by command line from Samuel on yourmail.123.com to Lindsey on mymail.xyz.com.

% telnet mymail.xyz.com.25

Trying 162.21.50.4…

Connected to mymail.xyz.com

Escape character is ‘^]’

220 mymail Sendmail 4.1/1.41 ready at Tue, 29 Dec 2012 19:23:01 PST

helo yourmail.123.com

250 mymail Hello yourmail.123.com, pleased to meet you

mail from:<[email protected] yourmail.123.com>

250 <[email protected] yourmail.123.com>… Sender ok

rcpt to: <[email protected]>

250 <[email protected]>… Recipient ok

data

354 Enter mail, end with “.” on a line by itself

Hello Lindsey, how are you?

.

250 Mail accepted

quit

221 mymail delivering mail

Internet Message Access Protocol

IMAP also allows access to and the downloading of emails from a mail server. However, IMAP is more complex in that it allows the users email client to access the emails on another server as if it were locally stored. This allows the user’s email client to manipulate emails stored on the server without transferring the messages between computers (Figure 3.17).

Figure 3.17. SMTP communications between clients.

Simple Mail Transfer Protocol

The Simple Mail Transfer Protocol (SMTP) is used to deliver e-mail messages over the Internet. This protocol is used by most e-mail clients to deliver messages to the server, and is also used by servers to forward messages to their final destination. SMTP is only used for delivery; it cannot be used to retrieve e-mail messages from servers. SMTP servers, also known as Mail Transfer Agents (MTAs), typically listen on port 25/TCR They use DNS Message Exchange (MX) records to determine the mail server address for a particular domain name. Like some of the previously discussed application layer protocols, SMTP is very old and was not designed with security in mind.

SMTP Protocol Overview

SMTP uses ASCII text for communication. Similar to FTP, the client sends commands to the server, and the server replies with a numeric response code followed by an optional message.

For the following example, we connect directly to an SMTP server with a Telnet client and feed it commands in order to send an e-mail. The output from this example is shown in Code Listing 8.3.

Code Listing 8.3

Sending an E-mail

[email protected] &gt; telnet localhost 25

Trying 127.0.0.1…

Connected to localhost (127.0.0.1).

Escape character is '^]'.

220-1ocalhost ESMTP Exim 4.52

HELO localhost

250 localhost Hello localhost [127.0.0.1]

MAIL FROM:[email protected]

250 OK

RCPT TO:[email protected]

250 Accepted

DATA

354 Enter message, ending with &quot;?&quot; on a line by itself

Subject: Hi!

From: [email protected]

To: [email protected]

Hello

·

250 Message queued for delivery

QUIT

221 localhost closing connection

Connection closed by foreign host.

After the TCP connection is established with the server, it sends a 220 code with a banner message. At this point, we introduce ourselves to the server using the HELO command followed by our domain name, and the server replies with a 250 code that returns the greeting.

After the greeting is complete, the MAIL command is used to tell the server who the sender of the message is. Next, the RCPT command is used to specify the recipients of the e-mail. In this example, there is only one recipient; however, more can be specified at this point.

After the sender and recipient are sent to the server, the DATA command is used to provide the message. After typing DATA, the server interprets all input from the client as part of the message, until it receives a period on a line by itself. At this point, the server adds the message to its queue and the QUIT command is sent to close the connection.

An e-mail message consists of a header followed by a body. For the most part, the header information is not used by the SMTP server. However, each SMTP server that relays e-mail alters the header by adding a line that includes the address of the relay that the e-mail came from, the address of the server that received it, and the time at which it was received. Each time an e-mail is passed between two relays, one of these lines is added.

Since the protocol was designed to only support 7-bit ASCII text, Multipurpose Internet Mail Extensions (MIME) extensions are used to send other types of files and data in e-mail messages. Using MIME, various types of files can be mapped into a, format that can be sent via SMTP, and then mapped back to their original format once they have been delivered to the recipient.

SMTP

The Simple Mail Transfer Protocol (SMTP) is used for sending and receiving e-mail between e-mail clients and servers. When an SMTP server receives an e-mail from a mail client, the SMTP server will then check the MX records for the domain in the e-mail address in order to exchange the mail with the remote SMTP server. It will then either process the mail (if it is the MX server) or forward it to the appropriate SMTP server.

For SMTP to work properly, a set of MX records has to be defined within the name server’s DNS database for the recipient’s domain. An MX record has two specific pieces of information: a preference number, and the DNS name of the mail server that’s configured to handle mail for that domain. If there is more than one mail server for the domain, the SMTP server will choose one based on its preference number. The lowest number will have the highest priority, and based on availability, the SMTP server will work its way up from there.

One can view the headers of a received e-mail to see the path the e-mail traveled from client to server to destination endpoint. Each time an e-mail is passed to and from an SMTP server, information regarding the server is recorded in the header. Fig. 2.13 shows an example of an email header with SMTP server information using the RFC 2822 (www.ietf.org/rfc/rfc2822.txt) format.

Once the local mail server receives the mail message, it is given an initial header (received by), which appears as:

Received: from [sending-host’s-name] [sending-host’s address] by [receiving-host’s-name]

[software-used]

with [message-ID]

for [recipient’s-address]; [date][time][time-zone-offset]

You can see two examples of such headers in Fig. 2.13. The message then progresses through numerous mail relays where the message is given appended header information. The mail is eventually received by the recipient’s mail server and is stored in the recipient’s mail account (inbox), where the user downloads it. At this stage, the message has received a final header. Additional information given by the headers includes Message IDs, Multipurpose Internet Mail Extensions (MIME) version, and content type.

MIME is a standard for handling various types of data and essentially it allows you to view mail as either text or HTML. Other MIME types are defined that enable mail to carry numerous attachment types. A message ID is assigned to a transaction by a particular host (the receiving host or the “by” host). Administrators use these message IDs to track transactions in the mail server logs.

Mail headers are interesting to us because they show us where the mail servers are. In addition, they tend to deserve special attention because mail servers are usually where the people are, and that’s usually right at the heart of the network. Mail servers are very seldom hosted outside the private network in large organizations and thus represent an organization’s core infrastructure to us. However, in smaller organizations, this may not necessarily be the case as mail servers may be hosted in cloud environments or by using third-party email providers.

Summary

SMTP is much simpler than POP3 and is ideal for embedded systems for sending e-mail messages. It is also appealing because all messages can be small, and easily handled by RAM limited systems. The steps required for a simple mail dial-up transaction are:

Client (assumed the embedded system) contacts the ISP over a dial-up telephone connection. The client uses AT commands to enable the modem dial the number and establish the connection.

The ISP will answer with a PPP (or SLIP) initialization sequence. The client will negotiate a convenient protocol using PPP commands. In the case of an embedded system, these could be the minimum necessary. During negotiations, the client will send its logon ID and password. If accepted, the ISP will return an allocated IP address for the session. The client will effectively become this IP address for the duration of the call.

Having negotiated PPP parameters, the client can now talk directly to the mail server. The first thing it will do is to open a TCP connection (port 25 for SMTP, port 110 for POP3).

When the connection is open, the client will send commands and e-mail body text sequences as described in the previous section, always waiting for replies.

At the end of the transaction, the client can drop the connection, either by closing the TCP link, or more crudely, by simply dropping the phone line.

Reading Email Headers

Here is a brief analysis of the life of a piece of email. This background material may be important for understanding how emails are transmitted, especially for automatically generated emails such as may be used in an embedded system. Let us assume that user [email protected] wants to send a simple email to [email protected] Let's further assume that the remote system is using a mail handler program called supermail (the IP addresses shown are fictitious). When myname wants to send an email to yourname, he or she composes it at their PC workstation, which is possibly connected to the myisp network via a dialup line. The computer at myname contacts the ISP via a dialup line (assume the web name of the mail handling system at the ISP is mail.myisp.com). The mail server, now seeing that it has a message to forward to another computer, contacts the mail server at the remote location and delivers it (assume this is called mail.yourisp.com). The remote destination user yourname retrieves the message by logging in to yourisp.com and downloading the message. During this processing, headers will be added to the message at least three times: At composition time, by whatever email programmyname is using. When the message passes through mail.myisp.com, And at the transfer between myisp.com to yourisp.com. Sometimes, the remote ISP program that downloads the messages may add a further header. At composition time, the message header is:

Note: The two command lines Date: and X-Mailer: were not included in the original sender's text. These are usually added by user mail applications. When myisp.com processes the message for sending to yourisp.com, the headers have now become:

When the message is received at yourisp.com the headers have now become:

This last set of headers is what the recipient sees on his email text. The first extra line (note the word wrap, it is all in a single line) shows that this email was received from a machine calling itself mail.myisp.com (i.e. the first reference to mail.myisp.com in the line). The next entry in brackets show who the real sender was. In this case the sender was IP address 192.123.1.12, which just happens to correspond to mail.myisp.com. (the system will do a reverse check). The same line shows that the machine that did the receiving was mail.myisp.com, and that is running a Supermail program version 1.1.2. The receiving machine assigned ID number LAA2001 to the message, this is used internally by the system for logging and administration purposes. The line also shows the message was destined for <[email protected]>. Note that this header is not related to the To: header. The second extra line shows the similar information but related to the previous hop. That is at the point between myisp.com and yourisp.com. Note the differences in travel times for the messages., and the repeated entries for source and destination machines. This seemingly irrelevant repetition of addresses only makes sense when considering Relays, these are intermediate machines in the transmission path. A message may not go directly from machine A to B, but possibly from A to C, then to D and finally to B. The third extra line is a Message Id line added by the first mail sender to identify it, and to be able to track it during its lifetime.

The situation above is rather simplistic. In reality, a message may pass through several more machines (including firewalls and relays) each adding an extra header to the message. This contributes to the strange system addresses found on some email messages.