WO2005055535A1

WO2005055535A1 - Computer network system and method

Info

Publication number: WO2005055535A1
Application number: PCT/GB2004/004968
Authority: WO
Inventors: Clive Homewood
Original assignee: Boots Board Limited
Priority date: 2003-11-27
Filing date: 2004-11-25
Publication date: 2005-06-16
Also published as: GB0327596D0

Abstract

A method of operating an Internet mail server is disclosed. The method comprises performing one or more tests upon an incoming e-mail message to assess whether it is an acceptable message; and in the event that the e-mail message is found to be unacceptable, preventing the server from sending acknowledgement of receipt of the message to indicate to the sender. In typical embodiments, mail is transported using the simple mail transport protocol. Acknowledgement of receipt of the message is advantageously achieved at a lower level in the protocol stack such that it is transparent to the mail transport protocol.

Description

Computer network system and method

This invention relates to a computer network system and method. In particular, it relates to a computer system and method for handling electronic mail.

In parallel with the recent rapid expansion in the use of electronic mail as a communication medium, there has been an expansion in unsolicited messages, which are commonly known as "spam". An unsolicited e-mail message is similar to junk mail people receive in the post. A sender of bulk unsolicited e-mail messages (known as a "spammer") typically uses other people's mail servers to deliver their e-mail campaigns at other people's expense. Unsolicited e-mail costs are covered by the ISP (Internet Service Provider) processing the e-mail messages and the individuals receiving the messages.

In most cases spam is unwanted commercial e-mail - junk e-mail and advertising. In a few cases, users of the net are unfortunate enough to receive unsolicited religious, racial or sexual messages, which is a somewhat more serious matter. Receiving one or two such messages a day is annoying. Receiving dozens of them is a genuine drain on your time and resources.

Internet service providers (ISPs) normally route e-mail to the correct addressee companies or individuals. Unsolicited e-mail campaigns are often seen as congestion on ISP networks. There are a number of different methods presently used in the fight against unsolicited e-mail by the ISPs.

One method is to delete all the unsolicited e-mail. This method requires the ISP to operate additional hardware and/or software to analyse each incoming e-mail message and make a decision based on rules to decide if the message is considered unsolicited. If it is, the e-mail is deleted. This method can cause false positives. A false positive is when an e-mail filter based on its rules has decided to delete the message when the message is not, in fact, unsolicited. Another method uses the same filters, but simply marks the message in the subject as a possible unsolicited message, allowing the individual that receives the e-mail to make the decision whether to delete it or not.

Both methods described above have disadvantages. The first method is the one most prone to error. A false positive can cause the deletion of good messages and the ISP has to incur costs for additional hardware and software. The second solution stops deletion of the false positives, by allowing the message to be delivered, but the individual still has to receive the messages and the network must transport them. If a spammer sends out hundreds of thousands of e-mail messages, all being identical, the filter will analyse each message. This is inefficient. Neither solution targets the original problem of the e-mail being received in the first place.

An aim of this invention is to provide e-mail handling systems and methods that discourage or prevent mass distribution of unsolicited e-mail messages. Simply handling messages after they have been received does not achieve this. Spammers sell their spamming techniques to customers by sending out large amounts of unsolicited e-mail messages and getting paid for it by their customers. The spammer has to show that the message was successfully sent from their systems to get paid, not that it was received by the targeted individual. So as long as a message is shown as sent the sparrimer does not mind if the ISPs or recipients delete them as that does not affect their income. In fact, the more systems the ISPs put in place to delete the messages the more the spammers can send.

Typically, e-mail is handled in the Internet using the simple mail transport protocol (SMTP), an application layer protocol in the OSI model. The SMTP handles sending and receiving of the e-mail traffic and a receiving SMTP server confirms that a message has been received by a mail transport agent on a destination host. This is achieved by an exchange of packets between the sender and the destination in accordance with the protocol. For practical purposes, it is not possible to change this protocol on an e-mail server because this would obstruct normal operation of e-mail transport, therefore, this invention is designed to be compatible with existing SMTP e-mail systems. From a first aspect, this invention provides a method of operating a mail server, the method comprising: performing one or more tests upon an incoming e-mail message to assess whether it is an acceptable message; and in the event that the e-mail message is found to be unacceptable, preventing the receiving server from sending acknowledgement of receipt of the message to the sender.

Methods embodying this aspect of the invention are surprisingly effective in discouraging spam messages. The impression given to the sender is that the message has failed to send, so the spammer does not receive payment for sending the message. Moreover, the sender will typically wait for some time for receipt of acknowledgement before timing out. This blocks each sending thread within the source machine, so limiting the total number of messages that can be sent by the sender.

Typical embodiments of the invention operate to receive messages using an application layer protocol that operates according to the simple mail transport protocol (SMTP). In such embodiments of the invention, prevention of acknowledgement may be implemented within a layer of the network stack lower than that of the application layer. For example, it may take place at the transport layer, which operates using the transmission control protocol (TCP) in the Internet. This allows the SMTP layer to operate without modification, non-delivery of the rejected message appearing to the SMTP layer as an apparent network error.

Information relating to servers that are sources of unacceptable e-mail may be stored in a database. Information that has been stored in the database may be used to determine how subsequent e-mail from a particular source server should be handled, or whether a further connection from a particular sending SMTP should be accepted at all. For example, if an e- ail message comes from known spammer, the SMTP server from which the spammer's messages originate may be denied connection for a predetermined period, for example, two days.

Optionally, the server may be operative, upon identifying a source of inappropriate e-mail messages, to send a message to one or more other server operating according to a method embodying the invention, the message identifying a server from which inappropriate messages have been received. Upon receipt of such a message, the server may store information (for example, in the database) to identify the server identified in the message as one from which messages should be rejected automatically and, for example, for a predetermined period, such as for two days. This may be implemented in a peer-to-peer network, or with a central server that co-ordinates distribution of information relating to sources of unsolicited e-mail messages amongst servers of which it is aware.

From a second aspect, this invention provides a mail server operative to process mail messages by means of a method according to the first aspect of the invention. The mail server may comprise conventional computer hardware and software that, when executed by the hardware, performs a method according to the first aspect of the invention.

From a third aspect, this invention provides a network stack that can be executed on computer hardware to process e-mail messages in accordance with the first aspect of the invention.

From a fourth aspect, this invention provides a software product that can be executed on computer hardware to process e-mail messages in accordance with the first aspect of the invention.

An embodiment of the invention will now be described in detail, by way of example, and with reference to the accompanying drawings, in which:

Figure 1 is a general overview of a spammer distributing unsolicited e-mail messages by way of the Internet according to known methods; and

Figure 2 illustrates components of an SMTP server embodying the invention.

With reference to Figure 1, a spammer uses a server 10 to send bulk unsolicited e-mail messages.

A spammer typically operates through use of an outgoing mail server 10 that has a direct connection to the Internet 12. The server may be a computer owned by the spammer, or may be a server operated by an Internet service provider (ISP). The server operates networking software that is capable of generating e-mail messages from a list of many e-mail addresses and sending them using the simple mail transport protocol. The messages are routed by Internet servers to the intended recipient mail host 14. Upon the host, each message is analysed by filters 16 to determine whether or not it is an unsolicited message. If it is not unsolicited, it is delivered to the individual addressee's mailbox 18. Otherwise, it is handled as unsolicited mail, either by deleting it or by changing its subject line to indicate that it is unsolicited before delivering it to the user, illustrated at 20.

This invention is implemented in the software systems of the recipient mail host 14.

When an e-mail client or server sends an message, it goes through a stage of communication, where it has to tell the incoming SMTP server where the message originated, the recipient and the data contained. This communication involves the sender 10 sending to the recipient 14 one or more SMTP commands, which the recipient 14 acknowledges by sending one or more SMTP replies back to the sender 10. Therefore, after each piece of this information is passed to the receiving SMTP server, the sending SMTP waits for the receiving server to acknowledge it has received the information correctly. If the sending server 10 does not receive answers to its requests in accordance with the specifications laid down in the simple mail transport protocol, it waits for a time that is not defined by SMTP to receive the acknowledgement. This time varies depending on the manufacturer of the SMTP server.

In implementing the invention, the receiving SMTP server 14 analyses incoming e-mail messages, and applies analysis rules to determine whether it has become the target of a campaign of unsolicited e-mail. The analysis applied to incoming messages includes rules to detect the following properties of a message:

• Known good domain list

• Known bad domain list

• Known good e-mail address

• Known bad e-mail address . Xmailer Detection

. Blocked IP's List

• Ignored IP List

• Blocked File Extensions . CC/BCC Count • Ignore Count based on domain/e-mail address E-mail Body Remark Count Hidden URL Detection Body Context Matching Filter HTML format checking RBL lookup Bayesian filtering technology Sent E-mail against IP Returned E-mail against IP Deleted E-mail against IP • SQL logging and analyse Employee E-mail Tracing Header change based on content Domain Routing Rules E-mail Routing Rules • Base64 decoding Virus Checking Star Network Technology Append Disclaimers based on Domain E-mail Traced on external systems

This is not an exhaustive list. Some embodiments may look for other properties, and some may not include all of the above.

The software receives the e-mail message and applies configured rules based on the above properties. The information is then passed to a database where information is stored to determine how any future events involving the sending sender are stored. If a rule applied to a message fails, that is to say, it concludes that the incoming message contains fake headers or has known unsolicited content, a message is sent to all SMTP servers that implement the invention instructing them to block messages from the sender. The current message that failed the test is deleted and the sending server is marked in the database as blocked.

If the tests suggest that a message might be unsolicited with a high probability, but not with certainty, it is marked for the individual receiving the message to make the decision as to whether or not it is an unsolicited e-mail message. Further checks are made in the background by the invention to see if the source of the e-mail is from a known spammer. If it is determined that the email is from a known spammer, a block command is sent to all incoming SMTP servers not to accept a connection from the source SMTP server.

When a sending server has been identified as a source of unsolicited messages, either by locally applied rules 32 or as a result of a message received from another server, this fact is stored in the database 30, and subsequent messages originating from the server are handled accordingly.

As discussed above, the communication between a sender and a receiver involves an interchange of commands and replies. If satisfactory replies are not received, the sender cannot assume that the message has been sent. The simple mail transport protocol 22 relies upon a transport-layer protocol 24, typically the transmission control protocol (TCP) to carry its data frames. TCP is a connection-based protocol (that, in turn, relies upon lower network layers 26 that will not be discussed here) that can establish reliable data transport between hosts. The SMTP layer 22 must establish a TCP connection to the well-known port 25 on the recipient before any transport of e-mail messages can take place. In this embodiment, the TCP network layer 24 of the receiving host 14 operates to examine all frames received on port 25, and consults the database 30 to determine how they should be processed. If a frame is received from a host that is marked in the database as a source of unsolicited e-mail, it is handled in a non-standard manner that ensures that SMTP communication cannot be established.

More specifically, an SMTP transaction can be considered to consist of the following events: the transport event and the routing event. The invention can be implemented by providing a hook into the transport event and between the transport event and the routing event.

When an incoming e-mail message is received by the transport layer, and before it replies (as required by TCP) to confirm success, the embodiment is invoked to check whether the message is for a valid user or from a valid sender. A valid sender, in this context, is one that is not listed in the database as a source of unsolicited e-mail messages. The incoming IP address of the sending SMTP server is compared with the database of blocked sending servers. This event is handled at the transport layer, and therefore takes place before any data reaches the SMTP layer. Thus, packets from blacklisted servers are blocked in much the same way as packets are blocked by a firewall. Thus, if the IP address of a server is in the database, this embodiment will stop any data passing from the blocked server to the SMTP layer. This occurs before the transport and routing layer.

If the message is found to be acceptable, the embodiment releases the message and the ■ transport event then continues and sends an acknowledgement to the sender, as required by the transport control protocol. If the message is rejected, an error code is returned by the transport layer.

The released message is then passed to the SMTP transport event. The second event then fires. At this time, the SMTP server is communicating with the sending SMTP server and exchanging information about the sender and recipient of the massage, etc. The software now has the information regarding the sender and recipient. The software then checks for valid senders and receivers. It also checks that the message is not coming from a known banned e-mail address or domain name. If this fails an error is returned to the sending SMTP server. If it passes, the second event is stopped and the standard SMTP server continues. This occurs during the transport layer. The second event does not have the e-mail message in a complete form, so a third event is required for the message to be processed in full.

The released message is then passed to the SMTP routing event. The software of the embodiment can now access the complete message. The third hook is then activated before the message is processed by the routing event. The software now analyses the content of the message, for example, for banned file extensions and for content that suggests known spammer techniques have been applied to generate the message. If this test fails the TCP block is applied to all SMTP servers. The software then checks for valid senders and receivers. It also checks that the message is not coming from a known banned e-mail address or domain name. The domain/e-mail address could be being sent from a mail server different to that specified as the sender or reply-to address, so it could have got through the first event, described above.

It will be noted that the operator of the sending SMTP server is penalised because all messages from it are blocked when it is determined to be a source of spam. This is done so that the problem is removed from the recipient's server and placed in the hands of the operators of the sending server. This has the effect that if the operators were unaware of the spammer the fact that their messages are being blocked will draw attention to the existence of a problem. The operators can then take action to stop the spammer.

The second event does not have the e-mail message in a complete form, so a third event is required for the message to be received in full. This is the mail event that does the checks on the format of the message and applies the rules described above. This occurs just before the routing layer and after the transport layer.

The SMTP uses the TCP for its network protocol. When the events described above are invoked, the software systems embodying the invention are activated. These systems communicate with one another directly with the TCP protocol. This is so as not to interfere with the SMTP; only the TCP protocol is affected. This is why the SMTP layer is unaware of the changes that have occurred.

Many users retrieve their e-mail messages from a post office server (e.g., operating the POP3 or IMAP protocol) that gathers messages sent to the user when the user's computer is not online. A user can then download their messages from the post office whenever it is convenient to do so. The system implemented by this embodiment operates in front of the post office server, so the post office server is completely unaffected by messages that are blocked. This provides a decrease in bandwidth usage and a corresponding increase in server capacity. Therefore, the post office server has more time to process good e-mail messages and because there are fewer e-mail messages coining in, there is less traffic on the Internet connections.

It will be seen that e-mail messages from a blocked host are prevented without any alteration to the operation of the mail transport protocol. The mail handling software on the sending host 10 is presented with an apparent lower-level networking failure, no usable connection being established at the transport layer. It is not, therefore, apparent to the sending server 10 that its messages have been blocked intentionally. The receiving SMTP server and sending SMTP server are unaware of the break that has occurred and go into an idle state waiting for each other to reply.

Since the receiver detects the arrival of a campaign of unsolicited messages, and prevents the spammer from sending further messages, this leaves the many messages generated by the spammer's software unsent on the spammer's server. It is then the spammer's problem in removing the undelivered messages. Moreover, the spammer cannot prove to their customers that the e-mail campaign was successful, so there is no reason for the customer to pay the spammer. Therefore, this invention directly targets the spammer's business model. This concept causes the spammer to stop targeting the ISP or companies running the servers that embody the invention. It can also reduce the spammer's ability to target other servers. Typically, a sending server will start a thread to send a message, and that thread will run until the message has been sent or it has been finally decided that the message cannot be sent. Some network activities, such as initiation of a TCP session, will cause the thread to block until completion or a timeout period has elapsed. Since a system can only support a finite number of threads at any one time, any thread that is blocked while trying to send a message to a server embodying the invention cannot become free to send a message to another server.

The TCP layer can react in several ways when it receives frames from a blocked server. For example, it may send no frames at all back to the sending server 10. In that case, the session setup fails entirely. Alternatively, it may allow the session to be set up, but then fail to acknowledge subsequent packets sent over the established session. As a further alternative, it may initiate a session teardown a short time after the session has been established.

A network that includes several mail servers embodying the invention can share information relating to sources of unsolicited e-mail messages. Thus, when one such server adds a source to its database, it can send a message to one or more other servers, each of which will add the same information to its database. This could be implemented in a peer-to-peer configuration, with each server broadcasting the messages to all other servers on the network. However, it is preferred that it will instead use a star network. In this configuration, if one server detects a source of unsolicited e-mail messages, it sends the information to a central server. The central server then sends a message to all servers of which it is aware, instructing those servers to update their databases.

Claims

1. A method of operating a mail server comprising: performing one or more tests upon an incoming e-mail message to assess whether it is an acceptable message; and in the event that the e-mail message is found to be unacceptable, preventing the server from sending acknowledgement of receipt of the message to the sender.

2. A method according to claim 1 in which the server is operative to receive messages using an application layer protocol that operates according to the simple mail transport protocol (SMTP).

3. A method according to claim 1 or claim 2 in which the server is prevented from sending acknowledgement by operation of a layer of the network stack lower than that of the application layer.

4. A method according to claim 3 in which the server is prevented from sending acknowledgement by operation of the transport layer.

5. A method according to claim 4 in which the transport layer operates in accordance with the transmission control protocol (TCP).

6. A method according to any preceding claim in which information identifying a server that is a source of unacceptable e-mail is stored in a database.

7. A method according to claim 6 in which information that has been stored in the database is used to determine how subsequent e-mail from a particular source server should be handled.

8. A method according to claim 7 in which all incoming e-mail messages from any server that is listed in the database may be blocked automatically.

9. A method according to any preceding claim which is operative, upon identifying a source of inappropriate e-mail messages, to send a message to one or more other server, the message identifying a server from which inappropriate messages have been received.

10. A method according to claim 9 in which, upon receipt of such a message from another server, the server stores information to identify the server identified in the message as one from which messages should be rejected automatically.

11. A method according to claim 9 or claim 10 in which, upon identifying a source of inappropriate e-mail messages, the server sends a message to a central repository, which then forwards that information to other servers of which it is aware.

12. A mail server operative to process mail messages by means of a method according to any preceding claim.

13. A mail server according to claim 12 comprising: computer hardware; and computer software that, when executed by the hardware, performs a method according to any one of claims 1 to 11.

14. A network stack that can be executed on computer hardware to process e-mail messages in accordance with a method according to any one of claims 1 to 11.

15. A software product that can be executed on computer hardware to process e-mail messages in accordance with a method according to any one of claims 1 to 11.