US20040177158A1

US20040177158A1 - Network address translation techniques for selective network traffic diversion

Info

Publication number: US20040177158A1
Application number: US10/430,997
Authority: US
Inventors: David Bauch; Warren Ryd
Original assignee: Individual
Current assignee: Hyperspace Communications Inc
Priority date: 2003-03-07
Filing date: 2003-05-06
Publication date: 2004-09-09
Also published as: WO2004082152A3; US7260599B2; US20040177359A1; WO2004082152A2

Abstract

A facility for diverting a network packet to a diverted destination is described. The facility selects for diversion a network packet that has been submitted for delivery and whose delivery is not yet complete. The network packet has a destination address, a destination port, a source address, and a source port, all with initial values. In the network packet, the facility: substitutes the initial value of the destination port in the source port; substitutes an address for the diverted destination in the destination address; and substitutes a port for the diverted destination in the destination port. After the substitution, the facility releases the network packet for delivery to the diverted destination.

Description

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent application Ser. No. 10/383,992 filed Mar. 7, 2003, which is hereby incorporated by reference in its entirety.[0001]

TECHNICAL FIELD

The present invention is directed to the field of computer networking, and, more particularly, to the field of exchanging application data via networks.

BACKGROUND

An application program (“application”) is a computer program that performs a specific task. A distributed application is one that executes on two or more different computer systems more or less simultaneously. Such simultaneous activity is typically coordinated via communications between these computer systems, generally via a network.

One example of a distributed application is Usenet, an application enabling large number of users to participate in conversations on specific topics. These topical discussions, called newsgroups, are comprised of textual messages. A user of the Usenet application executes a client portion of the Usenet application, called a news client or a newsreader, on the user's computer system. The news client communicates with a news server program executing on one of a large number of news server computer systems using a protocol called NNTP (Network News Transfer Protocol) in order to perform such functions as retrieving a list of newsgroups, retrieving a list of the messages in a particular newsgroup, retrieving a particular message to be displayed to the user, or posting to a newsgroup a message authored by the user.

Because messages received by the news server executing on a particular news server computer system are forwarded to most or all of the other news servers, comparable sets of messages are available on many or all of the available news servers. Typically the user configures the news client to contact the news server on a particular news server computer system by supplying the Internet address—or “IP (Internet Protocol) address”—of that news server computer system. The news client, thus configured, contacts this particular news server each time it needs to contact a news server to complete a task.

Unfortunately, for such client/server applications, the user often selects a server that is sub-optimal at the time of its selection, or that becomes sub-optimal at some future time. For example, a user may select a first news server that has a typical response time of 2 seconds, rather than a second news server unknown to the user that has a typical response time of 0.05 seconds. Similarly, a user that weeks ago selected the second news server may not manually switch to the first news server when a partial network failure raises the average response time of the second news server to 5.5 seconds.

Further, the protocols relied upon by many distributed applications to communicate between portions of the application executing on different computer systems fail to incorporate or otherwise accommodate such services as encryption and compression. Further, the few such protocols that do incorporate services such as encryption and compression incorporate particular variations of these services (e.g., 56-bit DES encryption), and make it difficult to utilize others instead (e.g., 128-bit DES encryption).

In view of the foregoing, an improved approach to facilitating the exchange of data by distributed applications that successfully automated the selection of a server, and/or that permitted the use of various different data processing techniques such as encryption and compression, would have significant utility.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a typical environment in which the facility operates. [0009]
FIG. 2 is a block diagram showing architectural details of a typical private application network client. [0010]
FIG. 3 is a block diagram showing architectural details of a typical private application network server. [0011]
FIG. 4 is a flow diagram showing steps typically performed by the facility in a private application network client to process an application request received from an application client. [0012]
FIG. 5 is a flow diagram showing steps typically performed by the facility in order to transmit application requests accumulated for a particular private application network server in an outgoing buffer of the private application network client. [0013]
FIG. 6 is a flow diagram showing steps typically performed by the facility to process application requests received by a private application network server from a private application network client. [0014]
FIG. 7 is a flow diagram showing steps typically performed by the facility in a private application network server to process an application response received from application servers. [0015]
FIG. 8 is a flow diagram showing steps typically performed by the facility in order to transmit application responses accumulated in a private application network server to a destination private application network client. [0016]
FIG. 9 is a flow diagram showing steps typically performed by the facility to process application responses received by a private application network client. [0017]
FIG. 10 is a flow diagram showing steps typically performed by the facility to process a request from a private application network client to provide a list of servers for a particular application. [0018]
FIG. 11 is a flow diagram showing steps typically performed by the facility to redirect application network traffic to a PAN tunnel. [0019]
FIG. 12 is a flow diagram showing steps typically performed by the facility in order to apply a rule table used by the facility to a packet. [0020]
FIG. 13 is a data structure diagram showing a sample rule table typically used by the facility in an initial state. [0021]
FIG. 14 is a data structure diagram showing the sample rule table of FIG. 13 in an updated state. [0022]
FIG. 15 is a data structure diagram showing a sample mapping table typically used by the facility in an initial state. [0023]
FIG. 16 is a data structure diagram showing the sample rule table of FIG. 14 in an updated state. [0024]
FIG. 17 is a data structure diagram showing the sample mapping table shown in FIG. 15 in an updated state. [0025]
FIG. 18 is a flow diagram showing steps typically performed by the facility to respond to a mapped destination port request made by the driver. [0026]
FIG. 19 is a data structure diagram showing a sample PAN tunnel channel table typically used by the facility. [0027]
FIG. 20 is a flow diagram showing steps typically performed by the facility to execute an accept loop for the parent socket. [0028]
FIG. 21 is a data structure diagram showing a sample PAN socket table typically used by the facility. [0029]
FIG. 22 is a flow diagram showing steps typically performed by the facility in order to route data received from an application server via a particular PAN tunnel channel. [0030]
FIG. 23 is a flow diagram showing steps typically performed by the facility in order to apply the facility's mapping table to a network packet.[0031]

DETAILED DESCRIPTION

A software facility for supporting the exchange of data by distributed applications (“the facility”) is provided. In particular, the facility establishes a private application network (“PAN”) comprised of private application tunnels (“PAN tunnels”) for exchanging data on behalf of distributed application in a manner that optimizes the selection of application servers; provides a negotiated level of transmission services such as encryption and compression, using an extensible set of transformation modules; and is easily adapted to operate with new applications, through the use of an extensible set of modular application agents. [0032]
On client computer systems, the facility uses an extensible set of modular client agents to intercept server requests from application clients, and combines application requests destined for the closely located server computer systems for transmission to those server computer systems. On server computer systems, the facility receives bundles of one or more application requests, dispatches each application request to the corresponding application server, collects application responses from application servers, and bundles them for transmission back to the originating client computer systems. Back on the client computer systems, the facility receives bundles of application responses, and dispatch each to the corresponding application client. [0033]
Some embodiments of the facility capture application traffic originating with application clients using a special network device driver. The device driver identifies network packets containing application requests that are transmitted by application clients to corresponding application servers, and modifies—or “mangles”—the addressing information in their headers in order to divert these packets to the PAN client, which forwards the packets via the PAN to a PAN server for delivery to the appropriate application servers. When responses to such packets are sent back from these application servers via the PAN to the PAN client, the PAN client generates packets that are identified by the network driver and mangled in the opposite direction, such that they are delivered by the network to the originating application clients. In particular, the mangling process maintains the source port of the request packet, so that this port number is present in the response packet to facilitate delivery of the response packet to the requesting application client. The mangling process also retains the destination address of the request packet in the source address of the mangled request packet, both to ensure that response packets are addressed to a remote computer system so that they are delivered to the special network device driver, and to enable the facility to determine how to mangle these response packets in the backwards direction. The use of this selective network traffic diversion approach obviates any need to modify the application clients or their configuration in order to intercept and process their traffic for the PAN. [0034]
Some embodiments of the facility use application routing techniques to identify an application server best suited to process application requests from each application client, using a configurable variety of routing criteria, and based upon information received from a central source, independently obtained by each client computer system, or both. [0035]
Some embodiments of the facility use an extensible set of modular agents to interface with application clients and servers, both (1) to intercept application requests sent by application clients and application responses sent by application servers for transmission through a PAN tunnel, and (2) to deliver application requests to application servers and application responses to application clients that have been received through a PAN tunnel. By adding a new agent for a new distributed application to the set of agents, the facility can be extended to operate with the new application. [0036]
Some embodiments of the facility use an extensible set of transformation modules to transform application requests and responses sent through a PAN tunnel in a way negotiated as part of establishing the PAN tunnel to provide such transmission services as encryption and compression. The negotiation of transformation techniques enables the facility to adapt the transformation techniques used to the particular circumstances of the PAN tunnel, as well as to the specific set of transformation modules installed on each computer system. Additionally, by adding a new transformation module for a new transformation technique to the set of transformation modules, the facility can be extended to utilize the new transformation technique. [0037]
In one example of the operation of the facility, a PAN client and application clients including a Simple Mail Transfer Protocol (“SMTP”) client for delivering email are installed on a laptop computer system. The laptop computer system is used by a user employed by a company. The laptop computer system is usually used in the company's Chicago office, but is currently being used in a hotel room in Las Vegas via dialup connection. While the laptop is being used in this manner, the user needs to send an email having a large attachment. When a request to send the email is generated by the SMTP client, the PAN client connects to a PAN server in the Chicago office. The PAN server responds with a list of SMTP servers that are available to process the request, including an SMTP server in the Chicago office, and another in the company's Los Angeles office. The PAN client determines that, given the laptop computer system's present network connection, the SMTP server in the Los Angeles office will provide faster service. The PAN client negotiates encryption and compression techniques with the PAN server in the Los Angeles office to be used to encrypt and compress the data making up the SMTP application request. This data, encrypted and compressed in this manner, is sent from a laptop computer system to the PAN server in the Los Angeles office, where it is decrypted and decompressed, and passed to an agent for the SMTP server, which has an account with the SMTP server that enables it to connect to and authenticate with the SMTP server. The agent connects the SMTP server and relays the SMTP request, which is processed by the SMTP server, and for which an application response confirming the sending of the email is returned to the laptop computer system. [0038]
In the foregoing example, the PAN provided the following advantages: The PAN identified an application server that was best-suited to handle an application request from the client computer system. The agent used by the PAN server was able to connect to and authenticate with the application server in order to relay the request. The security of the data was ensured by the encryption of the application request by the PAN, and the request was transmitted more quickly because of the compression of the application request by the PAN. [0039]
FIG. 1 is a block diagram showing a typical environment in which the facility operates. A number of [0040] computer systems 120, 130, 140, 150, 160, and 170 are shown. Each of these computer systems may be any of a wide variety of computing devices, including desktop computer systems, dedicated server computer systems, mainframes, many-processor computational arrays, hand-held computers, corded and cordless telephones, pagers, digital organizers, etc. All of these computer systems are connected by a public network 100, such as the Internet, to which they either connect directly, or via intermediate networks, such as private networks, semi-public networks, other public networks, dial-up connections, etc. Additionally, computer systems 120 and 130 are connected by a private network 101, which may enable these computer systems to exchange data more quickly and/or more securely than they can via the public network 100. The networks used to connect these computer systems may utilize a wide variety of networking technologies, including wired, wireless, guided optical, line-of-sight optical, power network piggybacking, etc. These networks may pass traffic using a wide variety of different networking protocols.
The facility is employed to facilitate the use of distributed applications—such as client-server applications—across the network. Each client-server application includes both a client portion and a server portion, each of which may be installed on one or more computer systems. When the client portion is executing on a first computer system and needs assistance from the server portion, it communicates with the server portion executing on a different computer system. For example, the [0041] application B client 122 executing on computer system 120 may communicate with the application B server 131 executing on computer system 130. When the facility is used to facilitate communication for application B, the application B client 122 on computer system 120 issues a request that is received by the private application network client (“PAN client”) 126 on computer system 120. In some cases, this request is intercepted as local network traffic on computer system 120 by the special network driver (not shown), which mangles the packets of the network traffic in a way that causes them to be delivered to the PAN client. The PAN client 126 communicates with the private application network server (“PAN server”) 136 on computer system 130, which passes the request to the application B server 131 on computer system 130. The response from application B server 131 is received by the PAN server, which sends it back to the PAN client on computer system 120. The PAN client passes the response back to the application B client.
In some embodiments, PAN clients dynamically select applications servers to which to forward application requests in a manner that optimizes for such criteria as response time, cost, reliability, security, workload distribution among application servers, etc. In some embodiments, the facility sends application requests and responses via a private application network tunnel (“PAN tunnel”), within which data is passed that has been transformed in accordance with a negotiated set of processing techniques, including such processing techniques as compression and encryption algorithms. In some embodiments, application requests and/or responses for different applications may be transmitted together through a PAN tunnel. [0042]
It can be seen that a computer system may have more than one application client (e.g., [0043] computer system 120 has two application clients, for applications A and B) and that any computer system that has at least one application client has a PAN client. Similarly, it can be seen that some computer systems may have more than one application server (e.g., computer system 140 has two application servers, for applications A and B), and that each computer system that has at least one application server has a PAN server. Finally, it can be seen that some computer systems may have both application clients and application servers (e.g., computer system 150 has one of each, an application client for application C and an application server for application A), and that, in this case, the computer system has both a PAN client and a PAN server.
FIG. 2 is a block diagram showing architectural details of a typical private application network client. The [0044] PAN client 220 operates to obtain or intercept application requests issued by a number of different application clients 201-203. Generally, each of the application requests can be serviced by any server for the same application. For each obtain application request, the PAN client selects a server for the same application to which to send the application request, combines the application request with any other application requests currently being sent to the PAN server executing on the same computer system as the selected application server, transforms the combined application requests in a manner agreed upon for the private application network tunnel (“tunnel”) established with the PAN server on the remote computer system, and sends the combined application requests to the PAN server on the remote computer system. The PAN client performs these actions in reverse when it receives application responses from the PAN server on the remote computer system via the tunnel, ultimately passing each received application response to the appropriate application client.
An application agent is typically provided for each application client. For example, an agent for [0045] application A 221 is provided for the client for application A 201. Each agent is designed to interface with its application client to obtain or intercept application requests issued by the application client. Depending upon the design of a particular application client, this may involve such measures as: explicitly registering with the application client to receive its application requests; substituting its own virtual network address for a network address for an application server maintained by the application client; intercepting function calls by the application client to send application requests to an application server, or network traffic that results from such function calls; etc. In some cases, an application agent may be customized to coordinate its operation with a security scheme utilized by application clients and/or servers. For example, for application servers that require authentication of application clients submitting application requests, the corresponding application agent may be registered as application clients that are permitted to submit application requests to these application servers.
In each case, the agent uses an agent API to pass application requests received from application clients to a [0046] PAN core 210 within the PAN client. Use of this standardized agent API 220 enables new agents to be developed for new applications and incorporated within the PAN client, in order to adapt the PAN to exchange data for additional distributed applications. While a separate, customized agent is shown in FIG. 2 for each application client, agents may be allocated to application clients in a variety of other manners. As one example, two or more application clients can share the same agent. Further, one or more agents may be provided that are less customized to the design of the application clients with which they interact, but rather use some more standardized type of interface for interacting with such application clients.
When the PAN core receives application requests from application agents via the agent API or in network packets mangled by the special device driver, the PAN core determines whether a particular server for the corresponding application has already been selected by the PAN core. If not, the PAN core requests a list of eligible servers from a remote computer system that maintains such a list, from which the PAN core selects a particular server for this application. In some cases, the remote computer system exercises control over the PAN core's selection of an application server by returning only a subset of the servers for that application known to the remote computer system. Once the PAN core has identified the particular server for the corresponding application to which the application requests should be sent, the PAN core determines whether a tunnel is already open to the PAN server executing on the same computer system as that application server. If not, the PAN core establishes a tunnel with that PAN server, which involves negotiating with the remote PAN server about the types of transformations (such as compression transformations and/or encryption transformations) that are to be applied to data traveling through the new tunnel. Once a tunnel exists with the destination PAN server, the current application request is added to an [0047] outgoing buffer 240 for that PAN server. At that point, or a short time later, all of the application requests stored in the outgoing buffer for the PAN server by any of the application clients is combined and subjected to the set of transformations negotiated during the opening of the tunnel. These transformations are performed by invoking corresponding transformation modules 231-233 via a standardized API. Use of this API enables new transformation modules to be added to the facility, such as new transformation modules implementing new encryption algorithms or compression schemes. Once the application requests combined from the buffer are transformed, a network module 250 sends them via a network 290 to the destination PAN server.
When application responses are received in the network module from the network, their transformation(s) are reversed, and the individual application responses are passed to the corresponding application client via the corresponding agent. [0048]
FIG. 3 is a block diagram showing architectural details of a typical private application network server. It can be seen by comparing FIG. 3 to FIG. 2 that the architecture for a PAN server is very similar to that for a PAN client. Application requests received via a [0049] network 390 in a network module are untransformed using transformation modules 331-333 and separated into individual application requests, each of which is passed via the corresponding agent to the server for the corresponding application. When the server for the corresponding application issues an application response, it is intercepted by the agent for that application server and placed in the outgoing buffer for the corresponding PAN client. Application responses in the outgoing buffer for a PAN client are combined, subjected to the transformations negotiated for the tunnel with the PAN client, and sent to the PAN client by the network module 350 via the network 390.
FIG. 4 is a flow diagram showing steps typically performed by the facility in a PAN client to process a received application request. In [0050] step 401, the facility receives an application request from a client for an application via an agent for that application or via the special network device driver. As discussed above, the facility typically uses an agent API to communicate between the PAN core of the PAN client and the agent for each application. In step 402, if a server is already selected for this application, then the facility continues in step 408, else the facility continues in step 403.
In [0051] step 403, the facility requests a list of servers for the application, as well as associated selection information for each of the listed servers for the application. In some embodiments, each application server in the list is identified by information such as a network address for the computer system on which the application server is executing, which may include a port number, or other information for identifying the application server within the computer system on which it is executing. The accompanying selection information may include information such as whether the application server is known to be active; recent measurements of workload or response time for the application server; an indication of a fixed cost associated with using the application server; etc. In step 404, the facility itself collects additional selection information for some or all of the application servers on the requested list. Collecting such additional information may include contacting the computer system on which each application server is executing to make such determinations as the number of network hops required to reach the computer system on which the application server is executing; the round-trip time for sending a request and eliciting a response, either from the computer system or the application server itself; etc. In step 405, the facility uses the server selection information to select one of the listed servers for the application.
In [0052] step 406, if a tunnel is already open with the PAN server on the network node on which the selected server for the application is executing, then the facility continues in step 408, else the facility continues in step 407. In step 407, the facility opens a new PAN tunnel with the PAN server on the network node on which the selected server for the application is executing, by negotiating transformation options for the new tunnel with this PAN server. A variety of factors are typically considered in the negotiation of transformation options, including the typical or actual size of the requests and responses; the typical or actual level of sensitivity of the requests and the responses; and the set of processing modules installed in both the PAN client and PAN server. As one example, the PAN client and PAN server may negotiate that a particular compression algorithm will first be applied to data that is to pass through the tunnel, and then a particular encryption algorithm is to be applied. After step 407, the facility continues in step 408.
In [0053] step 408, the facility stores the application request received in step 401 in an outgoing buffer corresponding to the PAN server on the network node on which the selected server for the application is executing for subsequent transmission to that PAN server. After step 408, these steps conclude.
FIG. 5 is a flow diagram showing steps typically performed by the facility in order to transmit application requests accumulated for a particular PAN server in an outgoing buffer of the PAN client. In different embodiments, these steps are performed at different times, such as when a particular number of application requests have accumulated in the outgoing buffer, or a particular period of time after the first application request is stored in the outgoing buffer. [0054]
In [0055] step 501, the facility combines application requests in the outgoing buffer for a particular PAN server. In step 502, the facility uses one or more transformation modules to transform the application requests combined in step 801 in accordance with the transformation options negotiated for the tunnel that is open with the PAN server to which the application requests are to be transmitted. As noted above, the facility typically uses a standardized transformation module API in order to invoke the transformation modules needed to process the combined application requests. Where the transformation options that have been negotiated are such that first a particular compression algorithm will be applied to the data, then a particular encryption algorithm will be applied to the data, the facility in step 502 first invokes a transformation module corresponding to the compression algorithm in order to compress the combined application requests, then invokes a transformation module for the encryption algorithm in order to encrypt the compressed combined application requests. In step 503, the facility sends the application requests combined in step 501 and transformed in step 502 to the destination PAN server via the tunnel open with the PAN server. After step 503, these steps conclude.
FIG. 6 is a flow diagram showing steps typically performed by the facility to process application requests received by a PAN server. In [0056] step 601, the facility receives combined and transformed application requests from a particular PAN client via a tunnel open between that PAN client and the current PAN server. In step 602, the facility uses one or more transformation modules to reverse the transformation of the application requests received in step 601 in accordance with the transformation options negotiated for the tunnel, in an order opposite the one in which they were applied by the PAN client. For example, if it was negotiated that the tunnel would employ a particular compression algorithm followed by a particular encryption algorithm, then in step 602 the facility first invokes the encryption module corresponding to the negotiated encryption algorithm to reverse the encryption transformation of the application requests, then uses the compression transformation module corresponding to the negotiated compression algorithm to reverse the compression transformation of the application requests. In step 603, the facility separates the combined application requests following the reversal of their transformation in step 602. In steps 604-606, the facility loops through each application request among the application requests separated in step 603. In step 605, the facility passes the application request to the server for the corresponding application via the agent for that application. In step 606, if additional application requests remain to be processed, the facility continues in step 604 process the next application request. After step 606, these steps conclude.
FIG. 7 is a flow diagram showing steps typically performed by the facility in a PAN server to process application responses received from application servers. In [0057] step 701, the facility receives an application response from a server for a particular application via the agent for that application. In step 702, the facility stores the application response in the outgoing buffer for the PAN client from which the corresponding application request was received for subsequent transmission to that PAN client. After step 702, these steps conclude.
FIG. 8 is a flow diagram showing steps typically performed by the facility in order to transmit application responses accumulated in a PAN server to a destination PAN client. In different embodiments, these steps are performed at different times, such as when a particular number of application responses have accumulated in the outgoing buffer, or a particular period of time after the first application response is stored in the outgoing buffer. In [0058] step 801, the facility combines application responses in the outgoing buffer for a particular PAN client. In step 802, the facility uses one or more transformation modules to transform the application responses combined in step 801 in accordance with the transformation options negotiated for the tunnel open with the PAN client. In step 803, the facility sends the application responses combined in step 801 and transformed in step 802 to the PAN client via the tunnel that is open to the PAN client. After step 803, these steps conclude.
FIG. 9 is a flow diagram showing steps typically performed by the facility to process application responses received by a PAN client. In [0059] step 901, the facility receives transformed and combined application responses from a particular PAN server via the tunnel open with the PAN server. In step 902, the facility uses one or more transformation modules to reverse the transformation(s) of the combined application responses received in step 901 in accordance with the transformation options negotiated for the tunnel via which the combined application responses were received. In step 903, the facility separates the combined application responses. In steps 904-906, the facility loops through each received application response among the application responses separated in step 903. In step 905, the facility passes the application response to the client for the corresponding application via the agent for that application. In step 906, if additional application responses remain to be processed, the facility continues in step 904 to process the next application response. After step 906, these steps conclude.
FIG. 10 is a flow diagram showing steps typically performed by the facility to process a request from a PAN client to provide a list of servers for a particular application. These steps may be performed by the facility in a variety of computer systems, including within a PAN server, or in a dedicated server computer system called a PAN coordinating server. In [0060] step 1001, the facility receives from a requesting PAN client a request for a list of servers for a particular application. In step 1002, the facility retrieves a list of servers for that application. In step 1003, the facility optionally subsets the list of servers retrieved in step 1002 based upon the identity of the requesting PAN client. For example, the facility may omit to include in the subset application servers on the retrieved list that: would have poor response times if used by the requesting PAN client; are too expensive for use by the requesting PAN client; are for some reason unable to support application requests that may be received from the requesting PAN client; are known to be out of operation or overloaded; etc. In step 1004, the facility returns the subset of list of servers for the application generated in step 1003 to the requesting PAN client. In some embodiments, the returned list of application servers is accompanied by information usable by the PAN client to select a particular one of the application servers, as is discussed further above. After step 1004, these steps conclude.
In order to communicate with the PAN core using the agent API, each agent implements the following methods. Each agent also provides a name and a desired start priority. The Core starts agents in priority order, enabling the priority to be used to manage interagent dependencies. [0061]
AgentLoad( ): This function is used by the agent subsystem to load the agent. [0062]
AgentUnload( ): This function is used by the agent subsystem to unload the agent. [0063]
Init( ): The agent initialization method. Configuration is performed in this method. [0064]
Instantiate( ): This method instantiates the agent. All necessary resource allocation is performed in this method such as memory allocation and thread creation. The agent thread must block and wait for a notification from the start( ) method in order to start executing. [0065]
Start( ): This method starts the operation of the agent. If the agent implements threads, they need to be unblocked by this method. [0066]
Stop( ): Stop the operation of an agent. [0067]
Destroy( ): Free all resources that was claimed when the agent was instantiated. [0068]
A PAN client uses the agent API to interact with each agent as follows: The core calls the agent API to load, configure and start the agent. The series of calls in order are: [0069]
AgentLoad( ) [0070]
Init( ) [0071]
Instantiate( ) [0072]
Start( ) [0073]
When the core wants to terminate the service of the agent it will make the following calls in order: [0074]
Stop( ) [0075]
Destroy( ) [0076]
AgentUnload( ) [0077]
Once the agent is up and running it does what it needs to do such as binding a socket or sockets for inbound connections. Other connection types between the client agent and the client application are possible. The agent also opens a connection to the core invoking a PPQOpen( ) method on the core. [0078]
The client agent then waits for connections and data to arrive from the client application. When a connection arrives, the client agent sends an instruction across the tunnel that requests the server end to create a corresponding connection to the server application by invoking a PPQPost( ) method. [0079]
Data that arrives from the client application is passed to the core using the PPQPost( ) method. Data that arrives from the tunnel is retrieved by the agent using the PPQGet( ) method. This data is then passed to the client application through whatever connection mechanism the client agent has established. [0080]
A PAN client uses the agent API to interact with each agent in a similar manner, as follows: Each transformation module is a loadable module that gets loaded and linked at runtime. Each transformation module has a well known transformation ID that is a 32 bit integer. Each transformation can also maintain an internal state. There exists a peer to peer relationship between the transmit transformation module (on the HyperTunnel client (or server)) and receive transformation module (on the HyperTunnel server (or client)). This allows a transmitting transformation to add its own packet headers, i.e., inject its own communication protocol, onto the data stream and can thus communicate with the receiving transformation that peels off the header. [0081]
A set of methods comprising the transformation API are defined in a table for each supported transformation module. A linked list of such tables is created at startup. [0082]
tx_fn( ): This is the transformation module's transmit method. [0083]
rx_fn( ): This is the transformation module's receive method. [0084]
init( ): This is a one time initialization method that is called by the transformation subsystem once at startup. [0085]
instantiate( ): This method is called to allocate transformation resources such as state each time the transformation module is pushed onto a HyperChannel. [0086]
uninstantiate( ): This method releases resources the transformation module previously has allocated in the instantiate( ) method. [0087]
The core calls the agent API to load, configure and start the agent. The series of calls in order are: [0088]
AgentLoad( ) [0089]
Init( ) [0090]
Instantiate( ) [0091]
Start( ) [0092]
When the core wants to terminate the service of the agent it will make the following calls in order: [0093]
Stop( ) [0094]
Destroy( ) [0095]
AgentUnload( ) [0096]
Once the agent is up and running it does what it needs to do such as making one or more outbound connections. It may also choose to do this at a later point in time such as when data has actually arrived from the tunnel. Again, many different connection types between the server agent and the server application are possible. The agent also opens a connection to the core by invoking the PPQOpen( ) method on the core. [0097]
When connection instructions arrive, the server agent will make or verify the pre-existence of a connection to the server application. [0098]
When data arrives over the tunnel, the server agent will obtain this data from the core using the PPQGet( ) method. This data is then passed to the server application via the connection established between the server agent and the server application. [0099]
Data may also arrive from the server application. This data is passed to the core using the PPQPost( ) method. [0100]
The transformations are implemented via a data structure which contains memory addresses of its init, instantiate, uninstantiate, transmit and receive methods. The core creates an instance of a transformation for each channel by allocation a transformation data structure. [0101]
The core then initializes the transformation by calling its init( ) method. The core then instantiates the transformation by calling its instantiate( ) method. Data is passed to the transformation via its tx( ) method which returns the transformed data. Data is received from the transformation via its rx( ) method which reverses the transformation and returns the untransformed data. [0102]
The methods of a given transformation are invoked by the core of the tunnel. There may be several transformations that are in an ordered list. The type and number of transformations may vary between channels that are open. Each tunnel client shares a unique channel with a tunnel server. The list of transformations to be used is negotiated between the tunnel client and tunnel server during channel establishment. Data is passed to the first transformation in the list by its tx( ) method. The transformed data is then passed to the next transformation via its tx( ) method. Once all of the transformations have been processed, the core can transport the data across the tunnel. Data received from the tunnel is passed through the transformation list using the rx( ) methods. Once all of the transformations are reversed, the data is passed to the agents. [0103]
In some embodiments, a special network device driver is used by the facility to capture for transmission via the PAN network traffic sent by application clients and addressed to corresponding application servers. In some embodiments, the special network device driver used by the facility is interposed on top of the standard network device driver, and activated to intercept packets bound for the standard network device driver only at times while the PAN tunnel is active to process diverted traffic, enabling application clients to make direct connections to application servers using the standard network device driver during times when the PAN or the PAN client are inactive. [0104]
FIG. 11 is a flow diagram showing steps typically performed by the facility in the special network driver to redirect application network traffic to a PAN tunnel. In some embodiments, the special network driver is installed in the operating system as a service driver, so that it receives network packets before they reach the standard network driver, which is installed in the operating system as an adapter. In some embodiments, however, these steps, as well as steps in the flow diagrams that follow, are performed by the facility in a variety of different contexts. As one example, in some embodiments, the facility performs the identification and mangling of these packets using an iptables function provided in the netfilter module of the Linux operating system. [0105]
In [0106] step 1101, the facility receives a network packet in transit on the local computer system. Table 1 below shows the header information for a sample packet received in step 1101, including its destination address, destination port, source address and source port. This packet is one sent from an application client executing on a computer system having the IP address 52.166.23.34, to an application server listening on port 80 of a computer system having the IP address 15.0.32.1. In some cases, in addition to the header information shown below on Table 1, the network packet received in step 1101 has a protocol header field, having a value such as TCP or UDP. Because the packet is addressed to an application server computer system having an IP address that is different from the IP address of the local computer system, and that is known to be routable, the packet is received by the network driver.

TABLE 1

Sample Outbound Packet Original Header

PACKET FIELD FIELD VALUE

destination address 15.0.32.1

destination port 80

source address 52.166.23.34

source port 1001
In [0107] step 1102, if the packet received in step 1101 is a SYN packet initiating a new TCP connection, then the facility continues in step 1103, else the facility continues in step 1104. In step 1103, the facility applies a rule table to the packet received in step 1101. Such application is discussed in greater detail below in conjunction with FIG. 12. After step 1103, the facility continues in step 1105.
In [0108] step 1104, the facility applies a mapping table to the packet received in step 1101. Such application is discussed in greater detail below in conjunction with FIG. 23.
In [0109] step 1105, if the packet was mangled in step 1103 or step 1104, then the facility continues in step 1106, else the facility continues in step 1107. In step 1106, the facility recalculates one or more checksums for the packet's header so that they accurately reflect the mangled contents of the header. In step 1107, the facility releases the packet. After step 1107, the execution continues in step 1101 to receive the next packet.
FIG. 12 is a flow diagram showing steps typically performed by the facility in order to apply a rule table used by the facility to a packet in [0110] step 1103. In steps 1201-1205, the facility loops through each entry in the rule table used by the facility.
FIG. 13 is a data structure diagram showing a sample rule table typically used by the facility in an initial state. The rule table [0111] 1300 is comprised of entries, such as entry 1301, each corresponding to a rule for recognizing application packets outbound from particular application clients. While rule table 1300 contains only one entry, those skilled in the art will recognize that, where the facility is used to support the exchange of data by a large number of distributed applications, the rule table may have a much larger number of entries.
Each rule table entry is divided into four columns, or fields: a [0112] destination address 1311 and destination port 1312 to which packets sent by a client of a particular application will be sent, and a mapped destination address 1313 and mapped destination port 1314 to which such packets will be redirected. In some embodiments, each rule table entry includes an additional column (not shown), which can contain the protocol with which packet sent by a client of a particular application will be sent. For a group of commonly-managed client computer systems, such as those operated by a single corporation, the clients for a particular application, such as electronic mail, will on every client computer system be configured to use the same application server computer system. Accordingly, for a rule corresponding to the email application, the IP address for this application server is included as the destination address for the rule. A rule table entry can, but need not, contain a destination port for that destination address. In some cases where the servers for a particular application listen on more than a single destination port, this field is empty. The mapped destination address, where included, is typically either the loop-back IP address of the local computer system on which the special network driver is executing or the external IP address of the local computer system, as the goal is to redirect selected packets to the PAN client executing on the same computer system as the application clients that send the packets. In some cases, the loop-back IP address of the local computer system is selected as the mapped destination address in order to provide more efficient delivery of the mangled packet to the PAN client. Ultimately, the mapped destination address can be any IP address on which the PAN client is capable of listening. In some cases, the rule table entry contains a mapped destination port at which the PAN client is listening to receive packets for the application to which the rule corresponds. In other cases, however, and particularly in those where no destination port is specified for the rule, no mapped destination port is specified for the rule. As discussed below, for rules containing no mapped destination port, the facility in the device driver typically requests a mapped destination port from the PAN client.
In some embodiments, rather than specifying a single destination address or destination port, a rule may specify a range of destination addresses and/or a range of destination ports. Such ranges may be expressed using a variety of different techniques, such as by specifying the top and bottom values of the range, or by specifying a single value together with a mask for transforming the single value into the range. In this way, a rule may match more than one destination address, and/or more than one destination port. [0113]
Returning to FIG. 12, in [0114] step 1202, if the packet matches the current entry of the rule table, then the facility continues in step 1203, else the facility continues in step 1205. A packet matches an entry of the rule table if the addressing fields in its header correspond to the fields of the rule table entry as shown below in Table 2.

TABLE 2

Rule Match

PACKET FIELD RULE TABLE ENTRY FIELD

destination address destination address

destination port destination port (if specified)

protocol protocol (if specified)
Returning to FIG. 13, it can be seen that, in the example, the sample packet whose header is shown in Table 1 matches [0115] rule table entry 1301, as the destination address is the same in both, and no destination port or protocol is specified in the rule table entry.
In [0116] step 1203, if the matching current entry contains a destination port, then the facility continues in step 1208, else the facility continues in step 1204. In step 1204, the facility flags the entry for possible future processing if the packet does not end up matching any rule table entries containing a destination port. In step 1205, if the rule table contains additional entries to examine, then the facility continues in step 1201 to examine the next entry, else the facility continues in step 1206. In step 1206, if any entry in the rule table was flagged in step 1204, then the facility continues in step 1207, else these steps conclude without mangling the packet. In step 1207, the facility generates a new entry in the rule table by copying the contents of the flagged entry to the new entry, and storing the destination port from the packet in the destination port field of the new entry.
FIG. 14 is a data structure diagram showing the sample rule table of FIG. 13 in an updated state. In [0117] step 1207, the facility has generated a new rule table entry 1302 from existing rule table entry 1301. In the new rule table entry 1302, the destination address and mapped destination address fields have been copied from rule table entry 1301. The destination port has been copied from the destination port of the sample packet shown in Table 1.
In [0118] step 1208, the facility creates a new entry in the mapping table. The facility copies data from the rule table entry into the new mapping table entry. The facility further copies the source address and source port from the packet into the new mapping table entry.
FIG. 15 is a data structure diagram showing a sample mapping table typically used by the facility in an initial state. The mapping table [0119] 1500 is divided into eight columns-an original source address column 1511, an original source port column 1512, an original destination address column 1513, an original destination port column 1514, a mapped destination address column 1515, a mapped destination port column 1516, a forward FIN flag column 1517, and a backward FIN flag column 1518. The mapping table contains one or more entries each corresponding to a connection in progress between an application client executing on the local computer system and an application server contacted via a PAN tunnel. The entry contains information for both identifying network packets moving in either direction that are part of this connection, and for mangling them in the appropriate direction. Mangling in the forward direction is performed on packets sent by an application client on the local computer system—also called request packets or outbound packets, while mangling in the reverse direction is performed on packets sent by an application server on a remote computer system—also called response packets or inbound packets. The entry further contains information for tracking the state of the connection and removing the entry when the connection is completed.
The mapping table [0120] 1500 contains the new mapping table entry 1501 created in step 1208. The original source address 1511 and original source port 1512 of the mapping table entry are copied from the source address and source port of the packet whose header is shown in Table 1. The original destination address 1513, original destination port 1514, and mapped destination address 1515 are copied from the destination address, destination port, and mapped destination address of rule table entry 1302. Entry 1501 does not yet contain a mapped destination port. Also, the two FIN flags 1517 and 1518 are initially not set, indicating that no FIN packet has yet been received in either direction on the application connection to which the mapping table entry corresponds. In some embodiments, the facility uses a different FIN tracking mechanism, such as a single FIN count field for each mapping table entry.
In [0121] step 1209, in each mapping table entry if the rule table entry already contains a mapped destination port, then the facility continues in step 1212, else the facility continues in step 1210. In the example, rule table entry 1402, does not yet contain a mapped destination port and the facility continues in step 1210. In step 1210, the facility contacts the PAN client to obtain a mapped destination port for the current rule table entry in the newly created mapping table entry. In cases in which the rule table entry does not yet contain a mapped destination address, step 1210 further includes obtaining a mapped destination address from the PAN client for the current rule table entry. In one embodiment, the driver contacts the PAN client in step 1210 using control messages, such as control messages passed across a Windows I/O channel maintained between the driver and the tunnel client, or control messages transmitted via UDP packets. In some embodiments, step 1208 is not performed until after step 1210 is performed, delaying creation of the new mapping table entry until the mapped destination port and mapped destination address are both known. In step 1211, the facility copies the mapped destination port obtained from the PAN client in step 1210 into both the current rule table entry, and the new mapping table entry created in step 1208.
FIG. 16 is a data structure diagram showing the sample rule table of FIG. 14 in an updated state. It can be seen in mapping table [0122] 1600 that the mapped destination port value 5000 has been added to the mapped destination port field 1614 of rule table entry 1602.
FIG. 17 is a data structure diagram showing the sample mapping table shown in FIG. 15 in an updated state. It can be seen in rule table [0123] 1700 that the mapped destination port value 5000 has been added to the mapped destination port field 1716 of mapping table entry 1701.
In [0124] step 1212, the facility mangles the packet in the forward direction in accordance with the new mapping table entry created in step 1208, using the correspondence shown below in Table 3.

TABLE 3

Forward Mangling From Mapping

PACKET FIELD VALUE, FROM MAPPING TABLE ENTRY

destination address mapped destination address

source address original destination address

destination port mapped destination port
The mangled packet produced by mangling the original packet shown in Table 1 in accordance with [0125] mapping table entry 1701 is shown below in Table 4. After step 1212, these steps conclude.

TABLE 4

Sample Outbound Packet Mangled Header

PACKET FIELD FIELD VALUE

destination address 52.166.23.34

destination port 5000

source address 15.0.32.1

source port 1001
Because the mangled packet has its source address copied from the original packet's destination address and its source port that is copied from the original packet's source port, there is no actual network destination for a reply to this packet to be delivered to. Rather, the only possible fate for such a reply packet is being received and mangled in the reverse direction by the special network driver. [0126]
FIG. 18 is a flow diagram showing steps typically performed by the facility to respond to a mapped destination port request made by the driver in [0127] step 1210. These steps are typically performed by the facility in the PAN client executing on the same computer system as the driver. In step 1801, the facility receives a mapping request from the driver. The mapping request specifies an original destination port and destination port number of a packet that the driver is attempting to divert and create a new mapping for.
In [0128] step 1802, the facility creates a new TCP socket, here referred to as the parent socket. The parent socket is typically created using a socket( ) function. TCP/IP sockets and their use are well known to those skilled in the art. Donahoo, Michael J. and Calvert, Kenneth L., The Pocket Guide to TCP/IP Sockets, C Version, Morgan Kauffman Publishers, 2001, describes TCP/IP sockets and their use, and is hereby incorporated by reference in its entirety. In particular, the implementation in the Microsoft Windows operating system of functions provided for using sockets are described at http://www.msdn.microsoft.com/library/enus/winsock/windows_sockets_functions_—2.asp and associated web pages, which are hereby incorporated by reference in their entirety. Where a UDP packet is being mangled, the facility creates a new UDP socket in step 1802 rather than a TCP socket, and proceeds to use the UDP socket in place of the TCP socket in the steps that follow.
In [0129] step 1803, the facility binds the parent socket created in step 1802 to an IP address for the local computer system and a free port—that is, a port that is not currently being used—with that IP address. In one embodiment, the facility binds the parent socket to an IP address and a port using a bind( ) function. The facility either itself tracks a set of free ports, an IP address and specifies one of these explicitly in its call to the bind( ) function, or calls the bind( ) function without specifying a port number, and permits the bind( ) function to itself select a free port. In this case, the facility typically also calls a getsockname( ) function to determine the port number to which the parent socket has been bound.
In [0130] step 1804, the facility uses the specified destination address, and/or the specified destination port, to identify the application to which the packet being mapped corresponds, as, for many applications, one or both of these values is uniquely characteristic of the application. In step 1805, the facility selects a server for the application identified in step 1804 to which to direct the current packet, and future packets from the same application client that are addressed similarly. Such selection is described in greater detail above in conjunction with FIG. 4.
In [0131] step 1806, if a PAN tunnel channel is already open to the server selected in step 1805, then the facility continues in step 1808, else the facility continues in step 1807. In step 1807, the facility opens a PAN tunnel channel to the server selected in step 1805. In some embodiments, the facility also adds an entry to a PAN tunnel channel table identifying the PAN tunnel channel opened in step 1807.
FIG. 19 is a data structure diagram showing a sample PAN tunnel channel table typically used by the facility. The PAN tunnel channel table [0132] 1900 is comprised of entries such as entry 1901. Each entry is divided into a local PAN tunnel channel identifier 1911, a remote PAN tunnel channel identifier 1912, and a remote address 1913. The local PAN tunnel channel identifier contains a value that uniquely identifies the PAN tunnel channel on the local computer system. The remote PAN tunnel channel identifier is a value that uniquely identifies the PAN tunnel channel to the PAN server at the other end of the PAN tunnel—that is, the PAN server on the same computer system as the server selected in step 1805. The remote address is the IP address of this remote computer system.
In [0133] step 1808, in a new thread, the facility executes an accept loop for the parent socket. In the accept loop, the facility uses the PAN tunnel channel identifier for the PAN tunnel channel to the selected server, as well as an application identifier for the identified application. The performance of step 1808 is discussed in greater detail below in conjunction with FIG. 20. In some embodiments, the facility performs step 1808 without spawning a new thread.
In [0134] step 1809, the facility sends to the driver a response to the mapping request received in step 1801. The response specifies the local IP address and port number to which the parent socket was bound as the mapped destination address, mapped port number, enabling the driver to complete its rule and mapping table entries and mangle the current packet for receipt by the PAN client on the port number to which the parent socket was bound. After step 1809, the facility continues in step 1801 to receive the next mapping request from the driver.
FIG. 20 is a flow diagram showing steps typically performed by the facility to execute an accept loop for the parent socket in accordance with [0135] step 1808. These steps are typically performed in the PAN client. In step 2001, the facility accepts a new connection on the parent socket using a child socket. In some embodiments, step 2001 comprises calling an accept( ) function with a descriptor identifying the parent socket. The accept( ) function returns a descriptor identifying a child socket from which packets sent to the local computer system's IP address and the port number to which the parent socket was bound from a particular source address and port can be read. In step 2002, the facility adds a row to a PAN socket table used by the facility. The new row added in step 2002 contains the descriptor for the child socket, the local PAN tunnel channel identifier, and an application identifier identifying the application identified in step 1804. In step 2003, in a new thread, the facility reads packets from the child socket and directs these packets to the PAN tunnel channel having the specified tunnel connection identifier. In the example, the mangled packet shown in Table 4 is read from the child socket in step 2003 and directed to the PAN tunnel channel having tunnel channel identifier 53. In some embodiments, the facility performs step 2003 without spawning a new thread.
After [0136] step 2003, the facility continues in step 2001 to accept a new connection on the parent socket using a new child socket. Such a new connection can occur when the same application client sends a packet containing a different source port value.
FIG. 21 is a data structure diagram showing a sample PAN socket table typically used by the facility. The PAN socket table [0137] 2100 is comprised of entries, such as entry 2101. Each entry corresponds to a particular child socket, and is divided into a socket descriptor 2111, a PAN tunnel channel identifier 2112, and an application identifier 2113. The contents of the PAN socket table are used to determine, for data received via a particular PAN tunnel channel, which child socket the data must be written to in order to direct the data to the appropriate application client.
FIG. 22 is a flow diagram showing steps typically performed by the facility in order to route data received from an application server via a particular PAN tunnel channel. These steps are typically performed in the PAN client. In [0138] step 2201, the facility receives data via a PAN tunnel channel having an identified PAN tunnel channel identifier. The facility can use this identified PAN tunnel channel identifier to look up the corresponding application identifier in the PAN tunnel channel table. In step 2202, the facility identifies a row of the PAN socket table containing this PAN tunnel channel identifier and application identifier. In step 2203, the facility reads the socket descriptor from the identified row of the PAN socket table. In step 2204, the facility writes the data received in step 2201 to the socket having the socket descriptor read from the identified row in step 2203. Writing the data in step 2204 has the effect of generating a packet whose addressing information is inverted from the packet originally received on this child socket. In the example, where the packet originally received on this child socket was the forward-mangled packet whose addressing information is shown in Table 4, the packet generated by writing to this child port contains the addressing information shown below in Table 5.

TABLE 5

Sample Inbound Packet Original Header

PACKET FIELD FIELD VALUE

destination address 15.0.32.1

destination port 1001

source address 52.166.23.34

source port 5000
It can be seen by comparing the contents of Tables 4 and 5 that the source address in Table 5 is the same as the destination address in Table 4; the source port in Table 5 is the same as the destination port in Table 4; the destination address in Table 5 is the same as the source address in Table 4; and the destination port in Table 5 is the same as the source port in Table 4. When the packet whose addressing information is shown in Table 5 is sent as the result of writing to the child socket in [0139] step 2204 because the destination address is a routable address different from the address of the local computer system, the packet is intercepted by the device driver, which performs the steps shown in FIG. 11, and ultimately those shown in FIG. 23.

FIG. 23 is a flow diagram showing steps typically performed by the facility in order to apply the facility's mapping table to a network packet in

step

1104. In steps 2301-2312, the facility loops through each entry in a mapping table used by the facility. In step 2302, if the packet matches the current entry in the mapping table in the forward direction, then the facility continues in step 2303, else the facility continues in step 2306. A packet matches an entry in the forward direction if the addressing fields in its header correspond to the fields of the mapping table entry as shown below in Table 6.

TABLE 6


Forward Mapping Match

	PACKET FIELD	MAPPING TABLE ENTRY FIELD

	destination address	original destination address
	destination port	original destination port
	source address	original source address
	source port	original source port

Because the response packet shown in Table 5 does not match the only entry in mapping table [0141] 1700 shown in FIG. 17, the packet does not match any entries in the mapping table in a forward direction in the example. In step 2303, the facility mangles the packet in the forward direction in accordance with the matching mapping table entry. The facility performs step 2303 by modifying addressing fields of the packet's header as shown above in Table 3. In step 2304, if the packet is a FIN packet denoting one party's ending of the TCP connection, then the facility continues in step 2305, else these steps conclude. In step 2305, the facility sets the forward FIN flag in the current mapping table entry. After step 2305, the facility continues in step 2310.

In

step

2306, if the packet matches the current mapping table entry in the backward direction, then the facility continues in step 2307, else the facility continues in step 2312. The packet matches the entry in the backward direction if the addressing fields of the packet's header correspond to the fields of the mapping table entry as shown below in Table 7.

TABLE 7


Backward Mapping Match

	PACKET FIELD	MAPPING TABLE ENTRY FIELD

	destination address	original destination address
	destination port	original source port
	source address	mapped destination address
	source port	mapped destination port

In [0143] step 2307, the facility mangles the packet in the backward direction in accordance with the current mapping table entry. The facility performs step 2307 by modifying the addressing fields of the packet's header as shown below in Table 8.

TABLE 8

Backward Mangling From Mapping

PACKET FIELD VALUE, FROM MAPPING TABLE ENTRY

destination address original source address

source address original destination address

source port original destination port
Because the response packet shown in Table 5 [0144] matches entry 1701 in mapping table 1700, the facility mangles the response packet in the reverse direction, producing a mangled packet having the addressing information shown below in Table 9.

TABLE 9

Sample Inbound Packet Mangled Header

PACKET FIELD FIELD VALUE

destination address 52.166.23.34

destination port 1001

source address 15.0.32.1

source port 80
It can be seen by comparing Table 9 to Table 1 both that (1) the packet shown in Table 9 will be delivered to the application client sending the message in Table 1 (as the destination address and destination port in Table 9 match the source address and source port in Table 1) and that (2) the application client will regard the packet in Table 9 as a response to the packet in Table 1 (as the source address and source port in Table 9 are the same as the destination address and destination port in Table 1). Accordingly, traffic from the application client to the application server and back has been effectively redirected in a manner that is completely transparent to the application client, and does not require any modification of the application client or its configuration. [0145]
In [0146] step 2308, if the packet is a FIN packet, then the facility continues in step 2309, else these steps conclude. In step 2309, the facility sets the backward FIN flag in the current mapping table entry.
In [0147] step 2310, if both of the FIN flags in the current mapping table entry are set, then the facility continues in step 2311, else these steps conclude. In step 2311, the facility removes the current mapping table entry from the mapping table, as the present packet is the last packet of the application connection to which the mapping table entry corresponds. After step 2311, the steps conclude.
In [0148] step 2312, if additional entries remain in the mapping table to be tested, the facility continues in step 2301 to test the next entry in the matching table, else these steps conclude without mangling the packet.
If a future outbound packet is sent by the application client having the same destination address, destination port, source address, and source port as the sample packet shown in Table 1 at a time when the mapping table still contains [0149] entry 1701, the facility will use mapping table entry 1701 to forward-mangle this packet, and to reverse-mangle this packet's response from the application server.
If an outbound packet is received having the same destination address, destination port, and source address as the sample packet, but a different source port, the facility does not find a matching mapping table entry, but does match [0150] rule table entry 1602. Accordingly, the facility uses rule table entry 1602 to create a new mapping table entry containing the same information and the source address and source port from the current packet (not shown). The facility uses this new mapping table entry to mangle the current packet in the forward direction, and to mangle its response in the backward direction. Because the new mapping table entry contains the same mapped destination port as mapping table entry 1701, the PAN client receives the forward-mangled packet at the original parent socket described above. However, because the source port of this mangled packet is different than the source port of the original forward-mangled packet shown in Table 7, the packet is accepted by the parent socket using a new child socket. Within the PAN client, this new child socket will distinguish traffic resulting from packets sent by the application client using the new source port from that resulting from packets sent by the application client using the old source port.
When an outbound packet is received having the same destination address as the sample packet but a different destination port, the packet is not matched in the mapping table, and matches [0151] rule table entry 1601. As a result, the facility creates a new rule table entry (not shown) and a new mapping table entry (not shown). The new mapping table entry will have a different mapped destination port obtained from the PAN client. When the new mapped destination port is used to mangle the outbound packet in the forward direction, it will be received by the PAN client at a new parent socket corresponding to the new mapped destination port.
Where an outgoing or incoming packet is received by the driver that matches neither a mapping table entry nor a rule table entry, the facility will permit this packet to pass without mangling it. [0152]
In some embodiments, entries are also removed from the mapping table when a RESET packet is received from either the application client or the application server, or when no packets have matched the mapping table entry for at least a predetermined amount of time. [0153]
In some embodiments, both the forward and backward mangling processes also include exchanging the source and destination addresses of the physical network layer—that is, the NIC layer. [0154]
In some embodiments, the network traffic diversion techniques described above are applied to selectively divert network traffic for a variety of other purposes. Examples include various types of application request filtering, such as parental controls for filtering requests for applications that are inappropriate for particular groups of users, such as young users; and systems that block trojan horses; that is, prevent programs surreptitiously installed on a computer system from transmitting information from that computer system to remote computer systems. As those skilled in the art will appreciate, this aspect of the facility is amenable to a wide variety of applications. [0155]
It will be appreciated by those skilled in the art that the above-described facility may be straightforwardly adapted or extended in various ways. For example, the facility may be used in networks of virtually any type, as well as in heterogeneous networks that employ multiple different networking technologies. Also, in addition to public networks, the facility may be used in semi-public and private networks. Further, the facility may use a variety of techniques to intercept and present application requests and responses. Additionally, the facility may be used to pass data on behalf of distributed applications that exchange data between network nodes that are of a type other than client-server applications, such as peer-to-peer applications. Also, a variety of different techniques may be used to identify and mangle network packets, including alternatives to the rules and mappings described above. Further, these techniques may be performed by code residing in various places and executed at various points in the flow of network traffic. Additionally, many of the techniques used by the facility are easily adapted to other networking protocols. While the foregoing description makes reference to preferred embodiments, the scope of the invention is defined solely by the claims that follow and the elements recited therein. [0156]

Claims

We claim:

1. A method in a client computing system for conveying network traffic for a plurality of distributed applications, each distributed application comprising a client portion executing on the client computer system and a server portion executing on a separate server computing system, the method comprising:

activating a private application network client for exchanging network traffic for the distributed application with the server computing system;

in response to activation of the private application network client, activating a distinguished network driver;

in the distinguished network driver:

receiving a first network packet generated by one of the client portions;

comparing the header of the first network packet to a plurality of interception rules to identify a distributed application to which the first network packet corresponds;

contacting a tunnel client to obtain a mapped port number to which to forward the first network packet;

generating a mapping for the first network packet containing information from the first network packet's header and the obtained mapped port number;

using the generated mapping to mangle the first network packet for delivery to the tunnel client at the obtained mapped port number;

in the tunnel client:

receiving the mangled first network packet at a selected socket listening on the mapped port number;

forwarding the contents of the mangled first network packet via a selected tunnel channel to a selected server portion of the identified distributed application;

receiving response data from the selected server portion of the identified distributed application via the selected tunnel channel;

writing the receiving response data to the selected socket;

in the distinguished network driver;

receiving a second network packet generated by writing the receiving response data to the selected socket;

comparing the generated mapping to the header of the second network packet to recognize the second networkpacket as being related to the first network packet; and

using the generated mapping to mangle the second network packet for delivery to the source of the first network packet.

2. The method of claim 1, further comprising:

in the distinguished network driver:

receiving a third network packet generated by one of the client portions;

comparing the generated mapping to the header of the third network packet to recognize the third network packet as being related to the first network packet; and

using the generated mapping to mangle the third network packet for delivery to the tunnel client at the mapped port number contained by the generated mapping.

3. A method in a first network node upon which a client for a selected application is executing, comprising, in a network driver:

capturing a packet sent by the client for the selected application and addressed to a second network node that is distinct from the first network node, a server for the selected application executing upon the second network node; and

passing the captured packet to a data tunneling program executing on the first network node.

4. The method of claim 3 further comprising, in the data tunneling program, transmitting data in the passed packet to the second network node for delivery to the server for the selected application executing on the second network node.

5. The method of claim 3 further comprising, in the data tunneling program, transmitting data in the passed packet to a third network node for delivery to a server for the selected application executing on the third network node, the third network node being distinct from the first and second network nodes.

6. The method of claim 3 wherein the captured packet has a header containing delivery information, and wherein passing the captured packet to the data tunneling program comprises:

modifying the delivery information contained by the header of the captured packet; and

permitting the captured packet to be delivered to the data tunneling program in accordance with the modified delivery information as local network traffic.

7. A first network node upon which a client for a selected application is executing, comprising:

a packet capture subsystem that captures a packet sent by the client for the selected application and addressed to a second network node that is distinct from the first network node, a server for the selected application executing upon the second network node; and

a packet passing subsystem that passes the captured packet to a data tunneling program executing on the first network node.

8. A computer-readable medium whose contents cause a first network node upon which a client for a selected application is executing to selectively redirect network traffic by, in a network driver:

passing contents of the captured packet to a data tunneling program executing on the first network node.

9. A method in a computing system for diverting a network packet to a diverted destination, comprising:

selecting for diversion a first network packet that has been submitted for delivery and whose delivery is not yet complete, the first network packet having a destination address having an initial value, a destination port having an initial value, a source address having an initial value, and a source port having an initial value;

in the first network packet, substituting the initial value of the destination address in the source address;

in the first network packet, substituting an address for the diverted destination in the destination address;

in the first network packet, substituting a port for the diverted destination in the destination port; and

after substitution within the first network packet is completed, releasing the first network packet for delivery to the diverted destination.

10. The method of claim 9, wherein the first network packet has a checksum covering its destination address, destination port, source address, and source port, the checksum having an initial value, the method further comprising:

recalculating the checksum based on the substituted values; and

before the first network packet is released, substituting the recalculated checksum for the checksum's initial value.

11. The method of claim 9, further comprising:

selecting a second network packet as a reply to the first network packet, the second network packet having a destination address having an initial value equal to the initial value of the first packet's destination address, a destination port having an initial value equal to the initial value of the first packet's source port, a source address having an initial value equal to the address for the diverted destination, and a source port having an initial value equal to the port for the diverted destination;

in the second network packet, substituting the initial value of the first packet's source address in the destination address;

in the second network packet, substituting the initial value of the first packet's destination address in the source address;

in the second network packet, substituting the initial value of the first packet's destination port in the source port; and

after substitution within the second network packet is completed, releasing the second network packet for delivery to the source of the first packet.

12. The method of claim 11, further comprising, contemporaneous to releasing the first network packet, storing the initial value of the first network packet's destination address, the initial value of the first network packet's destination port, the initial value of the first network packet's source address, the initial value of the first network packet's source port, the address for the diverted destination, and the port for the diverted destination,

and wherein selecting the second network packet comprises matching the initial value of the second network packet's destination address to the stored initial value of the first packet's destination address, matching the initial value of the second network packet's destination port to the stored initial value of the first packet's source port, matching the initial value of the second network packet's source address to the stored address for the diverted destination, and matching the initial value of the second network packet's source port to the stored port for the diverted destination,

and wherein it is the stored initial value of the first packet's source address that is substituted in the second network packet's destination address;

and wherein it is the stored initial value of the first packet's destination address that is substituted in the second network packet's source address;

and wherein it is the stored initial value of the first packet's destination port that is substituted in the second network packet's source port.

13. The method of claim 12, further comprising:

selecting a third network packet that has been submitted for delivery and whose delivery is not yet complete, the third network packet having a destination address having an initial value, a destination port having an initial value, a source address having an initial value, and a source port having an initial value, based upon determining that:

the initial value of the third network packet's destination address matches the stored initial value of the first packet's destination address,

the initial value of the third network packet's destination port matches the stored initial value of the first packet's destination port,

the initial value of the third network packet's source address matches the stored initial value of the first packet's source address, and

the initial value of the third network packet's source port matches the stored initial value of the first packet's source port;

in response to selection of the third network packet, in the third network packet:

substituting the stored address for the diverted destination in the destination address,

substituting the stored port for the diverted destination in the destination port; and

substituting stored initial value of the first packet's destination address in the source address, and

after substitution within the third network packet is completed, releasing the third network packet for delivery to the diverted destination.

14. The method of claim 9, further comprising accessing a plurality of rules, each rule specifying a destination address,

wherein the first network packet is selected for diversion based upon the initial value of the first packet's destination address matching the destination address specified by a distinguished one of the rules.

15. The method of claim 9, further comprising accessing a plurality of rules, each rule specifying a destination address and a destination port,

wherein the first network packet is selected for diversion based upon (1) the initial value of the first network packet's destination address matching the destination address specified by a distinguished one of the rules, and (2) the initial value of the first network packet's destination port matching the destination port specified by the distinguished rule.

16. The method of claim 15, wherein the first network packet has a protocol having an initial value, and wherein each of the plurality of accessed rules further specifies a protocol,

and wherein the first network packet is selected for diversion further based upon (3) the initial value of the first packet's protocol matching the protocol specified by the distinguished rule.

17. The method of claim 9, further comprising accessing a plurality of rules, each rule specifying a destination address, a mapped destination address, and a mapped destination port,

wherein the first network packet is selected for diversion based upon the initial value of the first packet's destination address matching the destination address specified by a distinguished one of the rules,

and wherein the address for the diverted destination substituted in the destination address of the first network packet is the mapped destination address specified by the distinguished rule,

and wherein the port for the diverted destination substituted in the destination port of the first network packet is the mapped destination port specified by the distinguished rule.

18. The method of claim 9 wherein the method is performed in a network device driver interposed before an original network device driver within local packet flow.

19. The method of claim 18 wherein the modified network device driver is activated to intercept packets bound for the original network device driver only while the diverted destination is available to receive diverted network packets.

20. The method of claim 9 wherein an operating system is executing on the computing system,

and wherein the method is performed using a network address translation facility provided by the operating system.

21. The method of claim 9 wherein the method is performed in a first network node,

and wherein the diverted first network packet is received at a data tunneling program listening to the address and port for the diverted destination for transmission to a second network node distinct from the first network node.

22. The method of claim 9 wherein the first network packet is received from an application client,

and wherein the diverted first network packet is received at a data tunneling program listening to the address and port for the diverted destination, the data tunneling program performing application routing for network packets received from the application client.

23. The method of claim 9 wherein the first network packet is received from an application client and contains an application request,

and wherein the diverted first network packet is received at a request filtering program listening to the address and port for the diverted destination, the request filtering program blocking network packets containing certain types of application requests.

24. The method of claim 23 wherein the request filtering program blocks network packets containing application requests not appropriate for one or more classes of users.

25. The method of claim 23 wherein the request filtering program blocks network packets containing application requests generated by a trojan horse program.

26. A computer-readable medium whose contents cause a computing system to divert a network packet to a diverted destination by:

in the first network packet, substituting the initial value of the destination port in the source port;

27. A computing system for diverting a network packet to a diverted destination, comprising:

a packet selection subsystem that selects for diversion a first network packet that has been submitted for delivery and whose delivery is not yet complete, the first network packet having a destination address having an initial value, a destination port having an initial value, a source address having an initial value, and a source port having an initial value;

a substitution subsystem that substitutes, in the first network packet:

the initial value of the destination port in the source port,

an address for the diverted destination in the destination address, and

a port for the diverted destination in the destination port; and

a packet release subsystem that releases the first network packet for delivery to the diverted destination after substitution within the first network packet is completed.

28. A method in a computing system for generating a rule collection data structure, comprising:

for each of a plurality of distributed applications:

creating a rule;

storing in the created rule information characterizing the header information of request packets sent by clients of the distributed application.

29. The method of claim 28, further comprising:

receiving a network packet; and

using the rules to identify a distributed application of whose request packets the header information of the received network packet is characteristic.

30. The method of claim 28, further comprising:

for at least a subset of the distributed applications, storing in the created rule information specifying a manner for forwarding request packets sent by clients of the distributed application to a distributed application request packet router.

31. The method of claim 30, further comprising:

receiving a network packet;

selecting a rule whose rule information characterizing the header information of request packets sent by clients of the distributed application matches the received network packet; and

forwarding the received network packet to a distributed application request packet router in accordance with the information stored in the selected rule specifying a manner for forwarding request packets sent by clients of the distributed application to a distributed application request packet router.

32. The method of claim 28, further comprising:

receiving a network packet;

using the rules to identify a distributed application of whose request packets the header information of the received network packet is characteristic;

contacting a distributed application request packet router to obtain a specification of a manner for forwarding to the distributed application request packet router request packets sent by clients of the identified distributed application; and

forwarding the received network packet to the distributed application request packet router in accordance with the obtained specification of a manner for forwarding to the distributed application request packet router request packets sent by clients of the identified distributed application.

33. The method of claim 32, further comprising:

storing the obtained specification of a manner for forwarding to the distributed application request packet router request packets sent by clients of the identified distributed application in the rule created for the identified distributed application.

34. A computer-readable medium whose contents cause a computing system to generate a rule collection data structure by:

for each of a plurality of distributed applications:

creating a rule;

35. One or more computer memories collectively containing a mapping data structure, the mapping data structure corresponding to a network packet earlier mangled in a forward direction, and comprising:

a destination address corresponding to a destination address of the network packet, with which the source address of the network packet was replaced during mangling;

a destination port corresponding to a destination port of the network packet;

a source address corresponding to a source address of the network packet;

a source port corresponding a source port of the network packet;

a mapped destination address with which the destination address of the network packet was replaced during mangling; and

a mapped destination port with which the destination port of the network packet was replaced during mangling,

such that the contents of the mapping data structure may be used to mangle a response to the earlier-mangled network packet in a backward direction,

and such that the contents of the mapping data structure may be used to mangle in a forward direction a subsequent network packet having the same destination address, destination port, source address, and source port as the earlier-mangled network packet.

36. The computer memories of claim 35 containing a plurality of a mapping data structures each corresponding to a different network packet earlier mangled in a forward direction.

37. One or more computer memories collectively containing a rule collection data structure, the data structure comprising a plurality of entries, each entry corresponding to a particular distributed application and comprising:

(1) information characterizing the header information of request packets sent by clients of the distributed application; and

(2) either

(a) information specifying a manner for forwarding request packets sent by clients of the distributed application to a distributed application request packet router, or

(b) an indication that information specifying a manner for forwarding request packets sent by clients of the distributed application to a distributed application request packet router should be obtained from a distributed application request packet router,

such that the contents of the data structure can be use to identify and forward to a distributed application request packet router request packets sent by clients of a plurality of distributed applications.

38. The computer memories of claim 37 wherein the indication is explicit.

39. The computer memories of claim 37 wherein the indication is implicit.

40. The computer memories of claim 37 wherein the information characterizing the header information of request packets sent by clients of the distributed application comprises a destination address.

41. The computer memories of claim 37 wherein the information characterizing the header information of request packets sent by clients of the distributed application comprises a range of destination addresses.

42. The computer memories of claim 37 wherein the information characterizing the header information of request packets sent by clients of the distributed application comprises a destination port.

43. The computer memories of claim 37 wherein the information characterizing the header information of request packets sent by clients of the distributed application comprises a range of destination ports.

44. The computer memories of claim 37 wherein the information characterizing the header information of request packets sent by clients of the distributed application comprises a destination address and a destination port.

45. The computer memories of claim 37 wherein the information specifying a manner for forwarding request packets sent by clients of the distributed application to a distributed application request packet router comprises a mapped destination address and mapped destination port to which to re-address the forwarding request packets sent by clients of the distributed application.

46. One or more generated data signals collectively conveying a mangled network packet data structure generated from an original network packet data structure, the mangled network packet data structure comprising:

a destination address field containing a value corresponding to an address at which a diversion target program is listening;

a destination port field containing a value corresponding to a port on which the diversion target program is listening;

a source address field containing a value corresponding to a destination address field of the original network packet data structure; and

a source port field containing a value corresponding to a source port field of the original network packet data structure,

such that the mangled network packet data structure may be received by the diversion target program and its correspondence to the original network packet data structure discerned.

47. One or more generated data signals collectively conveying a second network packet data structure generated from a first network packet data structure, the second network packet data structure comprising:

a destination address field containing a first value retrieved from a network address translation mapping matched by the first network packet data structure;

a destination port field containing a value corresponding to a destination port field of the first network packet data structure;

a source address field containing a value corresponding to a destination address field of the first network packet data structure; and

a source port field containing a second value retrieved from the network address translation mapping matched by the first network packet data structure,

such that the mangled network packet data structure may be received by a program that formerly sent a third network packet data structure comprising:

a destination address field containing a value corresponding to the source address field of the second network packet data structure;

a destination port field containing a value corresponding to the source port field of the second network packet data structure;

a source address field containing a value corresponding to the destination address field of the second network packet data structure; and

a source port field containing a value corresponding to the destination port field of the second network packet data structure.