US20070220183A1 - Receive Queue Descriptor Pool - Google Patents
Receive Queue Descriptor Pool Download PDFInfo
- Publication number
- US20070220183A1 US20070220183A1 US11/749,761 US74976107A US2007220183A1 US 20070220183 A1 US20070220183 A1 US 20070220183A1 US 74976107 A US74976107 A US 74976107A US 2007220183 A1 US2007220183 A1 US 2007220183A1
- Authority
- US
- United States
- Prior art keywords
- descriptors
- pool
- message
- data
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9047—Buffering arrangements including multiple buffers, e.g. buffer pools
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/901—Buffering arrangements using storage descriptor, e.g. read or write pointers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9031—Wraparound memory, e.g. overrun or underrun detection
Definitions
- the present invention relates generally to digital network communications, and specifically to network adapters for interfacing between a computing device and a packet data network.
- I/O serial input/output
- the computer industry is moving toward fast, packetized, serial input/output (I/O) bus architectures, in which computing hosts and peripherals are linked by a switching network, commonly referred to as a switching fabric.
- IB InfiniBandTM
- the IB architecture is described in detail in the InfiniBand Architecture Specification, Release 1.0 (October, 2000), which is incorporated herein by reference. This document is available from the InfiniBand Trade Association at www.infinibandta.org.
- Computing devices connect to the IB fabric via a network interface adapter, which is referred to in IB parlance as a channel adapter.
- a network interface adapter which is referred to in IB parlance as a channel adapter.
- the IS specification defines both a host channel adapter (HCA) for connecting a host processor to the fabric, and a target channel adapter (TCA), intended mainly for connecting peripheral devices to the fabric.
- the channel adapter is implemented as a single chip, with connections to the computing device and to the network.
- Client processes (referred to hereinafter as clients) running on a host processor communicate with the transport layer of the IB fabric by manipulating a transport service instance, known as a “queue pair” (QP), made up of a send work queue and a receive work queue.
- QP transport service instance
- the IB specification permits the HCA to allocate as many as 16 million (2 24 ) QPs, each with a distinct queue pair number (QPN). A given client may open and use multiple Q
- each WR has a data buffer associated with it, to be used for holding the data that is sent or received in executing the WQE.
- Each QP has its own WQE chain and associated data buffers. Each WQE in the chain and the buffer associated with it are passed to the control of the HCA when the WQE is posted. The HCA then executes the WQEs, so as to communicate with the corresponding QP of the channel adapter at the other end of the link.
- the HCA After it has finished servicing a WQE, the HCA typically writes a completion queue element (CQE) to a completion queue, to be read by the client.
- CQE completion queue element
- the QP that initiates a particular operation i.e. injects a message into the fabric
- the QP that receives the message is referred to as the responder.
- An IB operation is defined to include a request message generated by the requester and, as appropriate, its corresponding response generated by the responder. (Not all request messages have responses.)
- Each message consists of one or more IB packets.
- a given HCA will serve simultaneously both as a requester, transmitting requests and receiving responses on behalf of local clients, and as a responder, receiving requests from other channel adapters and returning responses accordingly.
- Each QP is configured for a certain transport service type, which determines how the requesting and responding QPs interact. Both the source and destination QPs must be configured for the same service type.
- the IB specification defines four service types: reliable connection, unreliable connection, reliable datagram and unreliable datagram.
- IB request messages include, inter alia, remote direct memory access (RDMA) write and send requests, RDMA read requests, and atomic read-modify-write requests.
- RDMA write and send requests carry data sent by the requester and cause the responder to write the data to a memory address at its own end of the link.
- RDMA write requests specify the address in the remote responder's memory to which the data are to be written
- send requests rely on the responder to determine the memory location at the request destination.
- the send operation is sometimes referred to as a “push” operation, since the initiator of the data transfer pushes data to the remote QP.
- the receiving node's channel adapter places the data into the next available receive buffer for that QP.
- the send operation is also referred to as having channel semantics, because it moves data much like a mainframe I/O channel: each packet of data is tagged with a discriminator, and the destination processor chooses where to place the data based on the discriminator.
- the discriminator is the destination address (i.e., the local identifier, or LID) of the receiving channel adapter and the destination QP number.
- a client on the host computing device must generate receive WQEs and place them in the receive queues of the appropriate QPs.
- the destination channel adapter takes the next WQE from the receive queue and places the received data in the memory location specified in that WQE.
- the destination channel adapter can receive and process incoming send packets on a given QP only when there is an appropriate WQE waiting to be read from the receive queue of the QP.
- the host computing device must prepare and hold in memory at least one receive WQE for every QP that is configured to receive send messages.
- the channel adapter cannot process the packet and must therefore discard it.
- the channel adapter returns a “Receiver Not Ready” (RNR) NACK packet to the requester. The requester may then retry the send request after a suitable waiting period has passed.
- RNR Receiveiver Not Ready
- the IB specification provides a flow control mechanism for send messages using reliable connection services, based on end-to-end credits.
- a requester cannot sent a request message unless it has the appropriate credits to do so.
- These credits are passed to the requester by the responder, wherein each credit represents the resources needed by the responder to receive one inbound request message. Specifically, each credit represents one WQE posted to the receive queue of the responding QP.
- a network adapter is configured to receive incoming messages over a network, containing data to be pushed to local memory locations that are to be determined by the receiving network adapter.
- the messages are carried over the network using multiple different transport service instances. At least some of the multiple transport service instances share a common pool of receive descriptors, indicating the buffers in the local memory to which the data are to be pushed.
- the adapter receives a message on any of these transport service instances that contains data to be pushed to the local memory, it reads the next descriptor from the pool and writes the data to the indicated location.
- the shared pool of descriptors thus obviates the need to maintain a separate queue of receive descriptors for each transport service instance.
- the network comprises an IS switch fabric.
- the network adapter is a host channel adapter (HCA)
- the messages pushing data to the adapter are send messages
- the transport service instances correspond to QPs serviced by the HCA
- the descriptors correspond to WQEs that would ordinarily be placed in the receive queues of the QPs. Sharing a pool of descriptors among multiple QPs, in place of maintaining an individual receive queue for each QP, is not an option that is provided by the IB specification.
- This novel technique allows channel adapters to support a large number of QPs without allocating excessive amounts of memory to hold WQEs on all the receive queues. It also reduces the burden imposed on the host processor to monitor all the receive queues and keep them filled with descriptors. Instead, the host processor need deal with only a single pool of descriptors that can serve many QPs.
- the methods of memory allocation provided by the present invention are conducive to balanced communication system design.
- host applications running in conjunction with a channel adapter should, on average, consume data at the rate the data arrives from the network. This rate depends on the speed of the network, and not on the number of connections (QPs) served. Therefore, even when the host is serving many connections at once, the total amount of memory that must be allocated by the host to hold incoming data from the channel adapter should depend mainly (or exclusively) on the expected fluctuations in the rates of data arrival and processing.
- the IB specification is wasteful of memory, as it requires memory allocation to scale up with the number of open connections.
- the present invention provides an alternative solution that maintains balance and efficiency in the use of memory resources.
- a method for network communication including:
- each of the descriptors in the pool including a scatter list indicating a buffer that is available in a local memory
- providing the pool of descriptors includes writing the descriptors to the local memory using a host processor, and receiving the message includes receiving the message at a network adapter, which reads the one of the descriptors and writes the data to the buffer for processing by the host processor.
- the network includes a switch fabric, and the network adapter includes a channel adapter.
- writing the data includes, upon completion of writing the data to the buffer, submitting a completion report from the channel adapter to the host processor, the completion report including a pointer to the scatter list for use by the host processor in processing the data.
- providing the pool of descriptors includes providing multiple pools of descriptors, each belonging to a respective group of the transport service instances, and reading the one of the descriptors includes determining the group to which the one of the transport service instances belongs, and reading the one of the descriptors from the pool belonging to the group.
- providing the pool of descriptors includes writing the descriptors to a circular list having a head and a tail, and reading the one of the descriptors includes selecting the one of the descriptors from the head of the list.
- selecting the one of the descriptors includes comparing the head and the tail of the list, and abstaining from writing the data to the buffer in the local memory if the head and tail coincide.
- receiving the message includes receiving one or more packets having payloads including the data.
- the network includes a switch fabric, and the transport service instances includes queue pairs, and the message includes a send message conveyed over the fabric by a remote requester.
- each of the descriptors corresponds to a work queue element in a receive queue of one of the queue pairs.
- a network adapter including:
- a network interface adapted to receive a message over a network on one of a plurality of transport service instances, the message including data to be pushed to a local memory
- a host interface adapted to be coupled to a host processor and to the local memory associated therewith;
- message processing circuitry coupled between the network interface and the host interface, which circuitry is adapted, responsive to receiving the message at the network interface, to read a descriptor from a pool of descriptors that is shared among the plurality of transport service instances, each of the descriptors in the pool including a scatter list indicating a buffer that is available in the local memory, and to write the data included in the message via the host interface to the buffer indicated by the scatter list included in the descriptor.
- the descriptors in the pool are prepared in the local memory by the host processor, and the message processing circuitry is adapted to read the descriptors from the pool via the host interface.
- FIG. 1 is a block diagram that schematically illustrates a network communication system, in accordance with a preferred embodiment of the present invention
- FIG. 2 is a block diagram that schematically shows details of a host network adapter, in accordance with a preferred embodiment of the present invention
- FIG. 3 is a block diagram that schematically illustrates data structures used by a network adapter in processing incoming messages, in accordance with a preferred embodiment of the present invention.
- FIG. 4 is a flow chart that schematically illustrates a method for processing an incoming message, in accordance with a preferred embodiment of the present invention.
- FIG. 1 is a block diagram that schematically illustrates an IB network communication system 20 , in accordance with a preferred embodiment of the present invention.
- a HCA 22 couples a host processor 24 to an IB network (or fabric) 26 .
- processor 24 comprises an Intel PentiumTM processor or other general-purpose computing device with suitable software.
- HCA 22 typically communicates via network 26 with other HCAs, such as a remote HCA 28 with a remote host 30 , as well as with target channel adapters (TCAs), such as a TCA 32 connected to an input/output (I/O) device 34 .
- TCAs target channel adapters
- Host 24 and HCA 22 are connected to a local system memory 38 via a suitable memory controller 36 , as is known in the art.
- the HCA and memory typically occupy certain ranges of physical addresses in a defined address space on a bus connected to the controller, such as a Peripheral Component Interface (PCI) bus.
- PCI Peripheral Component Interface
- memory 38 holds data structures that are accessed and used by HCA 22 . These data structures preferably include QP context information 42 maintained by the HCA, and descriptors 44 corresponding to WQEs to be carried out by HCA 22 .
- memory 38 is shown in FIG.
- system memory as used in the present patent application and in the claims, should be understood broadly to encompass all areas of local memory that can be accessed by host 24 .
- Descriptors 44 are written to memory 38 by client processes running on host 24 . They include send descriptors, corresponding to outgoing request messages to be sent over fabric 26 by HCA 22 , and receive descriptors, used by the HCA to handle incoming send messages from remote requesters, such as HCA 28 .
- the send descriptors are placed in the appropriate send queues of QPs for service by HCA 22 , and are not of direct relevance to the present invention. At least a portion of the receive descriptors, however, are not placed directly in the receive queues of individual QPs. Rather, they are held in a descriptor pool, as described below, which is shared among multiple QPs.
- Each of the receive descriptors contains a scatter list, comprising one or more scatter entries, each indicating a range of addresses in memory 38 to which HCA 22 should write the data contained in the send message.
- each scatter entry includes a base address and a length of the data to be written beginning at the base address.
- FIG. 2 is a block diagram that schematically shows details of HCA 22 , in accordance with a preferred embodiment of the present invention.
- HCA 22 that are not essential to an understanding of the present invention are omitted.
- the blocks and links that must be added to create a fully-operational RCA will be apparent to those skilled in the art. Further details of such a HCA are described in U.S. patent application Ser. No. 10/000,456, filed Dec. 4, 2001, which is assigned to the assignee of the present patent application, and whose disclosure is incorporated herein by reference.
- HCA 22 may be implemented either as hardware circuits or as software processes running on a programmable processor, or as a combination of hardware- and software-implemented elements. Although certain functional elements of HCA 22 are shown as separate blocks in the figure for the sake of conceptual clarity, the functions represented by these blocks may actually be carried out by different software processes on a single embedded processor. Preferably, all of the elements of the HCA are implemented in a single integrated circuit chip, but multi-chip implementations are also within the scope of the present invention.
- Incoming packets from fabric 26 are received by HCA 22 at an input port 50 , which serves as a network interface.
- a transport check unit (TCU) 52 processes and verifies transport-layer information contained in the incoming packets, in order to confirm the validity of the packets and to determine how they are to be handled. For this purpose, the TCU reads the destination QP of each incoming packet, and then looks up the relevant context information 42 for the QP.
- a cache 54 holds a copy of at least a portion of the context information that is required by the elements of HCA 22 for processing incoming and outgoing packets on active QPs. If the TCU does not find the required context information in cache 54 , the information is loaded into the cache from memory 38 .
- TCU 52 passes the packets to a receive data engine (RDE) 56 , which attends to executing the requests.
- RDE receive data engine
- a write engine 58 in RDE 56 writes the packet data, via a translation protection table (TPT) 58 , to addresses in memory 38 that are specified in the applicable scatter list.
- TPT 58 acts as a host interface in HCA 22 , performing address translation and protection checks to control access to memory 38 both by elements of HCA 22 and by other, remote entities connected to network 26 .
- Each RDMA write message carries its own scatter list, which is prepared by the remote requester.
- write engine 58 To process incoming send messages, however, write engine 58 must read a receive descriptor from memory 38 , and use the scatter list provided by the descriptor. Processing of incoming send messages by RDE 56 is described in greater detail hereinbelow. After processing of an incoming send message has been completed (and likewise, processing of other types of messages, when required), a completion reporter 60 writes a CQE to a completion queue in memory 38 . Write engine 58 and completion reporter 60 also use QP and completion queue context information that is held in cache 54 .
- the QP can be set up by host 24 either to have its own queue of receive descriptors, as prescribed by the IB specification, or to share descriptors in a pool with other QPs. Most preferably, multiple pools of this sort are supported by HCA 22 .
- the configuration of the QP as a pool member is preferably indicated by a flag in QP context 42 , as well as by a field in the context identifying the pool to which the QP belongs.
- FIG. 3 is a block diagram that schematically illustrates data structures 70 maintained in memory 38 , which are used by RDE 56 in processing incoming send messages, in accordance with a preferred embodiment of the present invention. These structures are built around a descriptor pool 71 , which serves a group of QPs. Other QPs may have their own, individual receive queues (not shown in the figures), as provided by the IB specification. Optionally, for more efficient operation of RDE 56 , some or all of the data in structures 70 are copied to cache 54 , as well.
- Host 24 writes descriptors 78 , or WQEs, to each open descriptor pool 71 .
- the host writes sequences of multiple descriptors without waiting for the pool to empty.
- the descriptors are preferably arranged in a cyclic buffer.
- When the host writes a descriptor to the buffer it moves a producer index (PI) so that it points to the tail of the pool, i.e., to the last descriptor it has created.
- a consumer index (CI) points to the head of the queue and is manipulated by RDE 56 to indicate the next descriptor to be read from the pool.
- the producer and consumer indices are recorded and maintained relative to a predetermined base address. The use of these pointers is described in greater detail hereinbelow.
- RDE 56 Upon receiving a send message on a given QP, and determining that the QP belongs to a receive descriptor pool, RDE 56 reads a pool number 72 for the QP from QP context information 42 (which is preferably held in cache 54 ).
- the pool number serves as an index (relative to a predetermined base address) to an entry 76 in a descriptor pool table (DPT) 74 .
- Entry 76 contains information for use by the RDE in finding descriptors 78 to read from the descriptor pool 71 that is assigned to this QP. The same entry 76 is used for all the QPs belonging to the same pool.
- Each entry 76 in DPT 74 preferably includes the following information:
- Start address base address of pool 71 in memory 38 .
- the size determines the length of the scatter lists that can be used.
- Total size of the pool i.e., the maximum number of descriptors that the pool can hold.
- the producer or consumer index reaches this value, it wraps back to the base address.
- Producer index address memory location to which host 24 writes and updates the value of the producer index of pool 71 .
- Each descriptor 78 comprises one or more scatter entries, each indicating a buffer in memory 38 to which write engine 58 should write the data contained in an incoming send message.
- each scatter entry includes a base address and a length of the data that can be written beginning at the base address.
- descriptor 78 may include other fields used for control and signaling to HCA 22 .
- the structure of descriptors 78 in pool 71 is preferably the same as that of the descriptors that are placed in the receive queues of QPs that are not pool members.
- FIG. 4 is a flow chart that schematically illustrates a method by which HCA 22 processes incoming send request messages, in accordance with a preferred embodiment of the present invention.
- the method is initiated when TCU 52 receives a send packet from a remote requester via fabric 26 , at a send reception step 80 . After completing the required transport checks, the TCU passes the packet to RDE 56 for service.
- IB send messages may comprise multiple packets, depending on the volume of data carried by the message and the maximum transfer unit (MTU) of the links over which the message travels.
- MTU maximum transfer unit
- Write engine 58 determines the destination QP of the send packet, based on the packet header, and then looks up the context of the QP in cache 54 , at a pool membership checking step 82 . As noted above, the context indicates whether or not this QP belongs to a descriptor pool. If the QP is not a pool member, then in order to receive a send message, there must be a WQE available in the specific receive queue of this QP. The write engine reads the WQE address from the QP context and then fetches the WQE from the receive queue, at a descriptor fetching step 84 . It then processes the send message in the usual way, as provided by the IB specification,
- write engine 58 determines at step 82 that the destination QP does, in fact, belong to a receive descriptor pool, it reads the number of the pool from the QP context, at a pool number reading step 86 . It uses this number to find the information necessary to access descriptor pool 71 to which this QP belongs, at an information lookup step 88 . This information is typically contained in entry 76 in table 74 ( FIG. 3 ), which is indexed by pool number 72 . Additionally, in order to access descriptor pool 71 in memory 38 , the write engine may need an access key, as is known in the art. This key is typically held in the QP context, and is preferably the same for all QPs belonging to the pool.
- write engine 58 uses the information in entry 76 to read the consumer index (CI) and producer index (PI) of descriptor pool 71 , at an index checking step 90 . If the values of these indices are equal, it means that all descriptors 78 in pool 71 have already been used. Without a valid descriptor, the write engine is unable to process the current send packet. Under these circumstances, the send packet is typically discarded. If the send packet was sent on a reliable service, write engine 58 instructs a send data engine (not shown) in HCA 22 to return a RNR NACK packet to the sender, at a NACK return step 92 . The sender may subsequently resend the packet.
- CI consumer index
- PI producer index
- the write engine preferably triggers an event, at an event submission step 94 , which is placed in an event queue to be read by host 24 .
- an interrupt may be generated, as well, to prompt the host to service the event queue.
- the host reads the event, it will be alerted to the fact that descriptors 78 in pool 71 have been exhausted. The host software should then generate new descriptors to replenish the pool.
- write engine 58 reads descriptor 78 from the head of the circular buffer in pool 71 , at the location indicated by the CI, at a descriptor reading step 96 . It increments the CI to point to the next descriptor in the pool, at an index incrementation step 98 .
- the write engine then uses the scatter list provided by the descriptor it has read in processing the send packet data, at a packet processing step 100 . To perform this processing, the write engine reads the first scatter entry from the scatter list in descriptor 78 , which points to the first buffer to receive the data in memory 38 . The write engine pushes the data from the packet to this first buffer, until the buffer is filled.
- write engine 58 Upon completion of an incoming send message, write engine 58 instructs completion reporter 60 to generate a completion queue element (CQE), at a CQE generation step 102 .
- the completion reporter places the CQE in a completion queue in memory 38 , to be read by client software on host 24 .
- an event or interrupt may also be generated to notify the host that there are new data in memory 38 waiting to be read.
- the CQE indicates the QP on which the incoming send message was received and includes a pointer to the descriptor 78 in pool 71 that was used in processing the message that has now been completed.
- Host 24 reads the scatter list from the descriptor in order to determine the location of the data to be read in memory 38 . Once the host has read the data, the descriptor is no longer needed and can be overwritten by a new descriptor.
- each credit represents one WQE posted to the receive queue of the responding QP.
- a QP that draws its WQEs from a shared descriptor pool has no WQEs posted to its receive queue. Instead, these QPs may send credits to the corresponding requester based on the number of descriptors 78 posted to pool 71 (preferably a smaller number of credits on each QP than there are actual descriptors in the pool). As long as an appropriate statistical relationship is maintained between the number of credits and the number of descriptors in the pool, there will usually be a descriptor available to handle each send message that arrives.
- the requester may still transmit send packets in limited mode, as described in section 9.7.7.2.5 of the IB specification.
- the principles of the present invention may similarly be applied to handling of data “push” operations and message transfers using channel semantics in networks of other types.
- the methods described hereinabove can be used in protocol bridge applications, in which multiple connections on a first network are served by a single sink to a second network.
- multiple hosts on the first network for instance, on an IB fabric
- a converter that channels their traffic to the second network (such as an Ethernet network).
Abstract
A method for network communication includes providing a pool of descriptors to be shared among a plurality of transport service instances used in communicating over a network, each of the descriptors in the pool including a scatter list indicating a buffer that is available in a local memory. When a message containing data to be pushed to the local memory is received over the network on one of transport service instances, one of the descriptors is read from the pool. The data contained in the message are written to the buffer indicated by the scatter list included in the one of the descriptors.
Description
- This is a Continuation of U.S. patent application Ser. No. 10/200,189 filed Jul. 23, 2003, currently pending.
- The present invention relates generally to digital network communications, and specifically to network adapters for interfacing between a computing device and a packet data network.
- The computer industry is moving toward fast, packetized, serial input/output (I/O) bus architectures, in which computing hosts and peripherals are linked by a switching network, commonly referred to as a switching fabric. A number of architectures of this type have been proposed, culminating in the “InfiniBand™” (IB) architecture, which has been advanced by a consortium led by a group of industry leaders (including Intel, Sun Microsystems, Hewlett Packard, IBM, Compaq, Dell and Microsoft). The IB architecture is described in detail in the InfiniBand Architecture Specification, Release 1.0 (October, 2000), which is incorporated herein by reference. This document is available from the InfiniBand Trade Association at www.infinibandta.org.
- Computing devices (hosts or peripherals) connect to the IB fabric via a network interface adapter, which is referred to in IB parlance as a channel adapter. The IS specification defines both a host channel adapter (HCA) for connecting a host processor to the fabric, and a target channel adapter (TCA), intended mainly for connecting peripheral devices to the fabric. Typically, the channel adapter is implemented as a single chip, with connections to the computing device and to the network. Client processes (referred to hereinafter as clients) running on a host processor communicate with the transport layer of the IB fabric by manipulating a transport service instance, known as a “queue pair” (QP), made up of a send work queue and a receive work queue. The IB specification permits the HCA to allocate as many as 16 million (224) QPs, each with a distinct queue pair number (QPN). A given client may open and use multiple QPs simultaneously.
- To send and receive messages over the network using a ECA, the client initiates work requests (WRS), which cause work items, called work queue elements (WQEs) , to be placed onto the appropriate work queues. Normally, each WR has a data buffer associated with it, to be used for holding the data that is sent or received in executing the WQE. Each QP has its own WQE chain and associated data buffers. Each WQE in the chain and the buffer associated with it are passed to the control of the HCA when the WQE is posted. The HCA then executes the WQEs, so as to communicate with the corresponding QP of the channel adapter at the other end of the link. After it has finished servicing a WQE, the HCA typically writes a completion queue element (CQE) to a completion queue, to be read by the client. The buffer associated with the WQE is freed for use by the client only after the CQE is generated.
- The QP that initiates a particular operation, i.e. injects a message into the fabric, is referred to as the requester, while the QP that receives the message is referred to as the responder. An IB operation is defined to include a request message generated by the requester and, as appropriate, its corresponding response generated by the responder. (Not all request messages have responses.) Each message consists of one or more IB packets. Typically, a given HCA will serve simultaneously both as a requester, transmitting requests and receiving responses on behalf of local clients, and as a responder, receiving requests from other channel adapters and returning responses accordingly. Each QP is configured for a certain transport service type, which determines how the requesting and responding QPs interact. Both the source and destination QPs must be configured for the same service type. The IB specification defines four service types: reliable connection, unreliable connection, reliable datagram and unreliable datagram.
- IB request messages include, inter alia, remote direct memory access (RDMA) write and send requests, RDMA read requests, and atomic read-modify-write requests. Both RDMA write and send requests carry data sent by the requester and cause the responder to write the data to a memory address at its own end of the link. Whereas RDMA write requests specify the address in the remote responder's memory to which the data are to be written, send requests rely on the responder to determine the memory location at the request destination. The send operation is sometimes referred to as a “push” operation, since the initiator of the data transfer pushes data to the remote QP. The receiving node's channel adapter places the data into the next available receive buffer for that QP. The send operation is also referred to as having channel semantics, because it moves data much like a mainframe I/O channel: each packet of data is tagged with a discriminator, and the destination processor chooses where to place the data based on the discriminator. In the case of IB send packets, the discriminator is the destination address (i.e., the local identifier, or LID) of the receiving channel adapter and the destination QP number.
- To specify the receive buffers to use for incoming send requests received by a channel adapter, a client on the host computing device must generate receive WQEs and place them in the receive queues of the appropriate QPs. Each time a valid send request is received, the destination channel adapter takes the next WQE from the receive queue and places the received data in the memory location specified in that WQE. Thus, every valid incoming send request engenders a receive queue operation by the responder.
- It follows from this paradigm of send message handling that the destination channel adapter can receive and process incoming send packets on a given QP only when there is an appropriate WQE waiting to be read from the receive queue of the QP. To meet this requirement, the host computing device must prepare and hold in memory at least one receive WQE for every QP that is configured to receive send messages. When an incoming send packet arrives at the destination channel adapter on a given QP, and there is no receive WQE available, the channel adapter cannot process the packet and must therefore discard it. In the case of reliable services, when there is no receive WQE on hand, the channel adapter returns a “Receiver Not Ready” (RNR) NACK packet to the requester. The requester may then retry the send request after a suitable waiting period has passed.
- To avoid this situation, the IB specification provides a flow control mechanism for send messages using reliable connection services, based on end-to-end credits. As a rule, a requester cannot sent a request message unless it has the appropriate credits to do so. These credits are passed to the requester by the responder, wherein each credit represents the resources needed by the responder to receive one inbound request message. Specifically, each credit represents one WQE posted to the receive queue of the responding QP.
- Given the large number of QPs (up to 16 million) that can be in use at any one time, the need to keep a WQE available in every receive queue can consume a great deal of memory. Practically speaking, it is much more efficient for both the host computing device and the channel adapter to create and maintain several WQEs in the receive queue at any given time, thus increasing even further the memory and computing resources needed for each QP. It can be seen that a prohibitive amount of memory is thus required if a large complement of QPs is to be supported, as provided by the IB specification.
- It is an object of some aspects of the present invention to provide methods for receive queue management that enable efficient handling of incoming send messages without restricting the number of QPs that a channel adapter can support, and without consuming excessive amounts of memory. It is a further object of these aspects of the present invention to provide channel adapters that implement such methods.
- It is a further object of some aspects of the present invention to provide improved methods for allocating buffers to receive data pushed over a network to a destination device using multiple transport service instances.
- In preferred embodiments of the present invention, a network adapter is configured to receive incoming messages over a network, containing data to be pushed to local memory locations that are to be determined by the receiving network adapter. The messages are carried over the network using multiple different transport service instances. At least some of the multiple transport service instances share a common pool of receive descriptors, indicating the buffers in the local memory to which the data are to be pushed. When the adapter receives a message on any of these transport service instances that contains data to be pushed to the local memory, it reads the next descriptor from the pool and writes the data to the indicated location. The shared pool of descriptors thus obviates the need to maintain a separate queue of receive descriptors for each transport service instance.
- In some preferred embodiments of the present invention, the network comprises an IS switch fabric. In these embodiments, the network adapter is a host channel adapter (HCA), the messages pushing data to the adapter are send messages, the transport service instances correspond to QPs serviced by the HCA, and the descriptors correspond to WQEs that would ordinarily be placed in the receive queues of the QPs. Sharing a pool of descriptors among multiple QPs, in place of maintaining an individual receive queue for each QP, is not an option that is provided by the IB specification. This novel technique, however, allows channel adapters to support a large number of QPs without allocating excessive amounts of memory to hold WQEs on all the receive queues. It also reduces the burden imposed on the host processor to monitor all the receive queues and keep them filled with descriptors. Instead, the host processor need deal with only a single pool of descriptors that can serve many QPs.
- The methods of memory allocation provided by the present invention are conducive to balanced communication system design. In balanced designs, host applications running in conjunction with a channel adapter should, on average, consume data at the rate the data arrives from the network. This rate depends on the speed of the network, and not on the number of connections (QPs) served. Therefore, even when the host is serving many connections at once, the total amount of memory that must be allocated by the host to hold incoming data from the channel adapter should depend mainly (or exclusively) on the expected fluctuations in the rates of data arrival and processing. In this respect, the IB specification is wasteful of memory, as it requires memory allocation to scale up with the number of open connections. The present invention provides an alternative solution that maintains balance and efficiency in the use of memory resources.
- There is therefore provided, in accordance with a preferred embodiment of the present invention, a method for network communication, including:
- providing a pool of descriptors to be shared among a plurality of transport service instances used in communicating over a network, each of the descriptors in the pool including a scatter list indicating a buffer that is available in a local memory;
- receiving a message over the network on one of transport service instances, the message including data to be pushed to the local memory; and
- responsive to receiving the message, reading one of the descriptors from the pool, and writing the data included in the message to the buffer indicated by the scatter list included in the one of the descriptors.
- Preferably, providing the pool of descriptors includes writing the descriptors to the local memory using a host processor, and receiving the message includes receiving the message at a network adapter, which reads the one of the descriptors and writes the data to the buffer for processing by the host processor. In a preferred embodiment, the network includes a switch fabric, and the network adapter includes a channel adapter. Most preferably, writing the data includes, upon completion of writing the data to the buffer, submitting a completion report from the channel adapter to the host processor, the completion report including a pointer to the scatter list for use by the host processor in processing the data.
- Preferably, providing the pool of descriptors includes providing multiple pools of descriptors, each belonging to a respective group of the transport service instances, and reading the one of the descriptors includes determining the group to which the one of the transport service instances belongs, and reading the one of the descriptors from the pool belonging to the group.
- Further preferably, providing the pool of descriptors includes writing the descriptors to a circular list having a head and a tail, and reading the one of the descriptors includes selecting the one of the descriptors from the head of the list. Most preferably, selecting the one of the descriptors includes comparing the head and the tail of the list, and abstaining from writing the data to the buffer in the local memory if the head and tail coincide.
- In a preferred embodiment, receiving the message includes receiving one or more packets having payloads including the data. Preferably, the network includes a switch fabric, and the transport service instances includes queue pairs, and the message includes a send message conveyed over the fabric by a remote requester. Most preferably, each of the descriptors corresponds to a work queue element in a receive queue of one of the queue pairs.
- There is also provided, in accordance with a preferred embodiment of the present invention, a network adapter, including:
- a network interface, adapted to receive a message over a network on one of a plurality of transport service instances, the message including data to be pushed to a local memory;
- a host interface, adapted to be coupled to a host processor and to the local memory associated therewith; and
- message processing circuitry, coupled between the network interface and the host interface, which circuitry is adapted, responsive to receiving the message at the network interface, to read a descriptor from a pool of descriptors that is shared among the plurality of transport service instances, each of the descriptors in the pool including a scatter list indicating a buffer that is available in the local memory, and to write the data included in the message via the host interface to the buffer indicated by the scatter list included in the descriptor.
- Preferably, the descriptors in the pool are prepared in the local memory by the host processor, and the message processing circuitry is adapted to read the descriptors from the pool via the host interface.
- The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings in which:
-
FIG. 1 is a block diagram that schematically illustrates a network communication system, in accordance with a preferred embodiment of the present invention; -
FIG. 2 is a block diagram that schematically shows details of a host network adapter, in accordance with a preferred embodiment of the present invention; -
FIG. 3 is a block diagram that schematically illustrates data structures used by a network adapter in processing incoming messages, in accordance with a preferred embodiment of the present invention; and -
FIG. 4 is a flow chart that schematically illustrates a method for processing an incoming message, in accordance with a preferred embodiment of the present invention. -
FIG. 1 is a block diagram that schematically illustrates an IBnetwork communication system 20, in accordance with a preferred embodiment of the present invention. Insystem 20, aHCA 22 couples ahost processor 24 to an IB network (or fabric) 26. Typically,processor 24 comprises an Intel Pentium™ processor or other general-purpose computing device with suitable software.HCA 22 typically communicates vianetwork 26 with other HCAs, such as aremote HCA 28 with aremote host 30, as well as with target channel adapters (TCAs), such as aTCA 32 connected to an input/output (I/O) device 34. -
Host 24 andHCA 22 are connected to alocal system memory 38 via asuitable memory controller 36, as is known in the art. The HCA and memory typically occupy certain ranges of physical addresses in a defined address space on a bus connected to the controller, such as a Peripheral Component Interface (PCI) bus. In addition to the host operating system, applications and other data (not shown),memory 38 holds data structures that are accessed and used byHCA 22. These data structures preferably includeQP context information 42 maintained by the HCA, anddescriptors 44 corresponding to WQEs to be carried out byHCA 22. Althoughmemory 38 is shown inFIG. 1 as a single unit, which holds both control information and message payload data, the functions of the memory may be broken up among several units for purposes of convenient organization and access byhost 24 andHCA 22. The term system memory, as used in the present patent application and in the claims, should be understood broadly to encompass all areas of local memory that can be accessed byhost 24. -
Descriptors 44 are written tomemory 38 by client processes running onhost 24. They include send descriptors, corresponding to outgoing request messages to be sent overfabric 26 byHCA 22, and receive descriptors, used by the HCA to handle incoming send messages from remote requesters, such asHCA 28. The send descriptors are placed in the appropriate send queues of QPs for service byHCA 22, and are not of direct relevance to the present invention. At least a portion of the receive descriptors, however, are not placed directly in the receive queues of individual QPs. Rather, they are held in a descriptor pool, as described below, which is shared among multiple QPs. Each of the receive descriptors contains a scatter list, comprising one or more scatter entries, each indicating a range of addresses inmemory 38 to whichHCA 22 should write the data contained in the send message. Preferably, each scatter entry includes a base address and a length of the data to be written beginning at the base address. -
FIG. 2 is a block diagram that schematically shows details ofHCA 22, in accordance with a preferred embodiment of the present invention. For the sake of simplicity, elements ofHCA 22 that are not essential to an understanding of the present invention are omitted. The blocks and links that must be added to create a fully-operational RCA will be apparent to those skilled in the art. Further details of such a HCA are described in U.S. patent application Ser. No. 10/000,456, filed Dec. 4, 2001, which is assigned to the assignee of the present patent application, and whose disclosure is incorporated herein by reference. - The various blocks that make up
HCA 22 may be implemented either as hardware circuits or as software processes running on a programmable processor, or as a combination of hardware- and software-implemented elements. Although certain functional elements ofHCA 22 are shown as separate blocks in the figure for the sake of conceptual clarity, the functions represented by these blocks may actually be carried out by different software processes on a single embedded processor. Preferably, all of the elements of the HCA are implemented in a single integrated circuit chip, but multi-chip implementations are also within the scope of the present invention. - Incoming packets from
fabric 26 are received byHCA 22 at aninput port 50, which serves as a network interface. A transport check unit (TCU) 52 processes and verifies transport-layer information contained in the incoming packets, in order to confirm the validity of the packets and to determine how they are to be handled. For this purpose, the TCU reads the destination QP of each incoming packet, and then looks up therelevant context information 42 for the QP. Preferably, a cache 54 holds a copy of at least a portion of the context information that is required by the elements ofHCA 22 for processing incoming and outgoing packets on active QPs. If the TCU does not find the required context information in cache 54, the information is loaded into the cache frommemory 38. Further details of the operation of cache 54 are described in U.S. patent application Ser. No. 10/052,413 entitled, “Queue Pair Context Cache,” filed Jan. 23, 2002, which is assigned to the assignee of the present patent application and whose disclosure is incorporated herein by reference. - When the incoming packet contains data to be written to
memory 38, such as a RDMA write or send request packet,TCU 52 passes the packets to a receive data engine (RDE) 56, which attends to executing the requests. Awrite engine 58 inRDE 56 writes the packet data, via a translation protection table (TPT) 58, to addresses inmemory 38 that are specified in the applicable scatter list.TPT 58 acts as a host interface inHCA 22, performing address translation and protection checks to control access tomemory 38 both by elements ofHCA 22 and by other, remote entities connected tonetwork 26. Each RDMA write message carries its own scatter list, which is prepared by the remote requester. To process incoming send messages, however, writeengine 58 must read a receive descriptor frommemory 38, and use the scatter list provided by the descriptor. Processing of incoming send messages byRDE 56 is described in greater detail hereinbelow. After processing of an incoming send message has been completed (and likewise, processing of other types of messages, when required), acompletion reporter 60 writes a CQE to a completion queue inmemory 38. Writeengine 58 andcompletion reporter 60 also use QP and completion queue context information that is held in cache 54. - Preferably, when a given QP on
HCA 22 is configured to receive send messages, the QP can be set up byhost 24 either to have its own queue of receive descriptors, as prescribed by the IB specification, or to share descriptors in a pool with other QPs. Most preferably, multiple pools of this sort are supported byHCA 22. The configuration of the QP as a pool member is preferably indicated by a flag inQP context 42, as well as by a field in the context identifying the pool to which the QP belongs. -
FIG. 3 is a block diagram that schematically illustratesdata structures 70 maintained inmemory 38, which are used byRDE 56 in processing incoming send messages, in accordance with a preferred embodiment of the present invention. These structures are built around adescriptor pool 71, which serves a group of QPs. Other QPs may have their own, individual receive queues (not shown in the figures), as provided by the IB specification. Optionally, for more efficient operation ofRDE 56, some or all of the data instructures 70 are copied to cache 54, as well. -
Host 24 writesdescriptors 78, or WQEs, to eachopen descriptor pool 71. Preferably, for efficient operation, the host writes sequences of multiple descriptors without waiting for the pool to empty. The descriptors are preferably arranged in a cyclic buffer. When the host writes a descriptor to the buffer, it moves a producer index (PI) so that it points to the tail of the pool, i.e., to the last descriptor it has created. A consumer index (CI) points to the head of the queue and is manipulated byRDE 56 to indicate the next descriptor to be read from the pool. For eachpool 71, the producer and consumer indices are recorded and maintained relative to a predetermined base address. The use of these pointers is described in greater detail hereinbelow. - Upon receiving a send message on a given QP, and determining that the QP belongs to a receive descriptor pool,
RDE 56 reads apool number 72 for the QP from QP context information 42 (which is preferably held in cache 54). The pool number serves as an index (relative to a predetermined base address) to anentry 76 in a descriptor pool table (DPT) 74.Entry 76 contains information for use by the RDE in findingdescriptors 78 to read from thedescriptor pool 71 that is assigned to this QP. Thesame entry 76 is used for all the QPs belonging to the same pool. - Each
entry 76 inDPT 74 preferably includes the following information: - Start address—base address of
pool 71 inmemory 38. - Size of the descriptors in this pool, The size determines the length of the scatter lists that can be used.
- Total size of the pool, i.e., the maximum number of descriptors that the pool can hold. When the producer or consumer index reaches this value, it wraps back to the base address.
- Owner of the pool (client software or HCA hardware—when the owner is “hardware,” it means that the descriptors in the pool are available for use by the HCA).
- Producer index address—memory location to which
host 24 writes and updates the value of the producer index ofpool 71. - Consumer index.
- Each
descriptor 78 comprises one or more scatter entries, each indicating a buffer inmemory 38 to whichwrite engine 58 should write the data contained in an incoming send message. Preferably, each scatter entry includes a base address and a length of the data that can be written beginning at the base address. In addition,descriptor 78 may include other fields used for control and signaling toHCA 22. The structure ofdescriptors 78 inpool 71 is preferably the same as that of the descriptors that are placed in the receive queues of QPs that are not pool members. -
FIG. 4 is a flow chart that schematically illustrates a method by whichHCA 22 processes incoming send request messages, in accordance with a preferred embodiment of the present invention. The method is initiated whenTCU 52 receives a send packet from a remote requester viafabric 26, at asend reception step 80. After completing the required transport checks, the TCU passes the packet toRDE 56 for service. Note that IB send messages may comprise multiple packets, depending on the volume of data carried by the message and the maximum transfer unit (MTU) of the links over which the message travels. In the description that follows, it is assumed that the packet received atstep 80 is the first or only packet in the send message. For multi-packet messages, the same descriptor that is fetched and used to scatter the data in the first packet is retained byRDE 56 for use in processing the subsequent packets in the message, as well. - Write
engine 58 determines the destination QP of the send packet, based on the packet header, and then looks up the context of the QP in cache 54, at a poolmembership checking step 82. As noted above, the context indicates whether or not this QP belongs to a descriptor pool. If the QP is not a pool member, then in order to receive a send message, there must be a WQE available in the specific receive queue of this QP. The write engine reads the WQE address from the QP context and then fetches the WQE from the receive queue, at adescriptor fetching step 84. It then processes the send message in the usual way, as provided by the IB specification, - If
write engine 58 determines atstep 82 that the destination QP does, in fact, belong to a receive descriptor pool, it reads the number of the pool from the QP context, at a poolnumber reading step 86. It uses this number to find the information necessary to accessdescriptor pool 71 to which this QP belongs, at aninformation lookup step 88. This information is typically contained inentry 76 in table 74 (FIG. 3 ), which is indexed bypool number 72. Additionally, in order to accessdescriptor pool 71 inmemory 38, the write engine may need an access key, as is known in the art. This key is typically held in the QP context, and is preferably the same for all QPs belonging to the pool. - Using the information in
entry 76, writeengine 58 reads the consumer index (CI) and producer index (PI) ofdescriptor pool 71, at anindex checking step 90. If the values of these indices are equal, it means that alldescriptors 78 inpool 71 have already been used. Without a valid descriptor, the write engine is unable to process the current send packet. Under these circumstances, the send packet is typically discarded. If the send packet was sent on a reliable service, writeengine 58 instructs a send data engine (not shown) inHCA 22 to return a RNR NACK packet to the sender, at aNACK return step 92. The sender may subsequently resend the packet. Meanwhile, in such a case, the write engine preferably triggers an event, at anevent submission step 94, which is placed in an event queue to be read byhost 24. Optionally, an interrupt may be generated, as well, to prompt the host to service the event queue. When the host reads the event, it will be alerted to the fact thatdescriptors 78 inpool 71 have been exhausted. The host software should then generate new descriptors to replenish the pool. - As long as the values of CI and PI are not equal, write
engine 58 readsdescriptor 78 from the head of the circular buffer inpool 71, at the location indicated by the CI, at adescriptor reading step 96. It increments the CI to point to the next descriptor in the pool, at anindex incrementation step 98. The write engine then uses the scatter list provided by the descriptor it has read in processing the send packet data, at apacket processing step 100. To perform this processing, the write engine reads the first scatter entry from the scatter list indescriptor 78, which points to the first buffer to receive the data inmemory 38. The write engine pushes the data from the packet to this first buffer, until the buffer is filled. It then reads the next scatter entry, and continues pushing the data to the location that this scatter entry indicates. For multi-packet send messages, as long asHCA 22 continues to receive additional packets in the same message, the write engine proceeds through the scatter list entries of the descriptor it has read from the pool, until the message is completed. - Upon completion of an incoming send message, write
engine 58 instructscompletion reporter 60 to generate a completion queue element (CQE), at aCQE generation step 102. The completion reporter places the CQE in a completion queue inmemory 38, to be read by client software onhost 24. Optionally, an event or interrupt may also be generated to notify the host that there are new data inmemory 38 waiting to be read. Preferably, the CQE indicates the QP on which the incoming send message was received and includes a pointer to thedescriptor 78 inpool 71 that was used in processing the message that has now been completed.Host 24 reads the scatter list from the descriptor in order to determine the location of the data to be read inmemory 38. Once the host has read the data, the descriptor is no longer needed and can be overwritten by a new descriptor. - As noted above, for send messages using reliable connection services, the IB specification provides a flow control mechanism based on end-to-end credits. Typically, each credit represents one WQE posted to the receive queue of the responding QP. A QP that draws its WQEs from a shared descriptor pool, however, has no WQEs posted to its receive queue. Instead, these QPs may send credits to the corresponding requester based on the number of
descriptors 78 posted to pool 71 (preferably a smaller number of credits on each QP than there are actual descriptors in the pool). As long as an appropriate statistical relationship is maintained between the number of credits and the number of descriptors in the pool, there will usually be a descriptor available to handle each send message that arrives. Alternatively, even if the QPs belonging to pool 71 do not send credits to their corresponding requesters, or if a requester exhausts its credits, the requester may still transmit send packets in limited mode, as described in section 9.7.7.2.5 of the IB specification. - Although preferred embodiments are described herein with specific reference to IB terminology and conventions, the principles of the present invention may similarly be applied to handling of data “push” operations and message transfers using channel semantics in networks of other types. For example, the methods described hereinabove can be used in protocol bridge applications, in which multiple connections on a first network are served by a single sink to a second network. In this manner multiple hosts on the first network (for instance, on an IB fabric) can be connected to a converter that channels their traffic to the second network (such as an Ethernet network). by means of this mechanism, the amount of memory required by the protocol bridge is substantially reduced.
- It will thus be appreciated that the preferred embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
Claims (14)
1. A method for network communication, comprising:
providing a pool of descriptors to be shared among a plurality of transport service instances used in communicating over a network, each of the descriptors in the pool comprising a scatter list indicating a buffer that is available in a local memory;
receiving a message over the network on one of transport service instances, the message comprising data to be pushed to the local memory; and
responsive to receiving the message, reading one of the descriptors from the pool, and writing the data comprised in the message to the buffer indicated by the scatter list comprised in the one of the descriptors.
2. A method according to claim 1 , wherein providing the pool of descriptors comprises writing the descriptors to the local memory using a host processor, and wherein receiving the message comprises receiving the message at a network adapter, which reads the one of the descriptors and writes the data to the buffer for processing by the host processor.
3. A method according to claim 2 , wherein the network comprises a switch fabric, and wherein the network adapter comprises a channel adapter.
4. A method according to claim 2 , wherein writing the data comprises, upon completion of writing the data to the buffer, submitting a completion report from the channel adapter to the host processor, the completion report including a pointer to the scatter list for use by the host processor in processing the data.
5. A method according to claim 1 , wherein providing the pool of descriptors comprises providing multiple pools of descriptors, each belonging to a respective group of the transport service instances, and wherein reading the one of the descriptors comprises determining the group to which the one of the transport service instances belongs, and reading the one of the descriptors from the pool belonging to the group.
6. A method according to claim 1 , wherein providing the pool of descriptors comprises writing the descriptors to a circular list having a head and a tail, and wherein reading the one of the descriptors comprises selecting the one of the descriptors from the head of the list.
7. A method according to claim 1 , wherein receiving the message comprises receiving one or more packets having payloads comprising the data.
8. A network adapter, comprising:
a network interface, adapted to receive a message over a network on one of a plurality of transport service instances, the message comprising data to be pushed to a local memory;
a host interface, adapted to be coupled to a host processor and to the local memory associated therewith; and
message processing circuitry, coupled between the network interface and the host interface, which circuitry is adapted, responsive to receiving the message at the network interface, to read a descriptor from a pool of descriptors that is shared among the plurality of transport service instances, each of the descriptors in the pool comprising a scatter list indicating a buffer that is available in the local memory, and to write the data comprised in the message via the host interface to the buffer indicated by the scatter list comprised in the descriptor.
9. An adapter according to claim 8 , wherein the descriptors in the pool are prepared in the local memory by the host processor, and wherein the message processing circuitry is adapted to read the descriptors from the pool via the host interface.
10. An adapter according to claim 9 , wherein the message processing circuitry is adapted, upon completion of writing the data to the buffer, to submit a completion report via the host interface to the host processor, the completion report including a pointer to the scatter list for use by the host processor in processing the data.
11. An adapter according to claim 8 , wherein the network comprises a switch fabric, and wherein the network adapter comprises a channel adapter.
12. An adapter according to claim 8 , wherein the pool of descriptors is one of multiple pools of descriptors, each belonging to a respective group of the transport service instances, and wherein the message processing circuitry is adapted to determine the group to which the one of the transport service instances belongs, and to read the descriptor from the pool belonging to the group.
13. An adapter according to claim 8 , wherein the descriptors are entered in the pool in a circular list having a head and a tail, and wherein the message processing circuitry is adapted to read the descriptor from the head of the list.
14. An adapter according to claim 8 , wherein the message comprises one or more packets having payloads comprising the data.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/749,761 US20070220183A1 (en) | 2002-07-23 | 2007-05-17 | Receive Queue Descriptor Pool |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/200,189 US7263103B2 (en) | 2002-07-23 | 2002-07-23 | Receive queue descriptor pool |
US11/749,761 US20070220183A1 (en) | 2002-07-23 | 2007-05-17 | Receive Queue Descriptor Pool |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/200,189 Continuation US7263103B2 (en) | 2002-07-23 | 2002-07-23 | Receive queue descriptor pool |
Publications (1)
Publication Number | Publication Date |
---|---|
US20070220183A1 true US20070220183A1 (en) | 2007-09-20 |
Family
ID=30769517
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/200,189 Active 2025-06-02 US7263103B2 (en) | 2002-07-23 | 2002-07-23 | Receive queue descriptor pool |
US11/749,761 Abandoned US20070220183A1 (en) | 2002-07-23 | 2007-05-17 | Receive Queue Descriptor Pool |
US11/749,762 Abandoned US20070211741A1 (en) | 2002-07-23 | 2007-05-17 | Receive Queue Descriptor Pool |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/200,189 Active 2025-06-02 US7263103B2 (en) | 2002-07-23 | 2002-07-23 | Receive queue descriptor pool |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/749,762 Abandoned US20070211741A1 (en) | 2002-07-23 | 2007-05-17 | Receive Queue Descriptor Pool |
Country Status (1)
Country | Link |
---|---|
US (3) | US7263103B2 (en) |
Cited By (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060047867A1 (en) * | 2004-08-30 | 2006-03-02 | International Business Machines Corporation | Method, system, and storage medium for providing queue pairs for I/O adapters |
US20100049876A1 (en) * | 2005-04-27 | 2010-02-25 | Solarflare Communications, Inc. | Packet validation in virtual network interface architecture |
US20100057932A1 (en) * | 2006-07-10 | 2010-03-04 | Solarflare Communications Incorporated | Onload network protocol stacks |
US20100135324A1 (en) * | 2006-11-01 | 2010-06-03 | Solarflare Communications Inc. | Driver level segmentation |
US20100161847A1 (en) * | 2008-12-18 | 2010-06-24 | Solarflare Communications, Inc. | Virtualised interface functions |
US20110023042A1 (en) * | 2008-02-05 | 2011-01-27 | Solarflare Communications Inc. | Scalable sockets |
US20110029734A1 (en) * | 2009-07-29 | 2011-02-03 | Solarflare Communications Inc | Controller Integration |
US20110087774A1 (en) * | 2009-10-08 | 2011-04-14 | Solarflare Communications Inc | Switching api |
US20110149966A1 (en) * | 2009-12-21 | 2011-06-23 | Solarflare Communications Inc | Header Processing Engine |
US20110173514A1 (en) * | 2003-03-03 | 2011-07-14 | Solarflare Communications, Inc. | Data protocol |
US8533740B2 (en) | 2005-03-15 | 2013-09-10 | Solarflare Communications, Inc. | Data processing system with intercepting instructions |
US8543729B2 (en) | 2007-11-29 | 2013-09-24 | Solarflare Communications, Inc. | Virtualised receive side scaling |
US8612536B2 (en) | 2004-04-21 | 2013-12-17 | Solarflare Communications, Inc. | User-level stack |
US8635353B2 (en) | 2005-06-15 | 2014-01-21 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US8650569B2 (en) | 2005-03-10 | 2014-02-11 | Solarflare Communications, Inc. | User-level re-initialization instruction interception |
US8737431B2 (en) | 2004-04-21 | 2014-05-27 | Solarflare Communications, Inc. | Checking data integrity |
US8763018B2 (en) | 2011-08-22 | 2014-06-24 | Solarflare Communications, Inc. | Modifying application behaviour |
US8782642B2 (en) | 2005-03-15 | 2014-07-15 | Solarflare Communications, Inc. | Data processing system with data transmit capability |
US8817784B2 (en) | 2006-02-08 | 2014-08-26 | Solarflare Communications, Inc. | Method and apparatus for multicast packet reception |
US8855137B2 (en) | 2004-03-02 | 2014-10-07 | Solarflare Communications, Inc. | Dual-driver interface |
US8868780B2 (en) | 2005-03-30 | 2014-10-21 | Solarflare Communications, Inc. | Data processing system with routing tables |
US8954613B2 (en) | 2002-09-16 | 2015-02-10 | Solarflare Communications, Inc. | Network interface and protocol |
US8959095B2 (en) | 2005-10-20 | 2015-02-17 | Solarflare Communications, Inc. | Hashing algorithm for network receive filtering |
US8996644B2 (en) | 2010-12-09 | 2015-03-31 | Solarflare Communications, Inc. | Encapsulated accelerator |
US9003053B2 (en) | 2011-09-22 | 2015-04-07 | Solarflare Communications, Inc. | Message acceleration |
US9008113B2 (en) | 2010-12-20 | 2015-04-14 | Solarflare Communications, Inc. | Mapped FIFO buffering |
US9210140B2 (en) | 2009-08-19 | 2015-12-08 | Solarflare Communications, Inc. | Remote functionality selection |
US9258390B2 (en) | 2011-07-29 | 2016-02-09 | Solarflare Communications, Inc. | Reducing network latency |
US9300599B2 (en) | 2013-05-30 | 2016-03-29 | Solarflare Communications, Inc. | Packet capture |
US9384071B2 (en) | 2011-03-31 | 2016-07-05 | Solarflare Communications, Inc. | Epoll optimisations |
US9391840B2 (en) | 2012-05-02 | 2016-07-12 | Solarflare Communications, Inc. | Avoiding delayed data |
US9391841B2 (en) | 2012-07-03 | 2016-07-12 | Solarflare Communications, Inc. | Fast linkup arbitration |
US9426124B2 (en) | 2013-04-08 | 2016-08-23 | Solarflare Communications, Inc. | Locked down network interface |
US9600429B2 (en) | 2010-12-09 | 2017-03-21 | Solarflare Communications, Inc. | Encapsulated accelerator |
US9674318B2 (en) | 2010-12-09 | 2017-06-06 | Solarflare Communications, Inc. | TCP processing for devices |
US9686117B2 (en) | 2006-07-10 | 2017-06-20 | Solarflare Communications, Inc. | Chimney onload implementation of network protocol stack |
US9948533B2 (en) | 2006-07-10 | 2018-04-17 | Solarflare Communitations, Inc. | Interrupt management |
US10015104B2 (en) | 2005-12-28 | 2018-07-03 | Solarflare Communications, Inc. | Processing received data |
US10394751B2 (en) | 2013-11-06 | 2019-08-27 | Solarflare Communications, Inc. | Programmed input/output mode |
US10505747B2 (en) | 2012-10-16 | 2019-12-10 | Solarflare Communications, Inc. | Feed processing |
US10742604B2 (en) | 2013-04-08 | 2020-08-11 | Xilinx, Inc. | Locked down network interface |
US10873613B2 (en) | 2010-12-09 | 2020-12-22 | Xilinx, Inc. | TCP processing for devices |
Families Citing this family (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7543037B2 (en) * | 2003-12-02 | 2009-06-02 | International Business Machines Corporation | RDMA completion and retransmit system and method |
US7609636B1 (en) * | 2004-03-29 | 2009-10-27 | Sun Microsystems, Inc. | System and method for infiniband receive flow control with combined buffering of virtual lanes and queue pairs |
US7263568B2 (en) * | 2004-03-31 | 2007-08-28 | Intel Corporation | Interrupt system using event data structures |
US7197588B2 (en) * | 2004-03-31 | 2007-03-27 | Intel Corporation | Interrupt scheme for an Input/Output device |
US20060004983A1 (en) * | 2004-06-30 | 2006-01-05 | Tsao Gary Y | Method, system, and program for managing memory options for devices |
KR20060012477A (en) * | 2004-08-03 | 2006-02-08 | 엘지전자 주식회사 | Method for restricting received message of push message service |
US7370174B2 (en) * | 2005-01-05 | 2008-05-06 | Intel Corporation | Method, system, and program for addressing pages of memory by an I/O device |
US7853957B2 (en) * | 2005-04-15 | 2010-12-14 | Intel Corporation | Doorbell mechanism using protection domains |
US8634431B1 (en) * | 2006-11-10 | 2014-01-21 | Marvell International Ltd. | Quality of service and flow control architecture for a passive optical network |
US9178713B1 (en) | 2006-11-28 | 2015-11-03 | Marvell International Ltd. | Optical line termination in a passive optical network |
US9071465B1 (en) * | 2010-05-18 | 2015-06-30 | Cellco Partnership | Method and system for SMS services and bind scaling |
US8645663B2 (en) | 2011-09-12 | 2014-02-04 | Mellanox Technologies Ltd. | Network interface controller with flexible memory handling |
US9143467B2 (en) | 2011-10-25 | 2015-09-22 | Mellanox Technologies Ltd. | Network interface controller with circular receive buffer |
US9397960B2 (en) | 2011-11-08 | 2016-07-19 | Mellanox Technologies Ltd. | Packet steering |
US8751701B2 (en) | 2011-12-26 | 2014-06-10 | Mellanox Technologies Ltd. | Host channel adapter with pattern-type DMA |
US9256545B2 (en) | 2012-05-15 | 2016-02-09 | Mellanox Technologies Ltd. | Shared memory access using independent memory maps |
US9871734B2 (en) * | 2012-05-28 | 2018-01-16 | Mellanox Technologies, Ltd. | Prioritized handling of incoming packets by a network interface controller |
US8761189B2 (en) | 2012-06-28 | 2014-06-24 | Mellanox Technologies Ltd. | Responding to dynamically-connected transport requests |
US9639464B2 (en) | 2012-09-27 | 2017-05-02 | Mellanox Technologies, Ltd. | Application-assisted handling of page faults in I/O operations |
US8914458B2 (en) | 2012-09-27 | 2014-12-16 | Mellanox Technologies Ltd. | Look-ahead handling of page faults in I/O operations |
US8745276B2 (en) | 2012-09-27 | 2014-06-03 | Mellanox Technologies Ltd. | Use of free pages in handling of page faults |
US9298642B2 (en) | 2012-11-01 | 2016-03-29 | Mellanox Technologies Ltd. | Sharing address translation between CPU and peripheral devices |
US9727503B2 (en) | 2014-03-17 | 2017-08-08 | Mellanox Technologies, Ltd. | Storage system and server |
US9696942B2 (en) | 2014-03-17 | 2017-07-04 | Mellanox Technologies, Ltd. | Accessing remote storage devices using a local bus protocol |
US10454991B2 (en) | 2014-03-24 | 2019-10-22 | Mellanox Technologies, Ltd. | NIC with switching functionality between network ports |
US10120832B2 (en) | 2014-05-27 | 2018-11-06 | Mellanox Technologies, Ltd. | Direct access to local memory in a PCI-E device |
US10031857B2 (en) | 2014-05-27 | 2018-07-24 | Mellanox Technologies, Ltd. | Address translation services for direct accessing of local memory over a network fabric |
US10148581B2 (en) | 2016-05-30 | 2018-12-04 | Mellanox Technologies, Ltd. | End-to-end enhanced reliable datagram transport |
US10516710B2 (en) | 2017-02-12 | 2019-12-24 | Mellanox Technologies, Ltd. | Direct packet placement |
US10210125B2 (en) | 2017-03-16 | 2019-02-19 | Mellanox Technologies, Ltd. | Receive queue with stride-based data scattering |
US11252464B2 (en) | 2017-06-14 | 2022-02-15 | Mellanox Technologies, Ltd. | Regrouping of video data in host memory |
US10367750B2 (en) | 2017-06-15 | 2019-07-30 | Mellanox Technologies, Ltd. | Transmission and reception of raw video using scalable frame rate |
US11128575B2 (en) | 2018-07-20 | 2021-09-21 | Bank Of America Corporation | System for increasing processing efficiency through automated resource field transformation |
US11398979B2 (en) | 2020-10-28 | 2022-07-26 | Mellanox Technologies, Ltd. | Dynamic processing trees |
US11940933B2 (en) | 2021-03-02 | 2024-03-26 | Mellanox Technologies, Ltd. | Cross address-space bridging |
US11934658B2 (en) | 2021-03-25 | 2024-03-19 | Mellanox Technologies, Ltd. | Enhanced storage protocol emulation in a peripheral device |
US11934333B2 (en) | 2021-03-25 | 2024-03-19 | Mellanox Technologies, Ltd. | Storage protocol emulation in a peripheral device |
US11726666B2 (en) | 2021-07-11 | 2023-08-15 | Mellanox Technologies, Ltd. | Network adapter with efficient storage-protocol emulation |
US11929934B2 (en) | 2022-04-27 | 2024-03-12 | Mellanox Technologies, Ltd. | Reliable credit-based communication over long-haul links |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020124117A1 (en) * | 2001-03-01 | 2002-09-05 | Ibm Corporation | Infiniband memory windows management directly in hardware |
US6553438B1 (en) * | 2000-04-24 | 2003-04-22 | Intel Corporation | Methods and system for message resource pool with asynchronous and synchronous modes of operation |
US6820091B2 (en) * | 2001-05-11 | 2004-11-16 | International Business Machines Corporation | Dynamic buffer allocation |
US6948004B2 (en) * | 2001-03-28 | 2005-09-20 | Intel Corporation | Host-fabric adapter having work queue entry (WQE) ring hardware assist (HWA) mechanism |
US7080377B2 (en) * | 2000-06-29 | 2006-07-18 | Eci Telecom Ltd. | Method for effective utilizing of shared resources in computerized system |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6064805A (en) * | 1997-07-02 | 2000-05-16 | Unisys Corporation | Method, system, and computer program product for intraconnect data communication using buffer pools and buffer pool management |
US6651117B1 (en) * | 1999-11-03 | 2003-11-18 | Adaptec, Inc. | Network stack layer interface |
US6947970B2 (en) * | 2000-12-19 | 2005-09-20 | Intel Corporation | Method and apparatus for multilevel translation and protection table |
US7190667B2 (en) * | 2001-04-26 | 2007-03-13 | Intel Corporation | Link level packet flow control mechanism |
US7036118B1 (en) * | 2001-12-20 | 2006-04-25 | Mindspeed Technologies, Inc. | System for executing computer programs on a limited-memory computing machine |
-
2002
- 2002-07-23 US US10/200,189 patent/US7263103B2/en active Active
-
2007
- 2007-05-17 US US11/749,761 patent/US20070220183A1/en not_active Abandoned
- 2007-05-17 US US11/749,762 patent/US20070211741A1/en not_active Abandoned
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6553438B1 (en) * | 2000-04-24 | 2003-04-22 | Intel Corporation | Methods and system for message resource pool with asynchronous and synchronous modes of operation |
US7080377B2 (en) * | 2000-06-29 | 2006-07-18 | Eci Telecom Ltd. | Method for effective utilizing of shared resources in computerized system |
US20020124117A1 (en) * | 2001-03-01 | 2002-09-05 | Ibm Corporation | Infiniband memory windows management directly in hardware |
US6948004B2 (en) * | 2001-03-28 | 2005-09-20 | Intel Corporation | Host-fabric adapter having work queue entry (WQE) ring hardware assist (HWA) mechanism |
US6820091B2 (en) * | 2001-05-11 | 2004-11-16 | International Business Machines Corporation | Dynamic buffer allocation |
Cited By (97)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8954613B2 (en) | 2002-09-16 | 2015-02-10 | Solarflare Communications, Inc. | Network interface and protocol |
US9112752B2 (en) | 2002-09-16 | 2015-08-18 | Solarflare Communications, Inc. | Network interface and protocol |
US9043671B2 (en) | 2003-03-03 | 2015-05-26 | Solarflare Communications, Inc. | Data protocol |
US20110173514A1 (en) * | 2003-03-03 | 2011-07-14 | Solarflare Communications, Inc. | Data protocol |
US11182317B2 (en) | 2004-03-02 | 2021-11-23 | Xilinx, Inc. | Dual-driver interface |
US8855137B2 (en) | 2004-03-02 | 2014-10-07 | Solarflare Communications, Inc. | Dual-driver interface |
US9690724B2 (en) | 2004-03-02 | 2017-06-27 | Solarflare Communications, Inc. | Dual-driver interface |
US11119956B2 (en) | 2004-03-02 | 2021-09-14 | Xilinx, Inc. | Dual-driver interface |
US8737431B2 (en) | 2004-04-21 | 2014-05-27 | Solarflare Communications, Inc. | Checking data integrity |
US8612536B2 (en) | 2004-04-21 | 2013-12-17 | Solarflare Communications, Inc. | User-level stack |
US20060047867A1 (en) * | 2004-08-30 | 2006-03-02 | International Business Machines Corporation | Method, system, and storage medium for providing queue pairs for I/O adapters |
US8055818B2 (en) * | 2004-08-30 | 2011-11-08 | International Business Machines Corporation | Low latency queue pairs for I/O adapters |
US8650569B2 (en) | 2005-03-10 | 2014-02-11 | Solarflare Communications, Inc. | User-level re-initialization instruction interception |
US9063771B2 (en) | 2005-03-10 | 2015-06-23 | Solarflare Communications, Inc. | User-level re-initialization instruction interception |
US8533740B2 (en) | 2005-03-15 | 2013-09-10 | Solarflare Communications, Inc. | Data processing system with intercepting instructions |
US9552225B2 (en) | 2005-03-15 | 2017-01-24 | Solarflare Communications, Inc. | Data processing system with data transmit capability |
US8782642B2 (en) | 2005-03-15 | 2014-07-15 | Solarflare Communications, Inc. | Data processing system with data transmit capability |
US9729436B2 (en) | 2005-03-30 | 2017-08-08 | Solarflare Communications, Inc. | Data processing system with routing tables |
US10397103B2 (en) | 2005-03-30 | 2019-08-27 | Solarflare Communications, Inc. | Data processing system with routing tables |
US8868780B2 (en) | 2005-03-30 | 2014-10-21 | Solarflare Communications, Inc. | Data processing system with routing tables |
US10924483B2 (en) | 2005-04-27 | 2021-02-16 | Xilinx, Inc. | Packet validation in virtual network interface architecture |
US9912665B2 (en) | 2005-04-27 | 2018-03-06 | Solarflare Communications, Inc. | Packet validation in virtual network interface architecture |
US8380882B2 (en) | 2005-04-27 | 2013-02-19 | Solarflare Communications, Inc. | Packet validation in virtual network interface architecture |
US20100049876A1 (en) * | 2005-04-27 | 2010-02-25 | Solarflare Communications, Inc. | Packet validation in virtual network interface architecture |
US8635353B2 (en) | 2005-06-15 | 2014-01-21 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US9043380B2 (en) | 2005-06-15 | 2015-05-26 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US10445156B2 (en) | 2005-06-15 | 2019-10-15 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US8645558B2 (en) | 2005-06-15 | 2014-02-04 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities for data extraction |
US11210148B2 (en) | 2005-06-15 | 2021-12-28 | Xilinx, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US10055264B2 (en) | 2005-06-15 | 2018-08-21 | Solarflare Communications, Inc. | Reception according to a data transfer protocol of data directed to any of a plurality of destination entities |
US9594842B2 (en) | 2005-10-20 | 2017-03-14 | Solarflare Communications, Inc. | Hashing algorithm for network receive filtering |
US8959095B2 (en) | 2005-10-20 | 2015-02-17 | Solarflare Communications, Inc. | Hashing algorithm for network receive filtering |
US10015104B2 (en) | 2005-12-28 | 2018-07-03 | Solarflare Communications, Inc. | Processing received data |
US10104005B2 (en) | 2006-01-10 | 2018-10-16 | Solarflare Communications, Inc. | Data buffering |
US9083539B2 (en) | 2006-02-08 | 2015-07-14 | Solarflare Communications, Inc. | Method and apparatus for multicast packet reception |
US8817784B2 (en) | 2006-02-08 | 2014-08-26 | Solarflare Communications, Inc. | Method and apparatus for multicast packet reception |
US8489761B2 (en) | 2006-07-10 | 2013-07-16 | Solarflare Communications, Inc. | Onload network protocol stacks |
US9686117B2 (en) | 2006-07-10 | 2017-06-20 | Solarflare Communications, Inc. | Chimney onload implementation of network protocol stack |
US20100057932A1 (en) * | 2006-07-10 | 2010-03-04 | Solarflare Communications Incorporated | Onload network protocol stacks |
US10382248B2 (en) | 2006-07-10 | 2019-08-13 | Solarflare Communications, Inc. | Chimney onload implementation of network protocol stack |
US9948533B2 (en) | 2006-07-10 | 2018-04-17 | Solarflare Communitations, Inc. | Interrupt management |
US20100135324A1 (en) * | 2006-11-01 | 2010-06-03 | Solarflare Communications Inc. | Driver level segmentation |
US9077751B2 (en) | 2006-11-01 | 2015-07-07 | Solarflare Communications, Inc. | Driver level segmentation |
US8543729B2 (en) | 2007-11-29 | 2013-09-24 | Solarflare Communications, Inc. | Virtualised receive side scaling |
US20110023042A1 (en) * | 2008-02-05 | 2011-01-27 | Solarflare Communications Inc. | Scalable sockets |
US9304825B2 (en) | 2008-02-05 | 2016-04-05 | Solarflare Communications, Inc. | Processing, on multiple processors, data flows received through a single socket |
US8447904B2 (en) | 2008-12-18 | 2013-05-21 | Solarflare Communications, Inc. | Virtualised interface functions |
US20100161847A1 (en) * | 2008-12-18 | 2010-06-24 | Solarflare Communications, Inc. | Virtualised interface functions |
US20110029734A1 (en) * | 2009-07-29 | 2011-02-03 | Solarflare Communications Inc | Controller Integration |
US9256560B2 (en) | 2009-07-29 | 2016-02-09 | Solarflare Communications, Inc. | Controller integration |
US9210140B2 (en) | 2009-08-19 | 2015-12-08 | Solarflare Communications, Inc. | Remote functionality selection |
US8423639B2 (en) | 2009-10-08 | 2013-04-16 | Solarflare Communications, Inc. | Switching API |
US20110087774A1 (en) * | 2009-10-08 | 2011-04-14 | Solarflare Communications Inc | Switching api |
US20110149966A1 (en) * | 2009-12-21 | 2011-06-23 | Solarflare Communications Inc | Header Processing Engine |
US8743877B2 (en) | 2009-12-21 | 2014-06-03 | Steven L. Pope | Header processing engine |
US9124539B2 (en) | 2009-12-21 | 2015-09-01 | Solarflare Communications, Inc. | Header processing engine |
US11134140B2 (en) | 2010-12-09 | 2021-09-28 | Xilinx, Inc. | TCP processing for devices |
US9674318B2 (en) | 2010-12-09 | 2017-06-06 | Solarflare Communications, Inc. | TCP processing for devices |
US9880964B2 (en) | 2010-12-09 | 2018-01-30 | Solarflare Communications, Inc. | Encapsulated accelerator |
US9892082B2 (en) | 2010-12-09 | 2018-02-13 | Solarflare Communications Inc. | Encapsulated accelerator |
US11132317B2 (en) | 2010-12-09 | 2021-09-28 | Xilinx, Inc. | Encapsulated accelerator |
US11876880B2 (en) | 2010-12-09 | 2024-01-16 | Xilinx, Inc. | TCP processing for devices |
US10873613B2 (en) | 2010-12-09 | 2020-12-22 | Xilinx, Inc. | TCP processing for devices |
US10572417B2 (en) | 2010-12-09 | 2020-02-25 | Xilinx, Inc. | Encapsulated accelerator |
US9600429B2 (en) | 2010-12-09 | 2017-03-21 | Solarflare Communications, Inc. | Encapsulated accelerator |
US10515037B2 (en) | 2010-12-09 | 2019-12-24 | Solarflare Communications, Inc. | Encapsulated accelerator |
US8996644B2 (en) | 2010-12-09 | 2015-03-31 | Solarflare Communications, Inc. | Encapsulated accelerator |
US9800513B2 (en) | 2010-12-20 | 2017-10-24 | Solarflare Communications, Inc. | Mapped FIFO buffering |
US9008113B2 (en) | 2010-12-20 | 2015-04-14 | Solarflare Communications, Inc. | Mapped FIFO buffering |
US10671458B2 (en) | 2011-03-31 | 2020-06-02 | Xilinx, Inc. | Epoll optimisations |
US9384071B2 (en) | 2011-03-31 | 2016-07-05 | Solarflare Communications, Inc. | Epoll optimisations |
US10425512B2 (en) | 2011-07-29 | 2019-09-24 | Solarflare Communications, Inc. | Reducing network latency |
US9456060B2 (en) | 2011-07-29 | 2016-09-27 | Solarflare Communications, Inc. | Reducing network latency |
US10469632B2 (en) | 2011-07-29 | 2019-11-05 | Solarflare Communications, Inc. | Reducing network latency |
US10021223B2 (en) | 2011-07-29 | 2018-07-10 | Solarflare Communications, Inc. | Reducing network latency |
US9258390B2 (en) | 2011-07-29 | 2016-02-09 | Solarflare Communications, Inc. | Reducing network latency |
US8763018B2 (en) | 2011-08-22 | 2014-06-24 | Solarflare Communications, Inc. | Modifying application behaviour |
US11392429B2 (en) | 2011-08-22 | 2022-07-19 | Xilinx, Inc. | Modifying application behaviour |
US10713099B2 (en) | 2011-08-22 | 2020-07-14 | Xilinx, Inc. | Modifying application behaviour |
US9003053B2 (en) | 2011-09-22 | 2015-04-07 | Solarflare Communications, Inc. | Message acceleration |
US9391840B2 (en) | 2012-05-02 | 2016-07-12 | Solarflare Communications, Inc. | Avoiding delayed data |
US9882781B2 (en) | 2012-07-03 | 2018-01-30 | Solarflare Communications, Inc. | Fast linkup arbitration |
US10498602B2 (en) | 2012-07-03 | 2019-12-03 | Solarflare Communications, Inc. | Fast linkup arbitration |
US11095515B2 (en) | 2012-07-03 | 2021-08-17 | Xilinx, Inc. | Using receive timestamps to update latency estimates |
US11108633B2 (en) | 2012-07-03 | 2021-08-31 | Xilinx, Inc. | Protocol selection in dependence upon conversion time |
US9391841B2 (en) | 2012-07-03 | 2016-07-12 | Solarflare Communications, Inc. | Fast linkup arbitration |
US11374777B2 (en) | 2012-10-16 | 2022-06-28 | Xilinx, Inc. | Feed processing |
US10505747B2 (en) | 2012-10-16 | 2019-12-10 | Solarflare Communications, Inc. | Feed processing |
US10742604B2 (en) | 2013-04-08 | 2020-08-11 | Xilinx, Inc. | Locked down network interface |
US10212135B2 (en) | 2013-04-08 | 2019-02-19 | Solarflare Communications, Inc. | Locked down network interface |
US9426124B2 (en) | 2013-04-08 | 2016-08-23 | Solarflare Communications, Inc. | Locked down network interface |
US10999246B2 (en) | 2013-04-08 | 2021-05-04 | Xilinx, Inc. | Locked down network interface |
US9300599B2 (en) | 2013-05-30 | 2016-03-29 | Solarflare Communications, Inc. | Packet capture |
US10394751B2 (en) | 2013-11-06 | 2019-08-27 | Solarflare Communications, Inc. | Programmed input/output mode |
US11249938B2 (en) | 2013-11-06 | 2022-02-15 | Xilinx, Inc. | Programmed input/output mode |
US11023411B2 (en) | 2013-11-06 | 2021-06-01 | Xilinx, Inc. | Programmed input/output mode |
US11809367B2 (en) | 2013-11-06 | 2023-11-07 | Xilinx, Inc. | Programmed input/output mode |
Also Published As
Publication number | Publication date |
---|---|
US20070211741A1 (en) | 2007-09-13 |
US7263103B2 (en) | 2007-08-28 |
US20040017819A1 (en) | 2004-01-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7263103B2 (en) | Receive queue descriptor pool | |
US11916781B2 (en) | System and method for facilitating efficient utilization of an output buffer in a network interface controller (NIC) | |
US7603429B2 (en) | Network adapter with shared database for message context information | |
US7152122B2 (en) | Queue pair context cache | |
US7076569B1 (en) | Embedded channel adapter having transport layer configured for prioritizing selection of work descriptors based on respective virtual lane priorities | |
US6480500B1 (en) | Arrangement for creating multiple virtual queue pairs from a compressed queue pair based on shared attributes | |
US7631106B2 (en) | Prefetching of receive queue descriptors | |
US6912604B1 (en) | Host channel adapter having partitioned link layer services for an infiniband server system | |
US20040252685A1 (en) | Channel adapter with integrated switch | |
US11750418B2 (en) | Cross network bridging | |
US7457845B2 (en) | Method and system for TCP/IP using generic buffers for non-posting TCP applications | |
US6816889B1 (en) | Assignment of dual port memory banks for a CPU and a host channel adapter in an InfiniBand computing node | |
US7209489B1 (en) | Arrangement in a channel adapter for servicing work notifications based on link layer virtual lane processing | |
US7466716B2 (en) | Reducing latency in a channel adapter by accelerated I/O control block processing | |
US7292593B1 (en) | Arrangement in a channel adapter for segregating transmit packet data in transmit buffers based on respective virtual lanes | |
CN116471242A (en) | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method | |
US7085869B1 (en) | Arrangement for managing transmitted packets requiring acknowledgement in a host channel adapter | |
US7266614B1 (en) | Embedded channel adapter having link layer configured for concurrent retrieval of payload data during packet transmission |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |