US20030058248A1 - System and method for communicating graphics over a network - Google Patents

System and method for communicating graphics over a network Download PDF

Info

Publication number
US20030058248A1
US20030058248A1 US09/960,523 US96052301A US2003058248A1 US 20030058248 A1 US20030058248 A1 US 20030058248A1 US 96052301 A US96052301 A US 96052301A US 2003058248 A1 US2003058248 A1 US 2003058248A1
Authority
US
United States
Prior art keywords
graphics information
frame
current frame
graphics
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US09/960,523
Inventor
Roland Hochmuth
John Marks
David Sweetser
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hewlett Packard Development Co LP
Original Assignee
Hewlett Packard Co
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett Packard Co filed Critical Hewlett Packard Co
Priority to US09/960,523 priority Critical patent/US20030058248A1/en
Assigned to HEWLETT-PACKARD COMPANY reassignment HEWLETT-PACKARD COMPANY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SWEETSER, DAVID JOHN, HOCHMUTH, ROLAND M., MARKS, JOHN
Publication of US20030058248A1 publication Critical patent/US20030058248A1/en
Assigned to HEWLETT-PACKARD DEVELOPMENT COMPANY L.P. reassignment HEWLETT-PACKARD DEVELOPMENT COMPANY L.P. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HEWLETT-PACKARD COMPANY
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8146Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04Protocols for data compression, e.g. ROHC
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/507Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction using conditional replenishment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23406Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving management of server-side video buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2381Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/131Protocols for games, networked simulations or virtual reality

Definitions

  • the present invention generally relates to networked computer systems, and more particularly to a system and method for communicating graphics over a network.
  • OS-dependent code that operates at the device driver layer, which must be written to control the devices. This OS-dependent code varies from system to system.
  • a further problem with many prior art systems relates to bandwidth consumption, particularly when communicating graphics information.
  • successive frames of data may be transmitted every ⁇ fraction (1/60) ⁇ th of a second (to support a 60 Hz refresh rate), or at some other rate (if other than 60 Hz refresh rate).
  • Transmitting entire frames of graphics information every ⁇ fraction (1/60) ⁇ th of a second may exceed the bandwidth limitations of many networks, particularly when communicating over a wide area network.
  • the present invention is generally directed to a system and method for communicating graphics across one or more networks.
  • the system comprises a frame buffer memory for storing and maintaining a previous frame of graphics information, a temporary memory configured to store at least a portion of a current frame of graphics information, comparison logic for comparing a portion of the current frame of graphics information with a corresponding portion of the previous frame, and transmission logic for transmitting the portion of the current frame to a destination computer, if the comparison logic determines that the portion of the current frame of graphics information differs from the corresponding portion of the previous frame by more than a predetermined measure.
  • a method that stores a frame of graphics information obtained from a video signal, receives at least a portion of a current frame of graphics information, compares a portion of the current frame of graphics information with a corresponding portion of the stored frame of graphics information. Then, if the compared portion of the current frame of graphics information differs by at least a predetermined amount from the corresponding portion of the stored graphics information, the method transmits the compared portion of the current frame of graphics information to a destination computer and overwrites the corresponding portion of the stored graphics information with the compared portion of the current frame of graphics information.
  • FIG. 1 is a block diagram illustrating a system environment for accommodating one embodiment of the invention
  • FIG. 2 is a block diagram of a network video apparatus (NVA);
  • FIG. 3 is a flowchart illustrating the top-level operation of an NVA
  • FIG. 4 is a block diagram similar to FIG. 1, but illustrating an alternative system configuration
  • FIG. 5 is a block diagram, similar to the diagram of FIG. 2, but illustrating components of a NVA constructed in accordance with the embodiment illustrated in FIG. 4;
  • FIG. 6 is a block diagram similar to FIG. 1, but illustrating a system constructed in accordance with another embodiment
  • FIG. 7 is a block diagram illustrating a system constructed in accordance with one embodiment of the present invention.
  • FIG. 8 is a flowchart illustrating the top-level operation of an NVA in accordance with one embodiment of the invention.
  • FIG. 9 is a flowchart illustrating the top-level operation of a destination computer constructed in accordance with one embodiment of the invention.
  • FIG. 10 is a block diagram illustrating certain components of an NVA constructed in accordance with an embodiment of the invention.
  • FIG. 11 is block diagram illustrating certain components of a destination computer constructed to receive graphics information from an NVA constructed in accordance with one embodiment of the invention.
  • FIG. 12 is a flowchart illustrating the top-level operation of an NVA constructed in accordance with an alternative embodiment of the invention.
  • FIGS. 1 - 6 are diagrams and flowcharts illustrating certain environmental elements and features, in which the present invention may reside or operate.
  • FIGS. 7 - 12 are diagrams and flowcharts that more particularly illustrate preferred embodiments of the present invention.
  • FIG. 1 is a diagram illustrating certain principal components of a system 10 in which the present invention may be used. More particularly, the system 10 comprises components that cooperate to communicate graphics information from a source computer 20 to a destination computer 50 across one or more networks.
  • the system 10 comprises both a local area network (LAN) 30 as well as a wide area network (WAN) 40 .
  • LAN local area network
  • WAN wide area network
  • the WAN 40 will be the Internet.
  • a novel NVA 100 is provided.
  • Various embodiments of such a NVA 100 will be described in more detail herein (e.g., FIGS. 2 and 5).
  • the NVA 100 operates by receiving a video signal at an input, formatting the video signal for network communication, and outputting the formatted video signal across one or more networks.
  • a standard or conventional video signal such as a video signal generated by a video graphics card, may be connected to the NVA 100 . This connection may be made through conventional cabling 22 and a connector 102 coupled to an input of the NVA 100 .
  • the NVA 100 may store the video signal in an internal memory.
  • the video signal may be either a digital video signal, such as a Digital Visual Interface (DVI) compliant signal, or may be an analog video signal. If the video signal carried on cable 22 is an analog signal, then the NVA 100 will perform an appropriate analog-to-digital conversion, before storing the video signal in memory.
  • a compression is performed of the graphics information stored in memory, before communicating the graphics information over the networks. Such a compression can significantly reduce the amount of data that is transmitted across the networks 30 and 40 . With regard to this compression, it will be appreciated that a variety of compression algorithms may be utilized.
  • the NVA 100 In addition to compressing the graphics information, the NVA 100 also formats the graphics information for communication across the networks 30 and 40 .
  • the graphics information is formatted into a plurality of Internet protocol (IP) packets 72 , 74 , 76 , 78 , and 80 that are communicated across the networks 30 and 40 .
  • IP Internet protocol
  • the formatted data packets will comprise, among other things, an identification of the destination computer 50 to which the graphics information is to be transmitted. If the graphics information is transmitted in accordance with the Internet protocol, this destination identification may simply be an IP address. Since the Internet protocol is well-known, no discussion need be provided herein with regard to this protocol or the manner in which the graphics information is formatted in compliance with such a protocol. Furthermore, alternative protocols may be utilized.
  • the source computer 20 will communicate to the NVA 100 the address (or other identifier) of the destination computer 50 . Preferably, this communication occurs across the network 30 . The source computer 20 , then may send a short network communication to the NVA to inform it of the destination computer 50 . Similarly, a wireless interface may also be implemented in accordance with the scope and spirit of the invention.
  • the source computer 20 may have a direct connection to the NVA 100 , over which the destination information is communicated.
  • a cable may be connected between a serial port, a parallel port, a USB port, etc. of the source computer to an appropriate connector or port of the NVA 100 .
  • the destination computer 50 will comprise an appropriate network interface for receiving the data packets 72 , 74 , 76 , 78 , and 80 that carry the graphics information. It is also presumed that the destination computer 50 comprises appropriate software for evaluating the received packets and placing the collective graphics data into a frame buffer or other appropriate graphics memory for re-creating the video signal at the destination computer 50 . Of course, this reconstructed video signal will drive an appropriate monitor or display (or display window).
  • the implementation of this functionality at a destination computer is considered to be within the level of one possessing ordinary skill in the art, based upon the teachings herein, and therefore need not be described further herein.
  • a power source 90 is also illustrated in FIG. 1 .
  • the NVA 100 is provided as a stand-alone appliance (as illustrated in FIG. 1).
  • this power may be provided through an internal battery.
  • this power may be provided by a standard wall outlet and a power cord connected through an appropriate connector 104 provided on the NVA 100 .
  • Such an implementation (i.e., stand-alone) of the NVA allows for extremely convenient installation and setup.
  • a system in accordance with one embodiment may be set up essentially by connecting a cable between the source computer 20 and the NVA 100 , and connecting the NVA 100 (via a cable 106 ) to the nework 30 .
  • a minimal set of appropriate software drivers may also need to be installed on the computer 20 , as well as the destination computer 50 .
  • FIG. 2 is a block diagram illustrating certain components of the NVA 100 constructed in accordance with one embodiment.
  • the NVA 100 may be connected to the video source through a cable 22 , and likewise may be connected to a LAN 30 through cabling 106 .
  • the NVA 100 may comprise a video input circuit 110 .
  • This circuit may vary from embodiment to embodiment, but will generally comprise the necessary circuit components for receiving a video signal and formatting the signal for interface with other components within the NVA 100 .
  • the video input circuit 110 may comprise an analog-to-digital converter 112 .
  • the video input circuit 110 may also comprise the necessary circuit components for storing discrete units of the composite video signal into a memory 140 .
  • the particular implementation of the video input circuit is not deemed to be a limitation on the scope and spirit of the present invention, and therefore is not described herein, as circuit designers or persons skilled in the art will appreciate how to design and/or implement a video input circuit consistent with the teachings herein.
  • the NVA 100 also comprises a video compression circuit 120 .
  • the video compression circuit 120 may operate to read video data stored in the memory 140 and compress that data for more efficient communication over the networks 30 and 40 .
  • a compression circuit 120 may be implemented using general-purpose hardware components, such as a digital signal processor or microprocessor, and appropriate software for controlling the operation of the hardware components.
  • the video compression circuit 120 may be implemented using dedicated hardware or an application-specific integrated circuit (ASIC) that is specially designed for carrying out the compression function.
  • ASIC application-specific integrated circuit
  • any of a variety of compression algorithms may be utilized in connection with the invention. Indeed, certain embodiments of the present invention may not compress the video data at all, thereby eliminating the video compression circuit 120 from the NVA 100 .
  • the NVA 100 comprises a network interface circuit 130 .
  • the network interface circuit 130 is designed to format the video data for communication over the networks 30 and 40 .
  • the network interface circuit 130 may be configured to packetize the video data (e.g., in accordance with the Internet protocol).
  • the network interface circuit 130 may also be configured to interface with the video compression circuit 120 and/or the memory 140 .
  • the network interface circuit 130 may be configured to operate “on the fly,” receiving data output from the video compression circuit 120 , perform the appropriate formatting of the data, and direct it to the network.
  • the video compression circuit 120 may operate to compress the video data and write the compressed video data back into the memory 140 .
  • the network interface circuit 130 may be configured to read the video data from the memory 140 , then format that data and direct it to the network.
  • one embodiment of the NVA 100 may be configured to transmit video data in an uncompressed format.
  • the graphics information may be directed from the video input circuit 110 directly to the network interface 130 for formatting and transmission.
  • the graphics information may be directed from the video input circuit 110 to the memory 140 .
  • the network interface circuit 130 may then read data directly from the memory 140 , and then format the video data for transmission.
  • other implementations and embodiments of the NVA 100 may be readily appreciated from the discussion herein, and may be so constructed.
  • FIG. 3 is a flowchart illustrating one embodiment 170 of such a method.
  • a video signal (originated by a source computer) is received at a video input (step 172 ). Thereafter, if the video signal is an analog signal, then it is converted into digital form (step 173 ). Since the step is optional, depending upon the format of the video signal, it has been illustrated in FIG. 3 in dashed outline. Thereafter, the received video signal may be stored in memory (step 174 ).
  • numerous discrete data units comprising the video signal are stored in memory.
  • the stored video data is compressed using any of a variety of compression algorithms (step 176 ).
  • the compressed video data is formatted for communication over one or more networks (step 177 ).
  • the compressed video data is transmitted over the one or more networks to a remote, destination computer (step 178 ).
  • FIG. 3 a flowchart of FIG. 3 is presented purely for purposes of illustrating one embodiment.
  • the video data may not be stored in memory, but rather may be received, formatted for network communication, and communicated over the one or more networks “on the fly.”
  • Such a real-time implementation may be realized whether the video data is compressed or uncompressed.
  • the destination computer must be equipped with appropriate software for reconstructing the original video signal.
  • the appropriate decompression software and/or hardware must be resident on the destination computer. Implementation details such as these will be readily apparent and understood by persons skilled in the art, and therefore need not be described further herein.
  • FIG. 4 illustrates an alternative embodiment of the present invention.
  • the embodiment of FIG. 4 is similar to the embodiment illustrated in FIG. 1, except that an additional source computer 25 and an additional destination computer 55 are shown. Where appropriate, the reference numerals used in FIG. 1 have been preserved.
  • the second source computer 25 may be connected to the NVA 200 through conventional video cabling 26 .
  • the NVA 200 of this embodiment may comprise an additional connector or port (not shown) for receiving the additional video cable 26 . Consistent with the concepts and teachings of the present invention, additional connectors or ports may be provided on the NVA 200 so that additional (not shown) graphics sources may be accommodated.
  • an additional destination computer 55 may be provided.
  • the graphics information from the first source computer 20 may be destined for a first remote computer 50 .
  • the NVA 200 may operate to convert the video signal carried on cable 22 into appropriate IP packets 72 , 74 , 76 , 78 , and 80 that may be communicated over the networks 30 and 40 to the remote computer 50 .
  • the NVA 200 may operate to convert the video signal carried on cable 26 into appropriate IP packets 71 , 73 , 75 , 77 , and 79 that may be communicated over networks 30 and 40 to the remote computer 55 .
  • remote computer 55 may be configured to reconstruct the original video image embodied in the video signal carried on cable 26 . This video image reconstruction may operate as previously described in connection with the destination computer 50 .
  • FIG. 5 is a block diagram similar to the diagram of FIG. 2, which illustrates certain principal components that may be provided within the NVA 200 .
  • a video input circuit 210 is provided to receive the video signal carried on cable 22 .
  • the video input circuit 210 may comprise an analog additional converter 212 , which may operate to convert an analog video signal into digital values.
  • the video compression circuit 220 , network interface 230 , and memory 240 may also be provided and configured to operate like the corresponding components described in FIG. 2.
  • a second video input circuit 250 may be provided and configured to receive the video signal carried on cable 26 .
  • the video input circuit 250 may comprise an analog-to-digital converter 252 .
  • a second video compression circuit 260 and a second network interface 270 may also be provided within NVA 200 .
  • the function or operation of these components will be similar to the video input circuit 110 , video compression circuit 120 , and network interface circuit 130 , that were described in connection with FIG. 2. It will be appreciated that, although these components have been illustrated separately for purposes of simplifying the discussion herein, the actual implementation may be designed so that certain components are shared.
  • the NVA 200 may be designed with a single network interface circuit that is designed and configured with the remainder of the circuitry to handle network communications for both video streams.
  • FIG. 6 is a diagram similar to the diagram of FIG. 1, but illustrating yet another embodiment. Where appropriate, the reference numerals used in FIG. 1 have been preserved.
  • the NVA 300 may be integrated within the computer 20 containing the graphics source.
  • the NVA 300 may be provided on a circuit card that may be plugged directly into, for example, a motherboard of the computer 20 .
  • the NVA 300 may utilize a network interface provided as a part of computer 20 (or a network card that may be plugged into the computer—e.g., an Ethernet card).
  • the NVA 300 could receive its power supply signals directly from the power supply of the computer 20 .
  • an embodiment such as that illustrated in FIG. 6 may require additional software drivers to be installed on the computer 20 to appropriately recognize and/or configure the circuit card 300 .
  • the structure and operation of the circuitry provided on the NVA 300 may be substantially the same as that described in connection with NVA 100 and 200 .
  • FIGS. 1, 4, and 6 show diagrams of how the invention could be used in a network topology.
  • the NVA may compress and convert the video output into a sequence of IP (Internet Protocol) packets or some other network protocol.
  • IP Internet Protocol
  • the video output may be a digital signal as in DVI, or it may be an analog video signal.
  • the IP packets may be sent to a destination computer node on the network.
  • the destination computer node 50 may receive the IP packets from the NVA 100 , 200 , or 300 , decompress them, assemble the image, and then display the results in a window on a display device of the computer 50 .
  • the decompression, assembly, and display of the image can be implemented in a software client written as a JAVA program, JAVA applet, native implementation, or some other method running on the destination computer node 50 , consistent with the invention.
  • the software client may optionally control the NVA 100 , 200 , or 300 by configuring the refresh rate, window size on the destination computer node 50 , or other parameters. Additional hardware to assist in this process may also be used on the destination computer node 50 . Such additional hardware will be understood by persons skilled in the art from the teachings provided herein, and need not be described further.
  • RGB-to-IP red green blue-to-IP
  • This converter e.g., the NVA 100 , 200 , or 300
  • the NVA may be configured to transmit the same graphics information to a plurality of destination computers.
  • the NVA 100 , 200 , or 300 may comprise one or more video input ports, either analog or digital, one or more DSPs or similar low-cost, high-performance embedded controller, CPU or custom chip, one or more Ethernet ports or similar network interface ports, RAM, and other miscellaneous parts.
  • the video input port is connected to a video output of the graphics port on the source computer node 20 .
  • the CPU or DSP preferably compresses the video signal and encapsulates the data in IP or other network protocol. The packets are then sent out the network interface port.
  • the appliance may comprise other ports such as RS- 232 or power.
  • the CPU or DSP could also be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an embedded CPU, or some combination or variant of such devices.
  • FPGA field programmable gate array
  • ASIC application specific integrated circuit
  • embedded CPU or some combination or variant of such devices.
  • Another advantage is that no additional computational overhead on the source computer node 20 is required.
  • certain traditional techniques involve significant computation in the color space coalescing conversion (e.g., the application of color look-up tables to window IDs) and compression stage. These significant computations are avoided in the system described herein.
  • the functionality embodied in the NVA may be integrated onto a peripheral component interface (PCI) based board that plugs into the source computer node.
  • PCI peripheral component interface
  • the PCI board could accept video input from the video output of the graphics device.
  • a DSP, FPGA, or ASIC on the PCI board would compress and convert the data to some IP-based protocol.
  • the IP would be sent on a network interface port.
  • the functionality embodied in the NVA may be embodied into a block of code, such as Verilog, and sold to a graphics chip design company to be implemented in the graphics adapters.
  • the present invention is directed to a system and method for communicating graphics information from a source computer to one or more destination computers over a network.
  • One distinct advantage of the present invention is that the communication of graphics information is bandwidth efficient, in that the preferred embodiment is configured to transmit only the portions of the graphics information that change in subsequent video frames (after a first video frame has been communicated).
  • the discussion will assume that an entire frame of video information from a previous frame has been stored in a frame buffer memory.
  • a portion e.g., a Window
  • a Window may be stored. For example, perhaps only a Window of information is stored. This Window, as in the case of an entire frame, may be partitioned into tiled, which are compared against corresponding tiles of previous frames. Implementation of such an embodiment will be appreciated by persons skilled in the art, and need not be separately described herein.
  • FIGS. 1 - 6 certain components and features described in connection with FIGS. 1 - 6 may be omitted for purposes of simplifying the description and emphasizing certain components that are unique to the preferred embodiments. However, it should be appreciated by persons skilled in the art that the steps or components described below will preferably be integrated into an operating environment similar to that described in connection with FIGS. 1 - 6 .
  • FIG. 7 is a system-level block diagram similar to the diagram of FIG. 1. Indeed, the reference numerals used to denote the various system components in FIG. 1 have generally been preserved in FIG. 7, where appropriate.
  • the NVA 400 of FIG. 7 is preferably configured to operate differently than the corresponding NVA 100 illustrated in FIG. 1. Like the NVA 100 of FIG. 1, the NVA 400 is preferably configured to receive a first frame of graphics information, compress and packetize that frame of graphics information, and transmit the packets 472 , 474 , 476 , 478 and 480 to a destination computer 50 .
  • the NVA 400 is preferably configured to achieve extremely efficient bandwidth utilization for the transmission of graphics information of subsequent video frames.
  • the NVA 400 is preferably configured to evaluate a plurality of discrete portions (or blocks) of each video frame, and compare each portion with the corresponding portion of the previous video frame. If the NVA 400 determines that there is no change in a given portion of a video frame, then that portion need not be transmitted to the remote computer 50 . Instead, the NVA 400 is preferably configured to compress and transmit graphics information of only those portions of the video frame that have changed since the previous video frame. In one embodiment, each video frame may be effectively partitioned into 16,384 blocks. As mapped to a video screen, these blocks may comprise an array of 128 blocks by 128 blocks.
  • each block has a corresponding resolution of 10 pixels by 8 pixels.
  • a number of different block sizes may be chosen based upon system resources, the nature of the graphics information to be transmitted, and other factors.
  • the NVA 400 may be configured to dynamically vary the block size, depending upon the number of blocks that are observed to have changing video data from frame to frame. It will be appreciated that, for most video applications, a tremendous amount of bandwidth savings is achieved by the present invention. It will be further appreciated that, consistent with the scope and spirit of the invention, multiple blocks may be combined together to form a larger region, which could be compressed.
  • the packets 472 , 474 , 476 , 478 , and 480 will preferably differ from the corresponding packets illustrated in FIG. 1.
  • the packets of data transmitted by the NVA 400 of the preferred embodiment preferably comprise an identification of the block (e.g., block number) within the video frame (as well as the corresponding data) that is being transmitted, as this information will be utilized by the destination computer 50 for updating its display.
  • a given packet e.g. packet 472
  • the communication over the network 40 is preferably transmitted in accordance with the Internet Protocol.
  • the destination computer may be configured to transmit acknowledgements back to the NVA 400 , to better ensure proper synchronization between the two.
  • FIG. 8 is a flowchart illustrating the top-level functional operation of the NVA 400 .
  • the NVA 400 is preferably configured to receive a first frame of graphics information from the source computer 20 and store that graphics information in a memory. This graphics information is “grabbed” from the video port of a graphics card, which information is generated by the display refresh unit of the computer graphics card.
  • the NVA 400 receives a first full frame of graphics information and stores it in an internal memory (step 502 ).
  • This full frame of graphics information is preferably compressed (compression logic illustrated in FIGS. 2 and 5), packetized, and transmitted over the network 40 to the destination computer(s) 50 (step 504 ).
  • the NVA 400 receives a subsequent frame of graphics information (step 506 ).
  • this subsequent frame of graphics information is stored in a temporary memory 430 (see FIG. 10).
  • the NVA 400 is then configured to compare discrete portions or blocks of the frame of graphics information stored in the temporary memory 430 with the corresponding portions or blocks of the graphics information stored in the memory 420 (step 508 ). For each portion or block of the video frame that is identified to be different from the corresponding portion or block in the frame buffer memory 420 , the NVA packetizes and transmits that block to the destination computer (step 510 ).
  • the NVA 400 may be configured to transmit each block of graphics information, if it is determined that any change at all exists from the previously stored block. In certain embodiments, however, it may be desirable to set some threshold of change. Specifically, the NVA 400 may be configured to evaluate the degree or amount in which a current block has changed from a previously stored block, and transmit the current block only if the graphics information has changed by some predetermined amount. The implementation of such an embodiment will vary from application to application and be predicated upon design constraints and objectives. Finally, the NVA 400 is configured to overwrite blocks within the frame buffer memory 400 with corresponding blocks stored in the temporary memory 430 that reflect or embody a change in the graphics information (step 512 ).
  • the NVA 400 may be configured to simply overwrite the entire frame buffer memory 420 with the contents of the temporary memory 430 .
  • such an approach may not be preferred as it would require additional bus transactions for the transfer of duplicative data from the temporary memory 430 to the frame buffer memory 420 .
  • multiple blocks could be combined together into a larger region that could be compressed.
  • An alternative approach may be to compare the blocks or tiles of the previous frame with the current frame, as the current frame is grabbed. This approach may be described as:
  • FIG. 9 is a flowchart illustrating the top-level functional operation of a destination (or client) computer 50 .
  • a destination computer 50 is configured to receive a first full frame of graphics information and store that graphics information in the frame buffer of the graphics cards (step 520 ).
  • the destination computer 50 is configured with the appropriate software drivers for interfacing between the components of the present invention and the graphics card installed in the destination computer 50 .
  • the graphics card on the destination computer 50 will retrieve information from the frame buffer on the graphics card and render that graphics information on a display.
  • the graphics card of the destination computer 50 will continue to accordingly update the display. Therefore, upon receipt of a first frame of graphics information by the destination computer, the destination computer will be configured to thereafter receive graphics information for portions or blocks of subsequent video frames (step 522 ). These blocks of subsequent graphics information are decompressed (step 524 ) in accordance with an appropriate algorithm (i.e., a decompression algorithm that is reciprocal to the compression embodied by the NVA 400 ). Thereafter, the destination computer 50 is configured to override the frame buffer blocks or portions that correspond to the received frame buffer blocks or portions. As the graphics card installed in the destination computer 50 , in accordance with its conventional operation, continues to render data from the frame buffer to the display screen, this changed data will automatically and continually be reflected on the display.
  • an appropriate algorithm i.e., a decompression algorithm that is reciprocal to the compression embodied by the NVA 400
  • the NVA 400 may comprise a video input circuit 410 that is configured to receive video input on cable 22 , digitize the graphics information if appropriate, and appropriately store a first frame of graphics information into a frame buffer memory 420 .
  • the frame buffer memory 420 may be configured to mimic the organization of a conventional frame buffer memory as contained on a graphics card.
  • the video input circuit 410 will preferably comprise the appropriate circuitry and/or software logic for identifying the start of a video frame and capturing the appropriate information for storing within the frame buffer memory 420 .
  • the frame buffer memory 420 is effectively partitioned into a plurality of blocks that are separately and individually analyzed and updated with corresponding blocks of subsequent video frames.
  • the NVA 400 preferably comprises a temporary memory 430 in which a current video frame (i.e., a subsequent video frame, after the first video frame is received and stored in the frame buffer memory 420 ) is temporarily stored.
  • the NVA 400 further comprises comparison logic for comparing blocks stored within the temporary memory 430 with corresponding blocks stored within the frame buffer memory 420 .
  • the comparison logic 440 may be implemented in hardware, software, or an appropriate combination of the two, depending upon design constraints and objectives.
  • the comparison logic 440 may be configured to identify those blocks of graphics information that differ from the corresponding blocks stored in the frame buffer memory 420 , and forward those blocks for transmission to the destination computer 50 . For each block of graphics information that is identified by the comparison logic 440 to have changed, the comparison logic 440 may be further configured to overwrite the corresponding block of the frame buffer memory 420 with the new, or changed, graphics information.
  • FIG. 10 illustrates a functional block 450 labeled as “packetize and transmit.”
  • This block functions to receive the changed graphics information from the comparison logic 440 , packetize that information, and transmit the information over the network 40 to the destination computer 50 .
  • block 450 is preferably configured to compress each block of graphics information before transmission, in order to minimize bandwidth consumption over the network 40 .
  • FIG. 11 is a block diagram illustrating certain components within the destination computer 50 .
  • the network interface 602 may be provided in connection with an Ethernet card or other conventional network interface that are well-known.
  • the network interface 602 is configured to receive data transmitted over the network 40 .
  • the frame buffer memory 604 may be provided on a graphics card installed within the destination computer 50 .
  • the video driver 606 and display 608 may be conventional components of the graphics and display system of the destination computer 50 .
  • the destination computer 50 may comprise receive circuitry 620 , which is configured to receive graphics information from the network interface 602 (i.e., the graphics information transmitted by the NVA 400 ).
  • the receive circuitry 620 may comprise logic 622 for decompressing graphics information, if the NVA 400 is configured to first compress information before transmission over the network.
  • the receive circuitry 620 may comprise a memory 624 for temporarily storing graphics information received from the network.
  • the receive circuitry 620 will overwrite information stored within the frame buffer memory 604 with new graphics information that is received over the network 40 .
  • the receive circuitry 620 may receive an entire frame of graphics information that it correspondingly writes into the frame buffer memory 604 .
  • the receive circuitry 620 may receive only blocks of graphics information that it writes into corresponding locations of the frame buffer memory 604 .
  • the NVA 400 of the preferred embodiment organizes packets of data transmitted over the network to identify, if appropriate, particular blocks of the frame buffer to which the data corresponds.
  • the receive circuitry 620 will, accordingly, be configured to identify such block descriptors and write the received information into the appropriate location of the frame buffer 604 . If an entire frame of graphics information is transmitted from the NVA 400 to the destination computer 50 , the receive circuitry may be configured to readily receive and overwrite the entire frame buffer 604 with the frame of graphics information. Alternatively, if only particular blocks of graphics information are transmitted from the NVA 400 to the destination computer 50 , the receive circuitry 620 is configured to overwrite only those corresponding blocks.
  • FIG. 12 is a flowchart illustrating the top-level operation of the NVA 400 constructed in accordance with one such alternative embodiment.
  • the NVA 400 may be constructed to evaluate individual blocks of graphics information as they are received from the source computer 20 , without first receiving an entire frame of subsequent graphics information.
  • Such an alternative configuration would advantageously reduce the size of the temporary memory 430 (see FIG. 10).
  • receive a first full frame of graphics information and store that frame into the frame buffer memory 420 (step 702 ). That first full frame of graphics information would then be packetized and transmitted over the network (step 704 ), as previously described.
  • the NVA 400 may be configured to evaluate the graphics information “on the fly” as it is received from the source computer 20 .
  • the NVA may be configured to receive a portion of the graphics information ( 706 ) and compare the received portion with a corresponding portion, or block, stored in the frame buffer memory 420 (step 708 ). If this portion is different than the previously stored portion, the NVA 400 may be configured to packetize and transmit this portion over the network 40 to the destination computer 50 (step 710 ). Finally, and if this portion of the video frame differed from the corresponding portion stored in the frame buffer memory 420 , the NVA 400 may be configured to overwrite the corresponding memory portion of frame buffer memory 420 with the newly-received portion of graphics information (step 712 ). This process is repeated for all portions of a current video frame, and cyclically repeated for subsequent video frames.
  • the NVA compresses the block in the current video frame
  • the NVA transmits the block to the destination computer over the network

Abstract

The present invention is generally directed to a system and method for communicating video across one or more computer networks. Broadly, the system comprises a frame buffer memory for storing and maintaining a previous frame of graphics information, a temporary memory configured to store at least a portion of a current frame of graphics information, comparison logic for comparing a portion of the current frame of graphics information with a corresponding portion of the previous frame, and transmission logic for transmitting the portion of the current frame to a destination computer, if the comparison logic determines that the portion of the current frame of graphics information differs from the corresponding portion of the previous frame by more than a predetermined measure. Similarly, a method is provided that stores a frame of graphics information obtained from a video signal, receives at least a portion of a current frame of graphics information, compares a portion of the current frame of graphics information with a corresponding portion of the stored frame of graphics information. Then, if the compared portion of the current frame of graphics information differs by at least a predetermined amount from the corresponding portion of the stored graphics information, the method transmits the compared portion of the current frame of graphics information to a destination computer and overwrites the corresponding portion of the stored graphics information with the compared portion of the current frame of graphics information.

Description

    BACKGROUND OF THE INVENTION
  • 1. Field of the Invention [0001]
  • The present invention generally relates to networked computer systems, and more particularly to a system and method for communicating graphics over a network. [0002]
  • Discussion of the Related Art As is known, it is often desired to display computer graphics information, that is generated on one computer, on a remote computer display. Sometimes it is desired to provide this type of remote graphics display across a local area network (LAN), and sometimes it is desired to provide this type of remote display across a wide area network (WAN), such as the Internet. [0003]
  • Various approaches have been undertaken to implement this goal. Generalizing, the approaches of prior art systems involve formatting graphics data into an appropriate network protocol and transmitting the data directly from the source computer. One approach involves reading of the frame-buffer back into a system memory, compressing and assembling IP (Internet Protocol) packets, and then sending the IP packets to the destination computer node. Unfortunately, the process of reading the frame-buffer back into system memory and sending the IP packets to the destination computer node involve significant computation in the color space computation/conversion and compression stage, which significantly impact the input/output (I/O) and memory sub-systems, as well as the central processing unit (CPU) and networking sub-system. [0004]
  • Another approach of prior art systems involves the reception or collection of commands and data being sent to, for example, a graphics card. In certain systems this data is collected, and communicated to a remote client. While this approach may be effective to communicate graphics to a remote client, the remote client generally must have all the capabilities to render graphics as the computer sending the commands and data. [0005]
  • Another disadvantage of prior art systems is that they generally require operating system-dependent (OS-dependent) code that operates at the device driver layer, which must be written to control the devices. This OS-dependent code varies from system to system. [0006]
  • A further problem with many prior art systems relates to bandwidth consumption, particularly when communicating graphics information. As is known, when transmitting graphics, successive frames of data may be transmitted every {fraction (1/60)}th of a second (to support a 60 Hz refresh rate), or at some other rate (if other than 60 Hz refresh rate). Transmitting entire frames of graphics information every {fraction (1/60)}th of a second may exceed the bandwidth limitations of many networks, particularly when communicating over a wide area network. [0007]
  • SUMMARY OF THE INVENTION
  • The present invention is generally directed to a system and method for communicating graphics across one or more networks. Broadly, the system comprises a frame buffer memory for storing and maintaining a previous frame of graphics information, a temporary memory configured to store at least a portion of a current frame of graphics information, comparison logic for comparing a portion of the current frame of graphics information with a corresponding portion of the previous frame, and transmission logic for transmitting the portion of the current frame to a destination computer, if the comparison logic determines that the portion of the current frame of graphics information differs from the corresponding portion of the previous frame by more than a predetermined measure. [0008]
  • In accordance with another embodiment, a method is provided that stores a frame of graphics information obtained from a video signal, receives at least a portion of a current frame of graphics information, compares a portion of the current frame of graphics information with a corresponding portion of the stored frame of graphics information. Then, if the compared portion of the current frame of graphics information differs by at least a predetermined amount from the corresponding portion of the stored graphics information, the method transmits the compared portion of the current frame of graphics information to a destination computer and overwrites the corresponding portion of the stored graphics information with the compared portion of the current frame of graphics information.[0009]
  • DESCRIPTION OF THE DRAWINGS
  • The accompanying drawings incorporated in and forming a part of the specification, illustrate several aspects of the present invention, and together with the description serve to explain the principles of the invention. In the drawings: [0010]
  • FIG. 1 is a block diagram illustrating a system environment for accommodating one embodiment of the invention; [0011]
  • FIG. 2 is a block diagram of a network video apparatus (NVA); [0012]
  • FIG. 3 is a flowchart illustrating the top-level operation of an NVA; [0013]
  • FIG. 4 is a block diagram similar to FIG. 1, but illustrating an alternative system configuration; [0014]
  • FIG. 5 is a block diagram, similar to the diagram of FIG. 2, but illustrating components of a NVA constructed in accordance with the embodiment illustrated in FIG. 4; [0015]
  • FIG. 6 is a block diagram similar to FIG. 1, but illustrating a system constructed in accordance with another embodiment; [0016]
  • FIG. 7 is a block diagram illustrating a system constructed in accordance with one embodiment of the present invention; [0017]
  • FIG. 8 is a flowchart illustrating the top-level operation of an NVA in accordance with one embodiment of the invention; [0018]
  • FIG. 9 is a flowchart illustrating the top-level operation of a destination computer constructed in accordance with one embodiment of the invention; [0019]
  • FIG. 10 is a block diagram illustrating certain components of an NVA constructed in accordance with an embodiment of the invention; [0020]
  • FIG. 11 is block diagram illustrating certain components of a destination computer constructed to receive graphics information from an NVA constructed in accordance with one embodiment of the invention; and [0021]
  • FIG. 12 is a flowchart illustrating the top-level operation of an NVA constructed in accordance with an alternative embodiment of the invention. [0022]
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
  • Turning now to the drawings, FIGS. [0023] 1-6 are diagrams and flowcharts illustrating certain environmental elements and features, in which the present invention may reside or operate. FIGS. 7-12 are diagrams and flowcharts that more particularly illustrate preferred embodiments of the present invention.
  • Reference is now made to FIG. 1, which is a diagram illustrating certain principal components of a [0024] system 10 in which the present invention may be used. More particularly, the system 10 comprises components that cooperate to communicate graphics information from a source computer 20 to a destination computer 50 across one or more networks. In the illustrated embodiment, the system 10 comprises both a local area network (LAN) 30 as well as a wide area network (WAN) 40. In many configurations, the WAN 40 will be the Internet.
  • To facilitate the communication of graphics information from a [0025] source computer 20 to a destination computer 50, a novel NVA 100 is provided. Various embodiments of such a NVA 100 will be described in more detail herein (e.g., FIGS. 2 and 5). In short, the NVA 100 operates by receiving a video signal at an input, formatting the video signal for network communication, and outputting the formatted video signal across one or more networks. More particularly, a standard or conventional video signal, such as a video signal generated by a video graphics card, may be connected to the NVA 100. This connection may be made through conventional cabling 22 and a connector 102 coupled to an input of the NVA 100.
  • The NVA [0026] 100 may store the video signal in an internal memory. The video signal may be either a digital video signal, such as a Digital Visual Interface (DVI) compliant signal, or may be an analog video signal. If the video signal carried on cable 22 is an analog signal, then the NVA 100 will perform an appropriate analog-to-digital conversion, before storing the video signal in memory. Preferably, a compression is performed of the graphics information stored in memory, before communicating the graphics information over the networks. Such a compression can significantly reduce the amount of data that is transmitted across the networks 30 and 40. With regard to this compression, it will be appreciated that a variety of compression algorithms may be utilized.
  • In addition to compressing the graphics information, the NVA [0027] 100 also formats the graphics information for communication across the networks 30 and 40. In one embodiment, the graphics information is formatted into a plurality of Internet protocol (IP) packets 72, 74, 76, 78, and 80 that are communicated across the networks 30 and 40. The formatted data packets will comprise, among other things, an identification of the destination computer 50 to which the graphics information is to be transmitted. If the graphics information is transmitted in accordance with the Internet protocol, this destination identification may simply be an IP address. Since the Internet protocol is well-known, no discussion need be provided herein with regard to this protocol or the manner in which the graphics information is formatted in compliance with such a protocol. Furthermore, alternative protocols may be utilized.
  • Of course, as will be appreciated by persons skilled in the art, the [0028] source computer 20 will communicate to the NVA 100 the address (or other identifier) of the destination computer 50. Preferably, this communication occurs across the network 30. The source computer 20, then may send a short network communication to the NVA to inform it of the destination computer 50. Similarly, a wireless interface may also be implemented in accordance with the scope and spirit of the invention.
  • Alternatively, the [0029] source computer 20 may have a direct connection to the NVA 100, over which the destination information is communicated. For example, a cable may be connected between a serial port, a parallel port, a USB port, etc. of the source computer to an appropriate connector or port of the NVA 100.
  • Regardless of whether the destination address is communicated over the [0030] LAN 30 or through a direct connection between the source computer 20 and the NVA 100, it should be appreciated that only a minimal set of software drivers will be required to be installed on the source computer 20. It should be further appreciated that the addition and installation of the NVA to the LAN 30 will be a rather simple task.
  • It is presumed, for purposes of this discussion, that the [0031] destination computer 50 will comprise an appropriate network interface for receiving the data packets 72, 74, 76, 78, and 80 that carry the graphics information. It is also presumed that the destination computer 50 comprises appropriate software for evaluating the received packets and placing the collective graphics data into a frame buffer or other appropriate graphics memory for re-creating the video signal at the destination computer 50. Of course, this reconstructed video signal will drive an appropriate monitor or display (or display window). The implementation of this functionality at a destination computer is considered to be within the level of one possessing ordinary skill in the art, based upon the teachings herein, and therefore need not be described further herein.
  • Also illustrated in FIG. 1 is a [0032] power source 90. In one embodiment of the present invention, the NVA 100 is provided as a stand-alone appliance (as illustrated in FIG. 1). Of course, if the NVA 100 is a stand-alone appliance, then it will need to be provided power for operation. In one embodiment, this power may be provided through an internal battery. In another embodiment, this power may be provided by a standard wall outlet and a power cord connected through an appropriate connector 104 provided on the NVA 100. These features, however, are not deemed to be limitations upon the broader scope and spirit of the present invention.
  • Such an implementation (i.e., stand-alone) of the NVA allows for extremely convenient installation and setup. In this regard, a system in accordance with one embodiment may be set up essentially by connecting a cable between the [0033] source computer 20 and the NVA 100, and connecting the NVA 100 (via a cable 106) to the nework 30. A minimal set of appropriate software drivers may also need to be installed on the computer 20, as well as the destination computer 50.
  • Reference is now made to FIG. 2, which is a block diagram illustrating certain components of the [0034] NVA 100 constructed in accordance with one embodiment. As mentioned in connection with FIG. 1, the NVA 100 may be connected to the video source through a cable 22, and likewise may be connected to a LAN 30 through cabling 106. Internally, the NVA 100 may comprise a video input circuit 110. This circuit may vary from embodiment to embodiment, but will generally comprise the necessary circuit components for receiving a video signal and formatting the signal for interface with other components within the NVA 100. By way of example, if the video signal carried on cable 22 is an analog signal, then the video input circuit 110 may comprise an analog-to-digital converter 112. The video input circuit 110 may also comprise the necessary circuit components for storing discrete units of the composite video signal into a memory 140. The particular implementation of the video input circuit is not deemed to be a limitation on the scope and spirit of the present invention, and therefore is not described herein, as circuit designers or persons skilled in the art will appreciate how to design and/or implement a video input circuit consistent with the teachings herein.
  • In a preferred embodiment, the [0035] NVA 100 also comprises a video compression circuit 120. Functionally, the video compression circuit 120 may operate to read video data stored in the memory 140 and compress that data for more efficient communication over the networks 30 and 40. In the preferred embodiment, such a compression circuit 120 may be implemented using general-purpose hardware components, such as a digital signal processor or microprocessor, and appropriate software for controlling the operation of the hardware components. Of course, consistent with the invention, the video compression circuit 120 may be implemented using dedicated hardware or an application-specific integrated circuit (ASIC) that is specially designed for carrying out the compression function. As previously mentioned, any of a variety of compression algorithms may be utilized in connection with the invention. Indeed, certain embodiments of the present invention may not compress the video data at all, thereby eliminating the video compression circuit 120 from the NVA 100.
  • Finally, the [0036] NVA 100 comprises a network interface circuit 130. The network interface circuit 130 is designed to format the video data for communication over the networks 30 and 40. In this regard, the network interface circuit 130 may be configured to packetize the video data (e.g., in accordance with the Internet protocol). The network interface circuit 130 may also be configured to interface with the video compression circuit 120 and/or the memory 140. In this regard, the network interface circuit 130 may be configured to operate “on the fly,” receiving data output from the video compression circuit 120, perform the appropriate formatting of the data, and direct it to the network. Alternatively, the video compression circuit 120 may operate to compress the video data and write the compressed video data back into the memory 140. In such an embodiment, the network interface circuit 130 may be configured to read the video data from the memory 140, then format that data and direct it to the network.
  • Again, as mentioned above, one embodiment of the [0037] NVA 100 may be configured to transmit video data in an uncompressed format. In such embodiment, the graphics information may be directed from the video input circuit 110 directly to the network interface 130 for formatting and transmission. Alternatively, the graphics information may be directed from the video input circuit 110 to the memory 140. The network interface circuit 130 may then read data directly from the memory 140, and then format the video data for transmission. Of course, other implementations and embodiments of the NVA 100 may be readily appreciated from the discussion herein, and may be so constructed.
  • Having described the principal operation and structure of one embodiment, it should be appreciated that various methods may exist for communicating video signals from the [0038] source computer 20, across one or more networks 30 and 40 to a remote, destination computer 50. In this regard, reference is made to FIG. 3, which is a flowchart illustrating one embodiment 170 of such a method. In accordance with this illustrated embodiment, a video signal (originated by a source computer) is received at a video input (step 172). Thereafter, if the video signal is an analog signal, then it is converted into digital form (step 173). Since the step is optional, depending upon the format of the video signal, it has been illustrated in FIG. 3 in dashed outline. Thereafter, the received video signal may be stored in memory (step 174). In this regard, numerous discrete data units comprising the video signal (e.g., one frame of video) are stored in memory. In one embodiment, the stored video data is compressed using any of a variety of compression algorithms (step 176). Thereafter, the compressed video data is formatted for communication over one or more networks (step 177). Finally, the compressed video data is transmitted over the one or more networks to a remote, destination computer (step 178).
  • It should be appreciated that a flowchart of FIG. 3 is presented purely for purposes of illustrating one embodiment. As described herein, various alternative embodiments may be implemented as well. For example, in one embodiment the video data may not be stored in memory, but rather may be received, formatted for network communication, and communicated over the one or more networks “on the fly.” Such a real-time implementation may be realized whether the video data is compressed or uncompressed. Of course, the destination computer must be equipped with appropriate software for reconstructing the original video signal. Thus, if the source video signal is compressed prior to transmission over the network, then the appropriate decompression software and/or hardware must be resident on the destination computer. Implementation details such as these will be readily apparent and understood by persons skilled in the art, and therefore need not be described further herein. [0039]
  • Having described the principal structure and operation of one embodiment, reference is now made to FIG. 4, which illustrates an alternative embodiment of the present invention. In short, the embodiment of FIG. 4 is similar to the embodiment illustrated in FIG. 1, except that an [0040] additional source computer 25 and an additional destination computer 55 are shown. Where appropriate, the reference numerals used in FIG. 1 have been preserved. Like the source computer 20, the second source computer 25 may be connected to the NVA 200 through conventional video cabling 26. Of course, the NVA 200 of this embodiment may comprise an additional connector or port (not shown) for receiving the additional video cable 26. Consistent with the concepts and teachings of the present invention, additional connectors or ports may be provided on the NVA 200 so that additional (not shown) graphics sources may be accommodated.
  • Likewise, an [0041] additional destination computer 55 may be provided. In this regard, the graphics information from the first source computer 20 may be destined for a first remote computer 50. As described in connection with FIG. 1, the NVA 200 may operate to convert the video signal carried on cable 22 into appropriate IP packets 72, 74, 76, 78, and 80 that may be communicated over the networks 30 and 40 to the remote computer 50. In similar fashion, the NVA 200 may operate to convert the video signal carried on cable 26 into appropriate IP packets 71, 73, 75, 77, and 79 that may be communicated over networks 30 and 40 to the remote computer 55. Upon receiving these packets, remote computer 55 may be configured to reconstruct the original video image embodied in the video signal carried on cable 26. This video image reconstruction may operate as previously described in connection with the destination computer 50.
  • Reference is now made briefly to FIG. 5, which is a block diagram similar to the diagram of FIG. 2, which illustrates certain principal components that may be provided within the [0042] NVA 200. As described in connection with FIG. 2, a video input circuit 210 is provided to receive the video signal carried on cable 22. If appropriate, the video input circuit 210 may comprise an analog additional converter 212, which may operate to convert an analog video signal into digital values. The video compression circuit 220, network interface 230, and memory 240 may also be provided and configured to operate like the corresponding components described in FIG. 2.
  • In addition, a second [0043] video input circuit 250 may be provided and configured to receive the video signal carried on cable 26. Like the video input circuit 210, the video input circuit 250 may comprise an analog-to-digital converter 252. Also, a second video compression circuit 260 and a second network interface 270 may also be provided within NVA 200. The function or operation of these components will be similar to the video input circuit 110, video compression circuit 120, and network interface circuit 130, that were described in connection with FIG. 2. It will be appreciated that, although these components have been illustrated separately for purposes of simplifying the discussion herein, the actual implementation may be designed so that certain components are shared. For example, the NVA 200 may be designed with a single network interface circuit that is designed and configured with the remainder of the circuitry to handle network communications for both video streams.
  • Although not explicitly shown in FIG. 5, it should be appreciated that additional components may be comprised within the [0044] NVA 200 to accommodate additional video streams.
  • Reference is now made to FIG. 6, which is a diagram similar to the diagram of FIG. 1, but illustrating yet another embodiment. Where appropriate, the reference numerals used in FIG. 1 have been preserved. As illustrated in FIG. 6, rather than providing the [0045] NVA 100 or 200 as a separate or stand-alone device, it may be integrated within the computer 20 containing the graphics source. In this regard, the NVA 300 may be provided on a circuit card that may be plugged directly into, for example, a motherboard of the computer 20. In such an embodiment, the NVA 300 may utilize a network interface provided as a part of computer 20 (or a network card that may be plugged into the computer—e.g., an Ethernet card). In this way, separate network connections between the NVA 300 in the network 30 may not be required. In addition, the NVA 300 could receive its power supply signals directly from the power supply of the computer 20. However, it will be appreciated that an embodiment such as that illustrated in FIG. 6 may require additional software drivers to be installed on the computer 20 to appropriately recognize and/or configure the circuit card 300. Aside from these differences, the structure and operation of the circuitry provided on the NVA 300 may be substantially the same as that described in connection with NVA 100 and 200.
  • From the foregoing discussion, it should be appreciated that certain embodiments provide a [0046] NVA 100, 200, or 300 that is attached to the video output of a graphics adapter on a source computer node. FIGS. 1, 4, and 6 show diagrams of how the invention could be used in a network topology. The NVA may compress and convert the video output into a sequence of IP (Internet Protocol) packets or some other network protocol.
  • The video output may be a digital signal as in DVI, or it may be an analog video signal. The IP packets may be sent to a destination computer node on the network. The [0047] destination computer node 50 may receive the IP packets from the NVA 100, 200, or 300, decompress them, assemble the image, and then display the results in a window on a display device of the computer 50. The decompression, assembly, and display of the image can be implemented in a software client written as a JAVA program, JAVA applet, native implementation, or some other method running on the destination computer node 50, consistent with the invention. The software client may optionally control the NVA 100, 200, or 300 by configuring the refresh rate, window size on the destination computer node 50, or other parameters. Additional hardware to assist in this process may also be used on the destination computer node 50. Such additional hardware will be understood by persons skilled in the art from the teachings provided herein, and need not be described further.
  • Another way to think about this is, instead of a monitor being attached to the video output of the [0048] source computer node 20, an RGB-to-IP (red green blue-to-IP) converter may be attached (of course both may be attached to receive the same video signal). This converter (e.g., the NVA 100, 200, or 300) can compress and transmit the output to one or more destination IP addresses. In one embodiment, the NVA may be configured to transmit the same graphics information to a plurality of destination computers.
  • The [0049] NVA 100, 200, or 300 may comprise one or more video input ports, either analog or digital, one or more DSPs or similar low-cost, high-performance embedded controller, CPU or custom chip, one or more Ethernet ports or similar network interface ports, RAM, and other miscellaneous parts. The video input port is connected to a video output of the graphics port on the source computer node 20. The CPU or DSP preferably compresses the video signal and encapsulates the data in IP or other network protocol. The packets are then sent out the network interface port. The appliance may comprise other ports such as RS-232 or power. As should be appreciated by persons skilled in the art, the CPU or DSP could also be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an embedded CPU, or some combination or variant of such devices.
  • It will be appreciated from the foregoing that numerous advantages are realized by the system that has been described herein through several embodiments. One such advantage is platform independence. No low-level OS or device driver changes are required on the [0050] source computer node 20. Indeed, the system described herein is OS-independent and hardware-independent.
  • Another advantage is that no additional computational overhead on the [0051] source computer node 20 is required. As is known, certain traditional techniques involve significant computation in the color space coalescing conversion (e.g., the application of color look-up tables to window IDs) and compression stage. These significant computations are avoided in the system described herein.
  • The embodiments described herein should not be construed as limiting on the system, but rather should be construed as illustrative. Indeed, numerous variations of the embodiments specifically described herein may be implemented, consistent with the scope and spirit of the invention. For example, the functionality embodied in the NVA described herein may be integrated onto a graphics adapter either using a separate chip or within the graphics chip itself. One place where this may be implemented is the point that the random access memory digital-to-analog converter (RAMDAC) is traditionally located in a graphics chip. [0052]
  • Alternatively, the functionality embodied in the NVA may be integrated onto a peripheral component interface (PCI) based board that plugs into the source computer node. This would simplify the power requirements for the apparatus. The PCI board could accept video input from the video output of the graphics device. A DSP, FPGA, or ASIC on the PCI board would compress and convert the data to some IP-based protocol. The IP would be sent on a network interface port. [0053]
  • In yet another embodiment, the functionality embodied in the NVA may be embodied into a block of code, such as Verilog, and sold to a graphics chip design company to be implemented in the graphics adapters. [0054]
  • Having described various operating environments in which the present invention may reside, reference will now be made to preferred embodiments of the present invention. In this regard, and as summarized above, the present invention is directed to a system and method for communicating graphics information from a source computer to one or more destination computers over a network. One distinct advantage of the present invention is that the communication of graphics information is bandwidth efficient, in that the preferred embodiment is configured to transmit only the portions of the graphics information that change in subsequent video frames (after a first video frame has been communicated). For purposes of simplifying the discussion herein, to best illustrate the inventive aspects of the preferred embodiment, the discussion will assume that an entire frame of video information from a previous frame has been stored in a frame buffer memory. It should be appreciated, however, that consistent with the scope and spirit of the invention, only a portion (e.g., a Window) of a video frame may be stored. For example, perhaps only a Window of information is stored. This Window, as in the case of an entire frame, may be partitioned into tiled, which are compared against corresponding tiles of previous frames. Implementation of such an embodiment will be appreciated by persons skilled in the art, and need not be separately described herein. [0055]
  • In the illustrations and description of the preferred embodiments that follow, certain components and features described in connection with FIGS. [0056] 1-6 may be omitted for purposes of simplifying the description and emphasizing certain components that are unique to the preferred embodiments. However, it should be appreciated by persons skilled in the art that the steps or components described below will preferably be integrated into an operating environment similar to that described in connection with FIGS. 1-6.
  • Reference is now made to FIG. 7, which is a system-level block diagram similar to the diagram of FIG. 1. Indeed, the reference numerals used to denote the various system components in FIG. 1 have generally been preserved in FIG. 7, where appropriate. However, the [0057] NVA 400 of FIG. 7 is preferably configured to operate differently than the corresponding NVA 100 illustrated in FIG. 1. Like the NVA 100 of FIG. 1, the NVA 400 is preferably configured to receive a first frame of graphics information, compress and packetize that frame of graphics information, and transmit the packets 472, 474, 476, 478 and 480 to a destination computer 50. The NVA 400 is preferably configured to achieve extremely efficient bandwidth utilization for the transmission of graphics information of subsequent video frames.
  • In this regard, the [0058] NVA 400 is preferably configured to evaluate a plurality of discrete portions (or blocks) of each video frame, and compare each portion with the corresponding portion of the previous video frame. If the NVA 400 determines that there is no change in a given portion of a video frame, then that portion need not be transmitted to the remote computer 50. Instead, the NVA 400 is preferably configured to compress and transmit graphics information of only those portions of the video frame that have changed since the previous video frame. In one embodiment, each video frame may be effectively partitioned into 16,384 blocks. As mapped to a video screen, these blocks may comprise an array of 128 blocks by 128 blocks. In a common display system having a resolution of 1,280 pixels by 1,024 pixels, each block has a corresponding resolution of 10 pixels by 8 pixels. Consistent with the scope and spirit of the present invention, however, a number of different block sizes may be chosen based upon system resources, the nature of the graphics information to be transmitted, and other factors. Indeed, consistent with the invention, the NVA 400 may be configured to dynamically vary the block size, depending upon the number of blocks that are observed to have changing video data from frame to frame. It will be appreciated that, for most video applications, a tremendous amount of bandwidth savings is achieved by the present invention. It will be further appreciated that, consistent with the scope and spirit of the invention, multiple blocks may be combined together to form a larger region, which could be compressed.
  • As is also illustrated in FIG. 7, the [0059] packets 472, 474, 476, 478, and 480 will preferably differ from the corresponding packets illustrated in FIG. 1. In this regard, the packets of data transmitted by the NVA 400 of the preferred embodiment preferably comprise an identification of the block (e.g., block number) within the video frame (as well as the corresponding data) that is being transmitted, as this information will be utilized by the destination computer 50 for updating its display. It should be further appreciated that a given packet (e.g. packet 472) may contain an entire block of video data, multiple blocks of video data, or only a portion of a block of video data. In this regard, the communication over the network 40 is preferably transmitted in accordance with the Internet Protocol.
  • Of course, numerous implementation details have not been described herein. For example, in one embodiment, it may be desired to periodically retransmit an entire frame buffer, in the event that packets have been lost during transmission, or if the frame buffer memories from the [0060] NVA 400 and the destination computer 50 have become unsynchronized. It may likewise be desired to retransmit an entire frame buffer if a large amount of the display has changed from the previous screen, or if a certain change threshold has been reached. In such a situation, it may be more efficient to compress the entire display than to compress and transmit each block independently. In another embodiment, the destination computer may be configured to transmit acknowledgements back to the NVA 400, to better ensure proper synchronization between the two.
  • Having described an aspect of the preferred embodiment, reference is now made to FIG. 8, which is a flowchart illustrating the top-level functional operation of the [0061] NVA 400. In this regard, the NVA 400 is preferably configured to receive a first frame of graphics information from the source computer 20 and store that graphics information in a memory. This graphics information is “grabbed” from the video port of a graphics card, which information is generated by the display refresh unit of the computer graphics card.
  • In keeping with the description of FIG. 8, the [0062] NVA 400 receives a first full frame of graphics information and stores it in an internal memory (step 502). This full frame of graphics information is preferably compressed (compression logic illustrated in FIGS. 2 and 5), packetized, and transmitted over the network 40 to the destination computer(s) 50 (step 504). Thereafter, the NVA 400 receives a subsequent frame of graphics information (step 506). In one embodiment, this subsequent frame of graphics information is stored in a temporary memory 430 (see FIG. 10). The NVA 400 is then configured to compare discrete portions or blocks of the frame of graphics information stored in the temporary memory 430 with the corresponding portions or blocks of the graphics information stored in the memory 420 (step 508). For each portion or block of the video frame that is identified to be different from the corresponding portion or block in the frame buffer memory 420, the NVA packetizes and transmits that block to the destination computer (step 510).
  • In this regard, and depending upon the particular application, the [0063] NVA 400 may be configured to transmit each block of graphics information, if it is determined that any change at all exists from the previously stored block. In certain embodiments, however, it may be desirable to set some threshold of change. Specifically, the NVA 400 may be configured to evaluate the degree or amount in which a current block has changed from a previously stored block, and transmit the current block only if the graphics information has changed by some predetermined amount. The implementation of such an embodiment will vary from application to application and be predicated upon design constraints and objectives. Finally, the NVA 400 is configured to overwrite blocks within the frame buffer memory 400 with corresponding blocks stored in the temporary memory 430 that reflect or embody a change in the graphics information (step 512). In an alternative embodiment, the NVA 400 may be configured to simply overwrite the entire frame buffer memory 420 with the contents of the temporary memory 430. However, such an approach may not be preferred as it would require additional bus transactions for the transfer of duplicative data from the temporary memory 430 to the frame buffer memory 420. As previously mentioned, multiple blocks could be combined together into a larger region that could be compressed.
  • It should be appreciated that, consistent with the scope and spirit of the present invention, there are a variety of way to implement the manner in which successive video frames are stored, compared, and overwritten. In addition to the other manners described herein, one alternative approach is to “grab” the current frame and compare it to previous the stored frame, and classify individual blocks or tiles as either changed or not changed. After processing the current frame, the current frame then becomes the stored (or previous) frame and the previous frame becomes the current frame. Under this approach, the current frame does not have to be copied from a temporary buffer to a “previous” buffer. [0064]
  • The logic to implement this may be described as: [0065]
  • for all incoming pixels [0066]
  • store pixel into current buffer [0067]
  • for all pixels [0068]
  • compare previous buffer against current buffer and classify [0069]
  • for all tiles that have changes [0070]
  • compress tile [0071]
  • transmit compressed tile [0072]
  • swap buffers [0073]
  • An alternative approach may be to compare the blocks or tiles of the previous frame with the current frame, as the current frame is grabbed. This approach may be described as: [0074]
  • for all incoming pixels [0075]
  • Store pixel into current buffer [0076]
  • compare current pixel against previous pixel [0077]
  • classify tile [0078]
  • for all tiles that have changed [0079]
  • compress tile [0080]
  • transmit compressed tile [0081]
  • swap buffers [0082]
  • Other ways for implementing this comparison feature will be appreciated by persons skilled in the art, and need not be described herein, as the present invention is not limited to any particular manner or methodology of implementing this feature. [0083]
  • Having described the top-level functional operation of the [0084] NVA 400 of the preferred embodiment, reference is now made to FIG. 9, which is a flowchart illustrating the top-level functional operation of a destination (or client) computer 50. In operation, a destination computer 50 is configured to receive a first full frame of graphics information and store that graphics information in the frame buffer of the graphics cards (step 520). In this regard, it will be assumed that the destination computer 50 is configured with the appropriate software drivers for interfacing between the components of the present invention and the graphics card installed in the destination computer 50. In a conventional manner, the graphics card on the destination computer 50 will retrieve information from the frame buffer on the graphics card and render that graphics information on a display. As the inventive components of the system of the present invention continued to update the frame buffer of the graphics card, the graphics card of the destination computer 50 will continue to accordingly update the display. Therefore, upon receipt of a first frame of graphics information by the destination computer, the destination computer will be configured to thereafter receive graphics information for portions or blocks of subsequent video frames (step 522). These blocks of subsequent graphics information are decompressed (step 524) in accordance with an appropriate algorithm (i.e., a decompression algorithm that is reciprocal to the compression embodied by the NVA 400). Thereafter, the destination computer 50 is configured to override the frame buffer blocks or portions that correspond to the received frame buffer blocks or portions. As the graphics card installed in the destination computer 50, in accordance with its conventional operation, continues to render data from the frame buffer to the display screen, this changed data will automatically and continually be reflected on the display.
  • Reference is now made to FIG. 10, which is a block diagram illustrating certain principle components of the preferred embodiment of the [0085] NVA 400. The NVA 400 may comprise a video input circuit 410 that is configured to receive video input on cable 22, digitize the graphics information if appropriate, and appropriately store a first frame of graphics information into a frame buffer memory 420. As previously mentioned, the frame buffer memory 420 may be configured to mimic the organization of a conventional frame buffer memory as contained on a graphics card. The video input circuit 410 will preferably comprise the appropriate circuitry and/or software logic for identifying the start of a video frame and capturing the appropriate information for storing within the frame buffer memory 420.
  • As previously described, in accordance with the preferred embodiment of the invention, the [0086] frame buffer memory 420 is effectively partitioned into a plurality of blocks that are separately and individually analyzed and updated with corresponding blocks of subsequent video frames. In this respect, the NVA 400 preferably comprises a temporary memory 430 in which a current video frame (i.e., a subsequent video frame, after the first video frame is received and stored in the frame buffer memory 420) is temporarily stored. The NVA 400 further comprises comparison logic for comparing blocks stored within the temporary memory 430 with corresponding blocks stored within the frame buffer memory 420. The comparison logic 440 may be implemented in hardware, software, or an appropriate combination of the two, depending upon design constraints and objectives. In operation, the comparison logic 440 may be configured to identify those blocks of graphics information that differ from the corresponding blocks stored in the frame buffer memory 420, and forward those blocks for transmission to the destination computer 50. For each block of graphics information that is identified by the comparison logic 440 to have changed, the comparison logic 440 may be further configured to overwrite the corresponding block of the frame buffer memory 420 with the new, or changed, graphics information.
  • Finally, FIG. 10 illustrates a [0087] functional block 450 labeled as “packetize and transmit.” This block functions to receive the changed graphics information from the comparison logic 440, packetize that information, and transmit the information over the network 40 to the destination computer 50. In this respect, block 450 is preferably configured to compress each block of graphics information before transmission, in order to minimize bandwidth consumption over the network 40.
  • Reference is now made to FIG. 11, which is a block diagram illustrating certain components within the [0088] destination computer 50. Several of the components illustrated in FIG. 11, such as the network interface 602, frame buffer memory 604, video driver 606, and display 608 are conventional components, and need not be described herein. In this respect, the network interface 602 may be provided in connection with an Ethernet card or other conventional network interface that are well-known. The network interface 602 is configured to receive data transmitted over the network 40. Likewise, the frame buffer memory 604 may be provided on a graphics card installed within the destination computer 50. Likewise, the video driver 606 and display 608 may be conventional components of the graphics and display system of the destination computer 50.
  • With respect to the inventive components, the [0089] destination computer 50 may comprise receive circuitry 620, which is configured to receive graphics information from the network interface 602 (i.e., the graphics information transmitted by the NVA 400). The receive circuitry 620 may comprise logic 622 for decompressing graphics information, if the NVA 400 is configured to first compress information before transmission over the network. Similarly, the receive circuitry 620 may comprise a memory 624 for temporarily storing graphics information received from the network. In operation, the receive circuitry 620 will overwrite information stored within the frame buffer memory 604 with new graphics information that is received over the network 40. In this regard, the receive circuitry 620 may receive an entire frame of graphics information that it correspondingly writes into the frame buffer memory 604. Alternatively, the receive circuitry 620 may receive only blocks of graphics information that it writes into corresponding locations of the frame buffer memory 604.
  • As previously described, the [0090] NVA 400 of the preferred embodiment organizes packets of data transmitted over the network to identify, if appropriate, particular blocks of the frame buffer to which the data corresponds. The receive circuitry 620 will, accordingly, be configured to identify such block descriptors and write the received information into the appropriate location of the frame buffer 604. If an entire frame of graphics information is transmitted from the NVA 400 to the destination computer 50, the receive circuitry may be configured to readily receive and overwrite the entire frame buffer 604 with the frame of graphics information. Alternatively, if only particular blocks of graphics information are transmitted from the NVA 400 to the destination computer 50, the receive circuitry 620 is configured to overwrite only those corresponding blocks.
  • Having described the structure and operation of the preferred embodiment of the present invention, it will be appreciated that a variety of alternative embodiments may be provided within the scope and spirit of the present invention. In this regard, reference is made briefly to FIG. 12, which is a flowchart illustrating the top-level operation of the [0091] NVA 400 constructed in accordance with one such alternative embodiment. In this regard, the NVA 400 may be constructed to evaluate individual blocks of graphics information as they are received from the source computer 20, without first receiving an entire frame of subsequent graphics information. Such an alternative configuration would advantageously reduce the size of the temporary memory 430 (see FIG. 10). In operation, such an embodiment would, like the previously described embodiment, receive a first full frame of graphics information and store that frame into the frame buffer memory 420 (step 702). That first full frame of graphics information would then be packetized and transmitted over the network (step 704), as previously described. Thereafter, the NVA 400 may be configured to evaluate the graphics information “on the fly” as it is received from the source computer 20.
  • Specifically, the NVA may be configured to receive a portion of the graphics information ([0092] 706) and compare the received portion with a corresponding portion, or block, stored in the frame buffer memory 420 (step 708). If this portion is different than the previously stored portion, the NVA 400 may be configured to packetize and transmit this portion over the network 40 to the destination computer 50 (step 710). Finally, and if this portion of the video frame differed from the corresponding portion stored in the frame buffer memory 420, the NVA 400 may be configured to overwrite the corresponding memory portion of frame buffer memory 420 with the newly-received portion of graphics information (step 712). This process is repeated for all portions of a current video frame, and cyclically repeated for subsequent video frames.
  • Although various alternative implementations may be implemented within the scope and spirit of the present invention based upon the foregoing description, the following is a pseudo-code algorithm of one implementation at the NVA: [0093]
  • 1. For each video refresh by the graphics card within the source computer, the NVA [0094] 400:
  • a. Reads the current video frame into memory for the current video frame from a video port; [0095]
  • b. For each block in the video frame; [0096]
  • i. Compare the block in the current video frame against the corresponding block in the previous video frame; [0097]
  • ii. If the current block is different from the previous block, then; [0098]
  • 1. The NVA compresses the block in the current video frame; [0099]
  • 2. The NVA transmits the block to the destination computer over the network; [0100]
  • iii. The current video frame is now made the previous video frame. [0101]
  • Likewise, the following is a pseudo-code algorithm of one implementation of the operation of the destination computer; [0102]
  • 1. For each transmitted video frame by the NVA; [0103]
  • a. For each transmitted block in the video frame; [0104]
  • i. Read the blocks from the network interface; [0105]
  • ii. Decompress the blocks; [0106]
  • iii. Assemble one or more images to be displayed; and [0107]
  • iv. Display the blocks at the correct location on the screen. [0108]

Claims (19)

What is claimed is:
1. An apparatus for communicating graphics across a network comprising:
a frame buffer memory for storing and maintaining at least a portion of a previous frame of graphics information, the graphics information being contained in a video signal;
a temporary memory configured to store at least a portion of a current frame of graphics information;
comparison logic for comparing a portion of the current frame of graphics information with a corresponding portion of the previous frame; and
transmission logic for transmitting the portion of the current frame to a destination computer, if the comparison logic determines that the portion of the current frame of graphics information differs from the corresponding portion of the previous frame by more than a predetermined measure.
2. The apparatus of claim 1, wherein the frame buffer memory is configured to store an entire frame of graphics information.
3. The apparatus of claim 1, wherein the predetermined measure is any quantifiable difference.
4. The apparatus of claim 1, wherein the video signal is an analog video signal.
5. The apparatus of claim 1, further comprising compression logic for compressing the portion of the current frame before transmission.
6. The apparatus of claim 1, further comprising a network interface circuit coupled to both the transmission logic and the network, the network interface circuit configured to format and communicate the graphics information over the network to a remote computer.
7. The apparatus of claim 1, wherein the network comprises a local area network (LAN).
8. The apparatus of claim 1, wherein the network comprises a wide area network (WAN).
9. The apparatus of claim 1, wherein the network interface circuit is configured to format the graphics information into a plurality of Internet Protocol (IP) packets that are communicated over the computer network to the remote computer.
10. The apparatus of claim 1, further comprising a second input for receiving a second video signal.
11. The apparatus of claim 10, further comprising a second frame buffer memory for storing and maintaining a previous frame of graphics information from the second video signal.
12. The apparatus of claim 10, further comprising a second temporary memory configured to store at least a portion of a current frame of graphics information from the second video signal.
13. The apparatus of claim 1, wherein temporary memory is configured to store an entire frame of the current frame of graphics information
14. An apparatus for displaying graphics information received from a remote computer and communicated across a network comprising:
an input for receiving packetized graphics information; and
input logic configured to format and store a portion of a frame of graphics information received at the input into an appropriate location of a frame buffer memory, the portion being an amount less than the whole frame buffer.
15. The apparatus of claim 14, wherein the input logic further comprises logic configured to decompress received graphics information.
16. A method for communicating graphics across a computer network comprising:
storing at least a portion of a frame of graphics information obtained from a video signal;
receiving at least a portion of a current frame of graphics information;
comparing a portion of the current frame of graphics information with a corresponding portion of the stored frame of graphics information;
if the compared portion of the current frame of graphics information differs by at least a predetermined amount from the corresponding portion of the stored graphics information, then transmitting the compared portion of the current frame of graphics information to a destination computer; and
if the compared portion of the current frame of graphics information differs by at least a predetermined amount from the corresponding portion of the stored graphics information, then overwriting the corresponding portion of the stored graphics information with the compared portion of the current frame of graphics information.
17. The method of claim 16, wherein the step of receiving at least a portion of the current frame of graphics information more specifically comprises receiving an entire frame of graphics information.
18. The method of claim 16, further comprising compressing the compared portion of the current frame of graphics information before transmitting to the destination computer.
19. The method of claim 16, wherein the predetermined measure is any quantifiable difference.
US09/960,523 2001-09-21 2001-09-21 System and method for communicating graphics over a network Abandoned US20030058248A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/960,523 US20030058248A1 (en) 2001-09-21 2001-09-21 System and method for communicating graphics over a network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09/960,523 US20030058248A1 (en) 2001-09-21 2001-09-21 System and method for communicating graphics over a network

Publications (1)

Publication Number Publication Date
US20030058248A1 true US20030058248A1 (en) 2003-03-27

Family

ID=25503280

Family Applications (1)

Application Number Title Priority Date Filing Date
US09/960,523 Abandoned US20030058248A1 (en) 2001-09-21 2001-09-21 System and method for communicating graphics over a network

Country Status (1)

Country Link
US (1) US20030058248A1 (en)

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030080971A1 (en) * 2001-10-31 2003-05-01 Hochmuth Roland M. System and method for communicating graphics image data over a communication network
US20030184549A1 (en) * 2002-03-26 2003-10-02 Samsung Electronics Co., Ltd. Image processing apparatus, and apparatus for and method of receiving processed image
US20040059782A1 (en) * 2002-09-20 2004-03-25 American Megatrends, Inc. Systems and methods for establishing interaction between a local computer and a remote computer
US20040073722A1 (en) * 2001-01-23 2004-04-15 Sharon Aviran System and method for data re-compression for communication over ip
US20040190773A1 (en) * 2003-03-31 2004-09-30 American Megatrends, Inc. Method, apparatus, and computer-readable medium for identifying character coordinates
US20040255276A1 (en) * 2003-06-16 2004-12-16 Gene Rovang Method and system for remote software testing
US20050046637A1 (en) * 2001-12-10 2005-03-03 American Megatrends, Inc. Systems and methods for capturing screen displays from a host computing system for display at a remote terminal
US20060085562A1 (en) * 2004-10-14 2006-04-20 Blaho Bruce E Devices and methods for remote computing using a network processor
US7053903B1 (en) * 2003-09-03 2006-05-30 Nvidia Corporation Methods and apparatus for write watch for vertex lists
US20070028173A1 (en) * 2002-12-18 2007-02-01 Doug Lauder Method and system for application broadcast
US20070168746A1 (en) * 2005-12-14 2007-07-19 Stefano Righi System and method for debugging a target computer using SMBus
US20070200837A1 (en) * 2006-02-24 2007-08-30 Quanta Computer Inc. Display device
US20070263005A1 (en) * 2006-05-12 2007-11-15 Au Optronics Corp. Liquid crystal display panel, timing control device thereof, and method for generating overdrive parameters for the same
US20070291004A1 (en) * 1999-08-25 2007-12-20 Avocent Redmond Corporation KVM switch including a terminal emulator
US20080040522A1 (en) * 2006-08-10 2008-02-14 Avocent Huntsville Corporation Rack interface pod with intelligent platform control
US20080052442A1 (en) * 2006-08-10 2008-02-28 Avocent Huntsville Corporation Rack Interface Pod with intelligent platform control
US7412625B2 (en) 2003-05-27 2008-08-12 American Megatrends, Inc. Method and system for remote software debugging
US20090033670A1 (en) * 2007-07-31 2009-02-05 Hochmuth Roland M Providing pixels from an update buffer
US7519749B1 (en) 2004-08-25 2009-04-14 American Megatrends, Inc. Redirecting input and output for multiple computers
US7543277B1 (en) 2003-06-27 2009-06-02 American Megatrends, Inc. Method and system for remote software debugging
US20100115136A1 (en) * 2007-02-27 2010-05-06 Jean-Pierre Morard Method for the delivery of audio and video data sequences by a server
US7783799B1 (en) 2006-08-31 2010-08-24 American Megatrends, Inc. Remotely controllable switch and testing methods using same
US7827258B1 (en) 2004-03-01 2010-11-02 American Megatrends, Inc. Method, system, and apparatus for communicating with a computer management device
US20110047476A1 (en) * 2008-03-24 2011-02-24 Hochmuth Roland M Image-based remote access system
US20110210976A1 (en) * 2009-09-16 2011-09-01 Nvidia Corporation Techniques for transferring graphics data from system memory to a discrete gpu
US20150195521A1 (en) * 2014-01-09 2015-07-09 Nvidia Corporation Candidate motion vector selection systems and methods
US9380177B1 (en) * 2004-01-30 2016-06-28 Ip Holdings, Inc. Image and augmented reality based networks using mobile devices and intelligent electronic glasses
GB2543064A (en) * 2015-10-06 2017-04-12 Displaylink Uk Ltd Managing display data
US10140514B1 (en) 2004-01-30 2018-11-27 Ip Holdings, Inc. Capturing and sharing images with mobile device users including for a limited duration of time
US10142496B1 (en) 2013-01-26 2018-11-27 Ip Holdings, Inc. Mobile device image capture and image modification including filters, superimposing and geofenced comments in augmented reality
US10431188B1 (en) * 2016-08-19 2019-10-01 Amazon Technologies, Inc. Organization of personalized content
US11455799B2 (en) 2004-01-30 2022-09-27 Airspace Reality Image networks for mobile communication
US20220358057A1 (en) * 2021-05-06 2022-11-10 Nexcom International Co., Ltd. Computer system, remote control monitoring system, and remote control monitoring method

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4949169A (en) * 1989-10-27 1990-08-14 International Business Machines Corporation Audio-video data interface for a high speed communication link in a video-graphics display window environment
US5117225A (en) * 1989-05-01 1992-05-26 Summit Micro Design Computer display screen monitoring system
US5210825A (en) * 1990-04-26 1993-05-11 Teknekron Communications Systems, Inc. Method and an apparatus for displaying graphical data received from a remote computer by a local computer
US5373307A (en) * 1990-11-09 1994-12-13 Sharp Kabushiki Kaisha Image display device for displaying an identical image on its display panel and an external display unit, but capable of freezing and erasing an image on the latter
US5508713A (en) * 1992-06-19 1996-04-16 Ricoh Company, Ltd. Control system for picture display apparatus having improved displaying data transmission system
US5812785A (en) * 1996-03-21 1998-09-22 Buy Phone, Inc. Method and apparatus for monitoring video signals in a computer
US5990852A (en) * 1996-10-31 1999-11-23 Fujitsu Limited Display screen duplication system and method
US6085252A (en) * 1996-04-23 2000-07-04 Motorola Inc. Device, system and method for real-time multimedia streaming
US6094453A (en) * 1996-10-11 2000-07-25 Digital Accelerator Corporation Digital data compression with quad-tree coding of header file
US6141693A (en) * 1996-06-03 2000-10-31 Webtv Networks, Inc. Method and apparatus for extracting digital data from a video stream and using the digital data to configure the video stream for display on a television set
US6151632A (en) * 1997-03-14 2000-11-21 Microsoft Corporation Method and apparatus for distributed transmission of real-time multimedia information
US6154771A (en) * 1998-06-01 2000-11-28 Mediastra, Inc. Real-time receipt, decompression and play of compressed streaming video/hypervideo; with thumbnail display of past scenes and with replay, hyperlinking and/or recording permissively intiated retrospectively
US6175871B1 (en) * 1997-10-01 2001-01-16 3Com Corporation Method and apparatus for real time communication over packet networks
US6189039B1 (en) * 1997-04-10 2001-02-13 International Business Machines Corporation Selective tunneling of streaming data
US6233634B1 (en) * 1996-08-17 2001-05-15 Compaq Computer Corporation Server controller configured to snoop and receive a duplicative copy of display data presented to a video controller
US6331855B1 (en) * 1999-04-28 2001-12-18 Expertcity.Com, Inc. Method and apparatus for providing remote access, control of remote systems and updating of display information

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5117225A (en) * 1989-05-01 1992-05-26 Summit Micro Design Computer display screen monitoring system
US4949169A (en) * 1989-10-27 1990-08-14 International Business Machines Corporation Audio-video data interface for a high speed communication link in a video-graphics display window environment
US5210825A (en) * 1990-04-26 1993-05-11 Teknekron Communications Systems, Inc. Method and an apparatus for displaying graphical data received from a remote computer by a local computer
US5373307A (en) * 1990-11-09 1994-12-13 Sharp Kabushiki Kaisha Image display device for displaying an identical image on its display panel and an external display unit, but capable of freezing and erasing an image on the latter
US5508713A (en) * 1992-06-19 1996-04-16 Ricoh Company, Ltd. Control system for picture display apparatus having improved displaying data transmission system
US5812785A (en) * 1996-03-21 1998-09-22 Buy Phone, Inc. Method and apparatus for monitoring video signals in a computer
US6085252A (en) * 1996-04-23 2000-07-04 Motorola Inc. Device, system and method for real-time multimedia streaming
US6141693A (en) * 1996-06-03 2000-10-31 Webtv Networks, Inc. Method and apparatus for extracting digital data from a video stream and using the digital data to configure the video stream for display on a television set
US6233634B1 (en) * 1996-08-17 2001-05-15 Compaq Computer Corporation Server controller configured to snoop and receive a duplicative copy of display data presented to a video controller
US6094453A (en) * 1996-10-11 2000-07-25 Digital Accelerator Corporation Digital data compression with quad-tree coding of header file
US5990852A (en) * 1996-10-31 1999-11-23 Fujitsu Limited Display screen duplication system and method
US6151632A (en) * 1997-03-14 2000-11-21 Microsoft Corporation Method and apparatus for distributed transmission of real-time multimedia information
US6189039B1 (en) * 1997-04-10 2001-02-13 International Business Machines Corporation Selective tunneling of streaming data
US6175871B1 (en) * 1997-10-01 2001-01-16 3Com Corporation Method and apparatus for real time communication over packet networks
US6154771A (en) * 1998-06-01 2000-11-28 Mediastra, Inc. Real-time receipt, decompression and play of compressed streaming video/hypervideo; with thumbnail display of past scenes and with replay, hyperlinking and/or recording permissively intiated retrospectively
US6331855B1 (en) * 1999-04-28 2001-12-18 Expertcity.Com, Inc. Method and apparatus for providing remote access, control of remote systems and updating of display information

Cited By (75)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8269783B2 (en) 1999-08-25 2012-09-18 Avocent Redmond Corporation KVM switch including a terminal emulator
US20070291004A1 (en) * 1999-08-25 2007-12-20 Avocent Redmond Corporation KVM switch including a terminal emulator
US20080082705A1 (en) * 1999-08-25 2008-04-03 Avocent Redmond Corporation KVM switch including a terminal emulator
US20040073722A1 (en) * 2001-01-23 2004-04-15 Sharon Aviran System and method for data re-compression for communication over ip
US7007113B2 (en) * 2001-01-23 2006-02-28 Flash Networks Ltd System and method for data re-compression for communication over IP
US20030080971A1 (en) * 2001-10-31 2003-05-01 Hochmuth Roland M. System and method for communicating graphics image data over a communication network
US7589737B2 (en) * 2001-10-31 2009-09-15 Hewlett-Packard Development Company, L.P. System and method for communicating graphics image data over a communication network
US20050046637A1 (en) * 2001-12-10 2005-03-03 American Megatrends, Inc. Systems and methods for capturing screen displays from a host computing system for display at a remote terminal
US7233336B2 (en) 2001-12-10 2007-06-19 American Megatrends, Inc. Systems and methods for capturing screen displays from a host computing system for display at a remote terminal
US20030184549A1 (en) * 2002-03-26 2003-10-02 Samsung Electronics Co., Ltd. Image processing apparatus, and apparatus for and method of receiving processed image
US20040222944A1 (en) * 2002-09-20 2004-11-11 American Megatrands, Inc. In-line video, keyboard and mouse remote management unit
US7260624B2 (en) 2002-09-20 2007-08-21 American Megatrends, Inc. Systems and methods for establishing interaction between a local computer and a remote computer
US7454490B2 (en) * 2002-09-20 2008-11-18 American Megatrends, Inc. In-line video, keyboard and mouse remote management unit
US20040059782A1 (en) * 2002-09-20 2004-03-25 American Megatrends, Inc. Systems and methods for establishing interaction between a local computer and a remote computer
US8310493B1 (en) 2002-12-18 2012-11-13 West Corporation Method and system for application broadcast
US7616208B2 (en) * 2002-12-18 2009-11-10 Genesys Conferencing Ltd. Method and system for application broadcast
US9171349B1 (en) 2002-12-18 2015-10-27 Intercall Canada, Inc. Method and system for application broadcast
US8754898B1 (en) 2002-12-18 2014-06-17 Genesys Conferencing, Ltd. Method and system for application broadcast
US20070028173A1 (en) * 2002-12-18 2007-02-01 Doug Lauder Method and system for application broadcast
US7418141B2 (en) 2003-03-31 2008-08-26 American Megatrends, Inc. Method, apparatus, and computer-readable medium for identifying character coordinates
US20040190773A1 (en) * 2003-03-31 2004-09-30 American Megatrends, Inc. Method, apparatus, and computer-readable medium for identifying character coordinates
US7412625B2 (en) 2003-05-27 2008-08-12 American Megatrends, Inc. Method and system for remote software debugging
US8539435B1 (en) 2003-06-16 2013-09-17 American Megatrends, Inc. Method and system for remote software testing
US20090235122A1 (en) * 2003-06-16 2009-09-17 Gene Rovang Method and System for Remote Software Testing
US7945899B2 (en) 2003-06-16 2011-05-17 American Megatrends, Inc. Method and system for remote software testing
US20040255276A1 (en) * 2003-06-16 2004-12-16 Gene Rovang Method and system for remote software testing
US7546584B2 (en) 2003-06-16 2009-06-09 American Megatrends, Inc. Method and system for remote software testing
US8898638B1 (en) 2003-06-27 2014-11-25 American Megatrends, Inc. Method and system for remote software debugging
US8046743B1 (en) 2003-06-27 2011-10-25 American Megatrends, Inc. Method and system for remote software debugging
US7543277B1 (en) 2003-06-27 2009-06-02 American Megatrends, Inc. Method and system for remote software debugging
US7053903B1 (en) * 2003-09-03 2006-05-30 Nvidia Corporation Methods and apparatus for write watch for vertex lists
US10755101B1 (en) 2004-01-30 2020-08-25 Sunil K Rao Image networks for mobile communication
US11455799B2 (en) 2004-01-30 2022-09-27 Airspace Reality Image networks for mobile communication
US10140514B1 (en) 2004-01-30 2018-11-27 Ip Holdings, Inc. Capturing and sharing images with mobile device users including for a limited duration of time
US9380177B1 (en) * 2004-01-30 2016-06-28 Ip Holdings, Inc. Image and augmented reality based networks using mobile devices and intelligent electronic glasses
US20110015918A1 (en) * 2004-03-01 2011-01-20 American Megatrends, Inc. Method, system, and apparatus for communicating with a computer management device
US8359384B2 (en) 2004-03-01 2013-01-22 American Megatrends, Inc. Method, system, and apparatus for communicating with a computer management device
US7827258B1 (en) 2004-03-01 2010-11-02 American Megatrends, Inc. Method, system, and apparatus for communicating with a computer management device
US7793019B1 (en) 2004-08-25 2010-09-07 American Megatrends, Inc. Redirecting input and output for multiple computers
US8001302B2 (en) 2004-08-25 2011-08-16 American Megatrends, Inc. Redirecting input and output for multiple computers
US7840728B1 (en) 2004-08-25 2010-11-23 American Megatrends, Inc. Redirecting input and output for multiple computers
US20110066773A1 (en) * 2004-08-25 2011-03-17 American Megatrends, Inc. Redirecting input and output for multiple computers
US7861020B1 (en) 2004-08-25 2010-12-28 American Megatrends, Inc. Redirecting input and output for multiple computers
US7519749B1 (en) 2004-08-25 2009-04-14 American Megatrends, Inc. Redirecting input and output for multiple computers
US20060085562A1 (en) * 2004-10-14 2006-04-20 Blaho Bruce E Devices and methods for remote computing using a network processor
US20070168746A1 (en) * 2005-12-14 2007-07-19 Stefano Righi System and method for debugging a target computer using SMBus
US8010843B2 (en) 2005-12-14 2011-08-30 American Megatrends, Inc. System and method for debugging a target computer using SMBus
US8566644B1 (en) 2005-12-14 2013-10-22 American Megatrends, Inc. System and method for debugging a target computer using SMBus
US20070200837A1 (en) * 2006-02-24 2007-08-30 Quanta Computer Inc. Display device
US20070263005A1 (en) * 2006-05-12 2007-11-15 Au Optronics Corp. Liquid crystal display panel, timing control device thereof, and method for generating overdrive parameters for the same
US8009173B2 (en) * 2006-08-10 2011-08-30 Avocent Huntsville Corporation Rack interface pod with intelligent platform control
US20080040522A1 (en) * 2006-08-10 2008-02-14 Avocent Huntsville Corporation Rack interface pod with intelligent platform control
US20080052442A1 (en) * 2006-08-10 2008-02-28 Avocent Huntsville Corporation Rack Interface Pod with intelligent platform control
US8427489B2 (en) 2006-08-10 2013-04-23 Avocent Huntsville Corporation Rack interface pod with intelligent platform control
US7979610B2 (en) 2006-08-31 2011-07-12 American Megatrends, Inc. Remotely controllable switch and testing methods using same
US7783799B1 (en) 2006-08-31 2010-08-24 American Megatrends, Inc. Remotely controllable switch and testing methods using same
US20100115136A1 (en) * 2007-02-27 2010-05-06 Jean-Pierre Morard Method for the delivery of audio and video data sequences by a server
US8127044B2 (en) * 2007-02-27 2012-02-28 Sagem Communications Sas Method for the delivery of audio and video data sequences by a server
WO2009017594A2 (en) * 2007-07-31 2009-02-05 Hewlett-Packard Development Company, L.P. Providing pixels from an update buffer
US7944451B2 (en) * 2007-07-31 2011-05-17 Hewlett-Packard Development Company, L.P. Providing pixels from an update buffer
GB2464044B (en) * 2007-07-31 2011-04-06 Hewlett Packard Development Co Providing pixels from an update buffer
WO2009017594A3 (en) * 2007-07-31 2009-03-19 Hewlett Packard Development Co Providing pixels from an update buffer
US20090033670A1 (en) * 2007-07-31 2009-02-05 Hochmuth Roland M Providing pixels from an update buffer
GB2464044A (en) * 2007-07-31 2010-04-07 Hewlett Packard Development Co Providing pixels from an update buffer
US20110047476A1 (en) * 2008-03-24 2011-02-24 Hochmuth Roland M Image-based remote access system
US20110210976A1 (en) * 2009-09-16 2011-09-01 Nvidia Corporation Techniques for transferring graphics data from system memory to a discrete gpu
US8780122B2 (en) * 2009-09-16 2014-07-15 Nvidia Corporation Techniques for transferring graphics data from system memory to a discrete GPU
US10142496B1 (en) 2013-01-26 2018-11-27 Ip Holdings, Inc. Mobile device image capture and image modification including filters, superimposing and geofenced comments in augmented reality
US20150195521A1 (en) * 2014-01-09 2015-07-09 Nvidia Corporation Candidate motion vector selection systems and methods
GB2543064A (en) * 2015-10-06 2017-04-12 Displaylink Uk Ltd Managing display data
GB2543064B (en) * 2015-10-06 2018-08-22 Displaylink Uk Ltd Managing display data
US10545868B2 (en) 2015-10-06 2020-01-28 Displaylink (Uk) Limited Managing display data
US10431188B1 (en) * 2016-08-19 2019-10-01 Amazon Technologies, Inc. Organization of personalized content
US20220358057A1 (en) * 2021-05-06 2022-11-10 Nexcom International Co., Ltd. Computer system, remote control monitoring system, and remote control monitoring method
US11636050B2 (en) * 2021-05-06 2023-04-25 Nexcom International Co., Ltd. Computer system, remote control monitoring system, and remote control monitoring method

Similar Documents

Publication Publication Date Title
US20030058248A1 (en) System and method for communicating graphics over a network
US7684483B2 (en) Method and apparatus for digitizing and compressing remote video signals
US8874812B1 (en) Method and apparatus for remote input/output in a computer system
US7606314B2 (en) Method and apparatus for caching, compressing and transmitting video signals
US7430681B1 (en) Methods and apparatus for interfacing a drawing memory with a remote display controller
JP5148505B2 (en) Display system and method
US7916956B1 (en) Methods and apparatus for encoding a shared drawing memory
USRE39763E1 (en) Isochronous channel having a linked list of buffers
US8176155B2 (en) Remote network management system
US8358656B2 (en) System and method for communication of uncompressed visual information through a network
US20030080971A1 (en) System and method for communicating graphics image data over a communication network
US20030017846A1 (en) Wireless display
WO2009017594A2 (en) Providing pixels from an update buffer
EP2936298B1 (en) Management of memory for storing display data
CN108900843A (en) Monochrome image compression method, device, medium and electronic equipment
US6446155B1 (en) Resource bus interface
US20140218379A1 (en) Device, system and method for transferring network data
US20030079919A1 (en) System and method for displaying an image on a network attachable display device
US20040237110A1 (en) Display monitor
JP2008539614A (en) Integrated wireless multimedia transmission system
US20090033669A1 (en) System And Method For Communicating Graphics Image Data Over A Communication Network
CN115550668A (en) Video image compression method, system, storage medium and equipment
WO2012163059A1 (en) Method, device and system for device redirection data transmission
US8411740B2 (en) System and method for low bandwidth display information transport
US20030041162A1 (en) System and method for communicating graphics images over a computer network

Legal Events

Date Code Title Description
AS Assignment

Owner name: HEWLETT-PACKARD COMPANY, COLORADO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HOCHMUTH, ROLAND M.;MARKS, JOHN;SWEETSER, DAVID JOHN;REEL/FRAME:012527/0001;SIGNING DATES FROM 20010919 TO 20010921

AS Assignment

Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY L.P., TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD COMPANY;REEL/FRAME:014061/0492

Effective date: 20030926

Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY L.P.,TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD COMPANY;REEL/FRAME:014061/0492

Effective date: 20030926

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION