US20090316778A1 - Method And System For Optimal Video Transcoding Based On Utility Function Descriptors - Google Patents

Method And System For Optimal Video Transcoding Based On Utility Function Descriptors Download PDF

Info

Publication number
US20090316778A1
US20090316778A1 US12/548,199 US54819909A US2009316778A1 US 20090316778 A1 US20090316778 A1 US 20090316778A1 US 54819909 A US54819909 A US 54819909A US 2009316778 A1 US2009316778 A1 US 2009316778A1
Authority
US
United States
Prior art keywords
adaptation
multimedia content
dropping
utility
quality
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/548,199
Inventor
Jae-Gon Kim
Yong Wang
Shih-Fu Chang
Kyeongok Kang
Jinwoong Kim
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US12/548,199 priority Critical patent/US20090316778A1/en
Publication of US20090316778A1 publication Critical patent/US20090316778A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00Digital computers in general; Data processing equipment in general
    • G06F15/16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/147Data rate or code amount at the encoder output according to rate distortion criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164Feedback from the receiver or from the transmission channel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present invention relates to techniques for delivering multimedia content across a network, and more specifically, to techniques for transparently and adaptively transporting multimedia content across a wide range of networks.
  • MPEG Motion Picture Experts Group
  • ISO International Organisation for Standardisation
  • part 7 of MPEG-7 proposes the concept of what is called “Digital Item Adaptation.” That concept involves the adaptation of resources and descriptions that constitute a digital item to achieve interoperable transparent access to universal multimedia from any type of terminal and network.
  • Digital Item Adaptation users. in a network would be unaware of network and terminal-specific issues that often affect the delivery of multimedia content, such as network congestion, quality limitations, and reliability of service. It is envisioned that a diverse community of users will therefor be able to share a multimedia experience, each to his or her individual acceptable level of quality.
  • transcoding which avoids the need to store content in different compressed formats for different network bandwidths and different terminals, is one of the most common methods of resource adaptation.
  • so called Transcoding Hints have been proposed in order to enable better transcoding by reducing computation complexity while preserving quality as much as possible.
  • An object of the present invention is to provide a technique for delivering multiple types of multimedia content over a network to a wide spectrum of multimedia users having different acceptable levels of quality.
  • Another object of the present invention is to provide multimedia content description techniques that are useful to fulfill several requirements.
  • a preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate a quality measurement for each adapted multimedia segment, and generating a utility based descriptor based on the portions of utility based descriptor information and corresponding quality measurements.
  • the compressed multimedia information is MPEG-4 data, and from ten to twenty portions of utility based descriptor information are utilized.
  • the portions of utility based descriptor information may be uniformly or non-uniformly sampled.
  • the adaptation operations may include frame dropping, either by dropping first B frames or all B frames, and may further include coefficient dropping.
  • the present invention provides systems and methods for delivering compressed multimedia information to two or more users, each having different target bit rates.
  • a method includes the steps of receiving at least a segment of compressed multimedia information and a corresponding utility based descriptor, parsing the utility based descriptor into portions, each corresponding to a unique target bit rate for each of the users, selecting a utility based descriptors portion that corresponds to the unique target bit rate for each user, and adapting the compressed multimedia segment by the selected utility based descriptors portion for each user.
  • Target bit rate feedback information from the users, or from the network may be utilized in the adaptation step.
  • FIG. 1 is functional diagram showing the relationships among adaptation spage, utility space, and resource space;
  • FIG. 2 is a block diagram of an exemplary system in accordance with the present invention.
  • FIG. 3 is an illustrative diagram showing a two dimensional adaptation space defined by a combination of frame dropping and coefficient dropping;
  • FIG. 4 is a graph showing an exemplary utility function in accordance with the present invention.
  • FIGS. 5( a )-( c ) are graphs showing variations of the exemplary utility function shown in FIG. 4 ;
  • FIG. 6 is a schematic diagram of an exemplary utility based description tool in accordance with the present invention.
  • FIG. 7 is a schematic diagram of an exemplary utility-based descriptor in accordance with the present invention.
  • a utility-based framework provides a systematic way of efficient video adaptation by modeling the relationships among essential parameters: adaptation operations, resources, and utility.
  • adaptation operations can take the form of spatial domain adaptation, temporal adaptation, or object-based adaptation.
  • Spatial domain adaptation may include spatial resolution reduction and quality or signal-to-noise adaptation, such as requantization or DCT coefficient dropping.
  • Temporal domain adaptation may include frame dropping, and object-based adaptation may include video object prioritization and/or dropping.
  • a particular operation defined by any of such adaptation methods is referred to herein as an adaptation operation.
  • Utility includes the quality of content resulted from a particular adaptation operation. Utility can be measured in an objective manner, such as by determining the peak signal-to-noise ratio (“PSNR”), or a subjective one, e.g., by use of a subjective quality score.
  • FIG. 1 illustrates the multi-dimensional space of adaptation, resources, and utility and the relations among them as applied to MPEG-4 compressed video.
  • the adaptation space 110 represents the conceptual space of all possible adaptation operations for one or more selected adaptation methods.
  • Each dimension of the adaptation space represents one type of adaptation method and has a certain cardinal index representing associated adaptation operations.
  • frame dropping and coefficient dropping there are two dimensions in the adaptation space: frame dropping and coefficient dropping.
  • the dimension of frame dropping can be indexed by the amount of frames dropped, e.g., no dropping, all B frames dropped in a sub-Group of Pictures (“GOP”) (a sub-GOP includes a set of successive frames beginning with an I or P frame and continuing to the next I or P frame), all B and P frames dropped in each GOP.
  • GOP sub-Group of Pictures
  • the coefficient dropping dimension can be indexed by the percentage of rate-reduction to be achieved by coefficient dropping, e.g., no dropping, 10%, 20%, etc. In this way, a set of discrete points in the adaptation space can be defined, each point representing an adaptation operation specified by a particular combination of frame dropping and coefficient dropping.
  • the resource limitation may include several types of resources. For example, in order to provide video-streaming service to certain handheld devices, factors such as spatial resolution or computational capability should also be taken into account along with bandwidth. In general, all types of resources to be satisfied are represented by a multidimensional resource space. Utility space may include attributes in multiple dimensions. In addition to PSNR, the subjective preference like mean opinion scale (“MOS”), temporal smoothness may be included in other dimensions together.
  • MOS mean opinion scale
  • a video segment 101 is a unit undergoing adaptation, with each point representing a particular adaptation operation in the adaptation space.
  • the adapted video segment has the resulting values of resources and utilities represented as corresponding points in the resource and utility spaces, respectively.
  • the shaded cube in the resource space represents the resource constraints specified by applications. Note that there may exist multiple adaptation operations that satisfy the same resource requirement.
  • the oval shaped region in the adaptation space that is mapped into a point in the resource space shows such a constant-resource set. Also, different adaptation operators may result in the same utility value.
  • the rectangular region in the adaptation space represents such a constant-utility set.
  • video adaptation can be formulated as follows: given certain resource constraints, determine the optimal adaptation operation so that the utility of the adapted video is maximized. Since most adaptation problems likely to be assumed in the UMA paradigm can be formulated so formulated, such resource-constrained utility maximization may be considered to be a basic scenario of multimedia adaptation. While the disclosure herein is directed to optimizing a frame and coefficient dropping transcoding to satisfy available bandwidth as an example of resource-constrained utility maximization, those skilled in the art should appreciate that the utility-based framework of the present invention can readily include constraints in the utility space and aim at overall resource minimization.
  • a server computer 210 is adapted to receive stored video 211 and/or live video 212 . That video is preferably in a compressed format, such as MPEG-1, MPEG-2, or MPEG-4, although uncompressed digital video could be provided to the server, with compression occurring thereon.
  • the server 210 includes software written in any available programming language for generating a utility function, in the form of utility-based descriptors, based on the received video.
  • descriptors is indicative of certain modifications to the compressed video, e.g., through the elimination of bidirectionally-predictive (“B”) frames or coefficients, that will result in predetermined levels of quality.
  • the compressed domain video and associated utility function are delivered over a transit network 220 , such as the Internet or an intranet of sufficient bandwidth to transmit the compressed video.
  • the transmitted information is received by a network computer 230 , which in turn serves as the video adaptation engine of the system.
  • the network computer 230 includes software, again written in any available programming language, to adapt the incoming compressed video to the particular bandwidth requirements of several client devices 250 , 251 , 252 , 253 that are served by associated access networks 240 .
  • the network computer 230 uses the utility-based descriptors generated by the server 210 to adapt the compressed video to such bandwidth requirements.
  • the network computer 230 may receive preference information 241 from the client users, and/or available bandwidth information 242 from the network, in order to optimize its adaptation operation.
  • the access network 240 may be the Internet, an intranet, or a proprietary network such as a wireless network that links mobile cellular user terminals 253 to the network computer 230 .
  • the bit rate of a video stream to be delivered is adapted to time-varying bandwidth by an adaptation tool in real-time.
  • a combination of frame dropping and coefficient dropping are used by the server computer 210 for the adaptation of nonscalable video to dynamic bandwidth.
  • transcoding techniques could be utilized to adjust the bit-rate of video streams for dynamic bandwidth adaptation, such as re-encoding, re-quantization of DCT coefficients, object-based transcoding, and image-size reduction.
  • the Fine-Granular-Scalability (“FGS”) and some of its variant forms that have been adopted as new scalable coding tools in MPEG-4 also enable the dynamic adaptation of a FGS-stream to time-varying bandwidth by selecting appropriate number of bitplanes of scalable streams.
  • Frame and coefficient dropping are simple ways of rate adaptation with low computational complexity since they involve the truncation of portions of a bit sequence corresponding to particular frames and symbols of DCT coefficients to be dropped by a compressed-domain processing. Further, for the application of video streaming over mobile wireless networks, they are more suitable for low delay real-time operation that is strongly required in a trancoding proxy.
  • frame dropping adjusts frame rate by dropping some frames
  • coefficient dropping adjusts spatial quality by dropping some of DCT coefficients that are associated with higher frequency components.
  • the dynamic range of rate reduction is increased by combining two or more transcoding methods.
  • Frame dropping is a typical kind of temporal transcoding that adjusts frame rate by dropping some frames from an incoming video stream. It is often used for rate adaptation to bandwidth variations in video streaming applications because of its efficiency and simplicity.
  • One factor should be considered is the selection of the frames to be dropped. For example, when an intra-coded frame (an “I frame”) or certain predectivly coded frames (a “P” frame) are dropped, frames that refer to the dropped frame need to be re-encoded.
  • I frame intra-coded frame
  • P predectivly coded frames
  • B frames and/or P frames that do not have a decoding dependency are dropped, in the unit a group of pictures (“GOP”), by taking into account the sequence structure of an incoming video stream.
  • GOP group of pictures
  • Frame dropping provides only a coarse approximation to the target rate since the smallest unit of data that can be removed is an entire frame. Therefore, the possible frame dropping operations are defined by specifying the frame type to be dropped rather than the reduction rate to be achieved by the dropping.
  • a set of frame dropping operations depending on an assumed GOP structure may be defined as follows: no frame dropping; one B frame dropped in each sub-GOP, all B frames dropped, and all B and P frames dropped, resulting in an I frame only sequence.
  • P frames are dropped from the end of each GOP, such as the last P frame dropped, the two last P frames dropped, etc. through all P frames dropped in each GOP.
  • Coefficient dropping will next be described.
  • the first is requantization, i.e., the modification of quantized coefficients by employing coarser quantization levels to reduce bit rate.
  • the second is coefficient dropping in which higher frequency coefficients which are less important for image quality are truncated. Coefficient dropping is preferred since it is more amenable to fast processing than requantization, which requires the implementation of recoding-type algorithms.
  • breakpoint the number of DCT coefficient codes within each block that will be kept after truncation is called breakpoint.
  • the breakpoint for each block may be determined using Lagrangian optimization, which minimizes the distortion caused by the coefficient dropping while meeting the required target rate in a frame-by-frame basis.
  • an algorithm which does not require memory can be employed, with such an algorithm ignoring accumulated errors caused by motion compensation and treating each picture as an intra-coded one due to its simplicity. Ignoring the accumulated errors does not much affect the quality and allows achieving essentially optimal (within 0.3 dB) performance.
  • coefficient dropping provides the ability to meet and available bandwidth quite accurately within the upper bound of rate reduction by adjusting the amount of dropped coefficients.
  • the upper bound of rate reduction depends on an incoming video stream. Numerous coefficient-dropping operations may be defined by specifying the percentage of rate reduction to be achieved, rather than directly specifying the dropped coefficients themselves. For example, the operation of coefficient dropping (10%) represents a 10% reduction of the bit rate of incoming video stream by coefficient dropping.
  • frame dropping or coefficient dropping alone may not be sufficient to accommodate available bandwidth.
  • only a few discrete points can be achievable by frame dropping, while a continuous rate adaptation is possible by using coefficient dropping. Therefore, the combination of frame dropping and coefficient dropping enables the extension of dynamic range of the reducible rate.
  • the combination of both may also yield better perceptual quality than either technique alone, especially for large rate reductions, by optimizing the trade-off between spatial and temporal quality. For example, in order to reduce frame jerkiness at very low frame rates, temporal resolution can be traded with the spatial quality while meeting the same rate reduction.
  • FIG. 3 a two-dimensional adaptation space defined by the combination of frame dropping and coefficient dropping is shown.
  • Each point represents a frame dropping/coefficient dropping-combined transcoding operation.
  • the effect of the order of operations should be considered in the combination of coefficient and frame dropping.
  • there are two combinations having different orders of operation to achieve the same point 310 either 20% coefficient dropping followed by B frame dropping, or B frame dropping followed by 20% coefficient dropping.
  • the results of both cases are the same if rate-based uniform coefficient dropping in which the same rate-reduction is applied across the frames is employed.
  • different reduction ratios are assigned among frames to achieve global optimum coefficient dropping based on rate allocation, different operation orders may result in different results of reduced rate and quality. While the present disclosure is directed to the former, the present invention contemplates both scenarios.
  • a utility function may be defined as a media quality metric representing a user satisfaction index as a function of resources.
  • the adaptation space is a two-dimensional space specifying combinations of frame dropping and coefficient dropping
  • the resource space includes the available bandwidth varied with time
  • the utility space includes the signal to noise measure of the transcoded video stream.
  • FIG. 4 an exemplary utility function generated by a combined frame dropping/coefficient dropping transcoding method applied to previously stored MPEG-4 compressed video data, “coastguard” coded at 1.5 Mbps and adapted over a bandwidth range less than 200 kbps, is shown.
  • FIG. 4 is a graph plotting target rate in kbits/sec against PSNR, and illustrates four curves 420 , 420 , 430 , 440 that represent the relationship between the target rates and PSNR qualities, each for a different adaptation operation within the exemplary utility function.
  • the frame dropping operations consist of no frames dropped, one B-frame dropped in each sub-GOP, all B-frames dropped, and all B- and P-frames dropped.
  • the six coefficient dropping operation are set at 0%, 10%, 20%, 30%, 40%, and 50% reduction of the bit rate of the original test video stream. In this way, there are 23 combined operations, which employ different combinations of the defined frame dropping and coefficient dropping operations.
  • FIG. 4 also illustrates a re-encoding curve 410 , obtained by a cascaded full-decoding and re-encoding, and thus may be considered as a reference for performance comparison of the transcoding operations. It is important to note that for a given target bandwidth, there are multiple adaptation operations satisfying the same target rate. The optimal operation with the highest video utility is selected.
  • the utility function depends on the type of video content, the chosen coding parameters of an incoming video stream, and the applied transcoding method. Given video segments which share the same content type and transcoding method, the generation of a utility function requires the repetitive computation of PSNR quality and rate for a family of defined adaptation operations by testing all possible operations.
  • Utility function generation for live video will next be described.
  • the utility function can be generated by off-line processing in a server in which computational cost is not a concern, such as in the case of FIG. 4 .
  • this option is generally not an acceptable solution for live video, given the reed for such extensive repetitive computation.
  • a content-based utility prediction solution may be used to predict a utility function in the case of live video.
  • video can be mapped to distinctive utility distribution classes prepared in advance based on computable content features, such as motion activity and spatial activity extracted from compressed streams. Accordingly, a utility function corresponding to an expected incoming stream of video is prepared in advance for live video.
  • Forming a prediction for a live utility function is a two step process. First, an adaptive content classification loop is employed; second, a real-time estimation path is utilized. A set of utility functions that can cover entire types of content are generated and classified in the adaptive content classification loop off-line. Later, when a live video stream is received, the real-time estimation path selects an associated utility function for each video segment in order to preserve the same content type in real-time.
  • the utility function that represents distributions of the adaptation, resource, and utility spaces, as well as the relationships among them, are delivered along with an associated video stream to an adaptation engine, e.g., located on network computer 230 .
  • the main goal of the descriptor is to describe the distributions of the three spaces (adaptation, resource, and utility) and the relationships among them to support various types of usage scenarios in an efficient way.
  • the descriptor should provide sufficient information to the adaptation engine as to what are possible adaptation operations satisfying constrained resources and the associated utilities.
  • the range of bit rates are sampled into a finite set of points, and then all feasible frame dropping-coefficient dropping-combined operations capable of achieving the resource and the associated values of PSNR are described using the sampled resource points as indexes.
  • a finite set of points over the multi-dimensional resource. space is defined as indexes in the description.
  • Linear or non-linear sampling of the resource space can be selected depending on the characteristics of distributions of adaptation space by taking into account the efficiency of description as well as the number of sampling points. Interpolation between two consecutive points of resource and corresponding adaptation operations and utilities may also occur in a linear or non-linear manner. In the case of adaptation, however, it should be noted that interpolation is not feasible between different frame dropping operations unlike coefficient dropping.
  • the descriptor can support most cases of resource-constrained scenarios.
  • Some adaptation operations may not be uniquely defined in terms of quality. For example, an operation of “10% reduction of bit rate by dropping DCT coefficients in each frame (represented as coefficient dropping (10%))” does not specify the exact set of coefficients to be dropped. Different implementations may choose different sets and result in slightly different utility values. As a result, the value of utility associated to a particular operation is not reliable.
  • some adaptation methods do not cause the ambiguity issue due to their unambiguous representation formats in terms of adaptation.
  • some scalable compression formats such as JPEG-2000 and MPEG-4 FGS, provide unambiguously defined scalable layers. Subsets of the layers can be truncated in a consistent way with the same resulted quality as long as the decoders are compliant with the standards.
  • Quality ranking may be employed in order to address this ambiguity issue.
  • the absolute values of utility of each adapted media are not important, but instead, the relative ranking of such values among different adaptation operations satisfying the same resource may be critical. In such cases, the likelihood of achieving ranking consistence is higher than absolute value consistence.
  • the descriptor describe the ranking instead of the utility value to provide the notion of quality even the case that the quality values are not reliable due to the ambiguity.
  • the descriptors may include a flag to represent whether or not the ranking is consistent over different implementations. Assuming there exists some consistence among practical implementations, the empirical value of the flag may be obtained.
  • FIGS. 5( a )-( c ) the variation of utility functions resulting from different implementations of coefficient dropping to obtain the value of the consistency flag is shown.
  • FIG. 5( a ) is a reproduction of FIG. 4 ;
  • FIG. 5( b ) shows the same curves applied to the same data, except that macroblock optimizing is selected; and
  • FIG. 5( c ) again shows the same curves applied to the same data, except with pure rate-based uniform coefficient dropping is used, without optimization across blocks.
  • FIGS. 5( a )( c ) there are noticeable variations of utility values among utility functions with different implementations. There may be several operations with different qualities achieving the same bit rate. In some parts of the bit rate range excepting the range covered by shaded box in FIG. 5( c ), the rank of such equal-rate operations in terms of quality is consistent among different implementations. Even in the shaded box, there is constancy of rank depending on operations. Namely, the operation of all B frame coefficient dropping and coefficient dropping has the worst utility regardless of implementations. Based on this observation, the descriptor describes the rank and the optional flag for each operator to represent the consistency of the rank fully.
  • the descriptor provides a set of adaptation descriptors 610 , each of which describes a utility function associated with an adaptation method by including elements of resource, utility, and utility function
  • the descriptor enables the selection of a defined adaptation method according to the intended scenario by specifying one of the enumerated ones by an attribute; such as combined frame and coefficient dropping.
  • the Resource 620 and Utility 630 descriptors define constrained resource and utility associated to the utility function 640 to be described in terms of name and unit, respectively. Especially, multiple instantiations of the Resource field 620 are allowed to accommodate the multi dimensional resource space.
  • the UtilityFunction descriptor 640 represents a set of possible adaptation operators and associated utilities as function of resource points.
  • the UtilityFunction descriptor 640 includes a set of ResourcePoints 710 , each of which includes a set of AdaptationOperators 720 to describe all possible adaptation operations satisfying the sampled values of constrained resources that are described by ResourceValues 730 .
  • a particular adaptation operation of the specific adaptation method is described by selecting corresponding element.
  • FrameCoeffDropping 740 may be used for describing a particular operation of frame dropping/coefficient dropping-combined transcoding by specifying the type and number of frame to be dropped and the percentage of bit rate to be reduced by truncating coefficients.
  • WaveletReduction 750 in order to describe the specific operation of wavelet reduction by specifying the number of levels and bitplanes to be truncated.
  • the adaptation operator FGS 770 may be used to describe the specific operation of an MPEG-4 Fine Granularity Scalability (“FGS”) stream by specifying the number of bitplanes of FGS-frames, and/or the number of bitplanes of FGST-frames to be truncated from the enhancement layer.
  • FGS Fine Granularity Scalability
  • the associated utility value is described by the UtilityValue 760 .
  • the UtilityRankInformation 761 is instantiated instead of the UtilityValue to describe the rank of the associated operation with an optional attribute of a consistency Flag representing the consistence of the rank.

Abstract

Techniques for generating utility-based descriptors from compressed multimedia information are disclosed. A preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate measurement for each adapted multimedia segment, and generating a utility based descriptors based on the portions of utility based descriptor information and corresponding quality measurements.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application is based on U.S. provisional patent applications Ser. No. 60/376,129, filed Apr. 26, 2002, and No. 60/384,939, filed May 31, 2002, which are incorporated herein by reference for all purposes and from which priority is claimed.
  • BACKGROUND OF THE INVENTION
  • 1. Technical Field
  • The present invention relates to techniques for delivering multimedia content across a network, and more specifically, to techniques for transparently and adaptively transporting multimedia content across a wide range of networks.
  • 2. Background Art
  • At the dawn of the 21st century, the Internet has achieved widespread use among businesses and consumers in the exchange of all forms of multimedia information. Graphic art, text, audio, video and other forms of information are continuously shared among users. In order to reduce bandwidth requirements to manageable levels, multimedia information is often stored and transported in the form of compressed bitstreams that are in a standard format. For example, in the case of audiovisual information, JPEG, Motion JPEG, MPEG-1, MPEG-2, MPEG-4, H.261 and H.263 are in widespread use. Unfortunately, while a multitude of differing types of standardized multimedia content have been developed and made available on the Internet, there presently exists no standard way to control the access, delivery, management and protection for such content. Recognizing this need, the Motion Picture Experts Group (“MPEG”) has recently commenced the MPEG-21 Multimedia Framework initiative in order to develop a solution. As further described in International Organisation for Standardisation (“ISO”) document ISO/IEC JTC1/SC29WG11/N5231 (2002), one of the goals of MPEG-21 is develop a technique for delivering different types of content in an integrated and harmonized way, so that the content delivery process is entirely transparent to a wide spectrum of multimedia users.
  • In order to accomplish such a technique, part 7 of MPEG-7 proposes the concept of what is called “Digital Item Adaptation.” That concept involves the adaptation of resources and descriptions that constitute a digital item to achieve interoperable transparent access to universal multimedia from any type of terminal and network. By implementing Digital Item Adaptation, users. in a network would be unaware of network and terminal-specific issues that often affect the delivery of multimedia content, such as network congestion, quality limitations, and reliability of service. It is envisioned that a diverse community of users will therefor be able to share a multimedia experience, each to his or her individual acceptable level of quality.
  • Probably transcoding, which avoids the need to store content in different compressed formats for different network bandwidths and different terminals, is one of the most common methods of resource adaptation. In MPEG-7, so called Transcoding Hints have been proposed in order to enable better transcoding by reducing computation complexity while preserving quality as much as possible.
  • Unfortunately, the proposed MPEG-7 Transcoding Hints do not provide information about feasible transcoding operators and their expected performance in order to meet specific target rates. They likewise do not provide a solution that may be useful to fulfill the multiple requirements necessary to ensure a transparent, adaptive multimedia content delivery. Accordingly, there remains a need for a technique for delivering multiple types of multimedia content over a network to a wide spectrum of multimedia users having different acceptable levels of quality.
  • SUMMARY OF THE INVENTION
  • An object of the present invention is to provide a technique for delivering multiple types of multimedia content over a network to a wide spectrum of multimedia users having different acceptable levels of quality.
  • Another object of the present invention is to provide multimedia content description techniques that are useful to fulfill several requirements.
  • In order to meet these. and other objects of the present invention, which will become apparent with reference to further disclosure set forth below, the present invention provides techniques for generating utility-based descriptors from compressed multimedia information. A preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate a quality measurement for each adapted multimedia segment, and generating a utility based descriptor based on the portions of utility based descriptor information and corresponding quality measurements.
  • In a preferred embodiment, the compressed multimedia information is MPEG-4 data, and from ten to twenty portions of utility based descriptor information are utilized. The portions of utility based descriptor information may be uniformly or non-uniformly sampled. Advantageously, the adaptation operations may include frame dropping, either by dropping first B frames or all B frames, and may further include coefficient dropping.
  • In another embodiment, the present invention provides systems and methods for delivering compressed multimedia information to two or more users, each having different target bit rates. In one arrangement, a method includes the steps of receiving at least a segment of compressed multimedia information and a corresponding utility based descriptor, parsing the utility based descriptor into portions, each corresponding to a unique target bit rate for each of the users, selecting a utility based descriptors portion that corresponds to the unique target bit rate for each user, and adapting the compressed multimedia segment by the selected utility based descriptors portion for each user. Target bit rate feedback information from the users, or from the network, may be utilized in the adaptation step.
  • The accompanying drawings, which are incorporated and constitute part of this disclosure, illustrate preferred embodiments of the invention and serve to explain the principles of the invention.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is functional diagram showing the relationships among adaptation spage, utility space, and resource space;
  • FIG. 2 is a block diagram of an exemplary system in accordance with the present invention;
  • FIG. 3 is an illustrative diagram showing a two dimensional adaptation space defined by a combination of frame dropping and coefficient dropping;
  • FIG. 4 is a graph showing an exemplary utility function in accordance with the present invention;
  • FIGS. 5( a)-(c) are graphs showing variations of the exemplary utility function shown in FIG. 4;
  • FIG. 6 is a schematic diagram of an exemplary utility based description tool in accordance with the present invention; and
  • FIG. 7 is a schematic diagram of an exemplary utility-based descriptor in accordance with the present invention.
  • Throughout the Figs., the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the present invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • Referring to FIG. 1, an exemplary embodiment of the present invention will be described. A utility-based framework provides a systematic way of efficient video adaptation by modeling the relationships among essential parameters: adaptation operations, resources, and utility. In general, adaptation operations can take the form of spatial domain adaptation, temporal adaptation, or object-based adaptation. Spatial domain adaptation may include spatial resolution reduction and quality or signal-to-noise adaptation, such as requantization or DCT coefficient dropping. Temporal domain adaptation may include frame dropping, and object-based adaptation may include video object prioritization and/or dropping. A particular operation defined by any of such adaptation methods is referred to herein as an adaptation operation.
  • Resources include available support from terminal devices and network capabilities like bandwidth, computation capability, power, and display size, etc. Utility includes the quality of content resulted from a particular adaptation operation. Utility can be measured in an objective manner, such as by determining the peak signal-to-noise ratio (“PSNR”), or a subjective one, e.g., by use of a subjective quality score. FIG. 1 illustrates the multi-dimensional space of adaptation, resources, and utility and the relations among them as applied to MPEG-4 compressed video.
  • The adaptation space 110 represents the conceptual space of all possible adaptation operations for one or more selected adaptation methods. Each dimension of the adaptation space represents one type of adaptation method and has a certain cardinal index representing associated adaptation operations. For example, where frame dropping and coefficient dropping are both utilized, there are two dimensions in the adaptation space: frame dropping and coefficient dropping. The dimension of frame dropping can be indexed by the amount of frames dropped, e.g., no dropping, all B frames dropped in a sub-Group of Pictures (“GOP”) (a sub-GOP includes a set of successive frames beginning with an I or P frame and continuing to the next I or P frame), all B and P frames dropped in each GOP. The coefficient dropping dimension can be indexed by the percentage of rate-reduction to be achieved by coefficient dropping, e.g., no dropping, 10%, 20%, etc. In this way, a set of discrete points in the adaptation space can be defined, each point representing an adaptation operation specified by a particular combination of frame dropping and coefficient dropping.
  • In some applications, the resource limitation may include several types of resources. For example, in order to provide video-streaming service to certain handheld devices, factors such as spatial resolution or computational capability should also be taken into account along with bandwidth. In general, all types of resources to be satisfied are represented by a multidimensional resource space. Utility space may include attributes in multiple dimensions. In addition to PSNR, the subjective preference like mean opinion scale (“MOS”), temporal smoothness may be included in other dimensions together.
  • Referring again to FIG. 1, a video segment 101 is a unit undergoing adaptation, with each point representing a particular adaptation operation in the adaptation space. The adapted video segment has the resulting values of resources and utilities represented as corresponding points in the resource and utility spaces, respectively. The shaded cube in the resource space represents the resource constraints specified by applications. Note that there may exist multiple adaptation operations that satisfy the same resource requirement. The oval shaped region in the adaptation space that is mapped into a point in the resource space shows such a constant-resource set. Also, different adaptation operators may result in the same utility value. The rectangular region in the adaptation space represents such a constant-utility set.
  • Using the utility-based framework, video adaptation can be formulated as follows: given certain resource constraints, determine the optimal adaptation operation so that the utility of the adapted video is maximized. Since most adaptation problems likely to be assumed in the UMA paradigm can be formulated so formulated, such resource-constrained utility maximization may be considered to be a basic scenario of multimedia adaptation. While the disclosure herein is directed to optimizing a frame and coefficient dropping transcoding to satisfy available bandwidth as an example of resource-constrained utility maximization, those skilled in the art should appreciate that the utility-based framework of the present invention can readily include constraints in the utility space and aim at overall resource minimization.
  • Referring next to FIG. 2 a system in accordance with the present invention will now be described. A server computer 210 is adapted to receive stored video 211 and/or live video 212. That video is preferably in a compressed format, such as MPEG-1, MPEG-2, or MPEG-4, although uncompressed digital video could be provided to the server, with compression occurring thereon. The server 210 includes software written in any available programming language for generating a utility function, in the form of utility-based descriptors, based on the received video. In accordance with the present invention and described in further detail below, that descriptors is indicative of certain modifications to the compressed video, e.g., through the elimination of bidirectionally-predictive (“B”) frames or coefficients, that will result in predetermined levels of quality. The compressed domain video and associated utility function are delivered over a transit network 220, such as the Internet or an intranet of sufficient bandwidth to transmit the compressed video. The transmitted information is received by a network computer 230, which in turn serves as the video adaptation engine of the system.
  • In particular, the network computer 230 includes software, again written in any available programming language, to adapt the incoming compressed video to the particular bandwidth requirements of several client devices 250, 251, 252, 253 that are served by associated access networks 240. In accordance with the present invention and described in further detail below, the network computer 230 uses the utility-based descriptors generated by the server 210 to adapt the compressed video to such bandwidth requirements. Further, the network computer 230 may receive preference information 241 from the client users, and/or available bandwidth information 242 from the network, in order to optimize its adaptation operation.
  • The access network 240 may be the Internet, an intranet, or a proprietary network such as a wireless network that links mobile cellular user terminals 253 to the network computer 230. In the application of video streaming over bandwidth-limited network, the bit rate of a video stream to be delivered is adapted to time-varying bandwidth by an adaptation tool in real-time.
  • In a preferred arrangement, a combination of frame dropping and coefficient dropping are used by the server computer 210 for the adaptation of nonscalable video to dynamic bandwidth. However, those skilled in the art should appreciate that other transcoding techniques could be utilized to adjust the bit-rate of video streams for dynamic bandwidth adaptation, such as re-encoding, re-quantization of DCT coefficients, object-based transcoding, and image-size reduction. The Fine-Granular-Scalability (“FGS”) and some of its variant forms that have been adopted as new scalable coding tools in MPEG-4 also enable the dynamic adaptation of a FGS-stream to time-varying bandwidth by selecting appropriate number of bitplanes of scalable streams.
  • Frame and coefficient dropping are simple ways of rate adaptation with low computational complexity since they involve the truncation of portions of a bit sequence corresponding to particular frames and symbols of DCT coefficients to be dropped by a compressed-domain processing. Further, for the application of video streaming over mobile wireless networks, they are more suitable for low delay real-time operation that is strongly required in a trancoding proxy.
  • Moreover, the combination of frame and coefficient dropping enables the adaptation of the rate of a video stream by adjusting spatial and temporal qualities: frame dropping adjusts frame rate by dropping some frames, and coefficient dropping adjusts spatial quality by dropping some of DCT coefficients that are associated with higher frequency components. The dynamic range of rate reduction is increased by combining two or more transcoding methods.
  • Frame dropping will next be descried. Frame dropping is a typical kind of temporal transcoding that adjusts frame rate by dropping some frames from an incoming video stream. It is often used for rate adaptation to bandwidth variations in video streaming applications because of its efficiency and simplicity. One factor should be considered is the selection of the frames to be dropped. For example, when an intra-coded frame (an “I frame”) or certain predectivly coded frames (a “P” frame) are dropped, frames that refer to the dropped frame need to be re-encoded.
  • Thus, it is preferred that only B frames and/or P frames that do not have a decoding dependency are dropped, in the unit a group of pictures (“GOP”), by taking into account the sequence structure of an incoming video stream. Frame dropping provides only a coarse approximation to the target rate since the smallest unit of data that can be removed is an entire frame. Therefore, the possible frame dropping operations are defined by specifying the frame type to be dropped rather than the reduction rate to be achieved by the dropping.
  • For a GOP having the sub-group of 3 pictures between anchor frames (M=3), a set of frame dropping operations depending on an assumed GOP structure may be defined as follows: no frame dropping; one B frame dropped in each sub-GOP, all B frames dropped, and all B and P frames dropped, resulting in an I frame only sequence. For a GOP having a sub-group of one I picture between two successive anchor frames (M=1), it is assumed that P frames are dropped from the end of each GOP, such as the last P frame dropped, the two last P frames dropped, etc. through all P frames dropped in each GOP.
  • Although the selection of the frames to be dropped frame is limited, this approach may be sufficient enough in terms of the amount of the bit rate reduction and quality alone, or may be combined with coefficient dropping (to be discussed below) in order to balance the desired temporal adaptation of frame dropping with the spatial adaptation of coefficient dropping. It should be noted that dropping frames may cause frame jerkiness since the dropped frames usually are replaced by previous frames. In the first case of GOP structure that has more than one picture between anchor frames (M >1), the defined transcoding operations evenly distribute the dropped frames in the temporal range results in more comfortable temporal quality. On the other hand, a special dynamic player is needed that can adjust the presentation time for each decoded frame from the transcoded stream in the case of GOP with (M=1) to reduce annoying effect cause by non-uniform frame dropping within a GOP.
  • Coefficient dropping will next be described. There are two fundamental ways in spatial adaptation that perform operations in the frequency domain on DCT coefficients. The first is requantization, i.e., the modification of quantized coefficients by employing coarser quantization levels to reduce bit rate. The second is coefficient dropping in which higher frequency coefficients which are less important for image quality are truncated. Coefficient dropping is preferred since it is more amenable to fast processing than requantization, which requires the implementation of recoding-type algorithms.
  • More specifically, assuming that a set of DCT coefficient run-length codes at the end of each block is eliminated, the number of DCT coefficient codes within each block that will be kept after truncation is called breakpoint. The breakpoint for each block may be determined using Lagrangian optimization, which minimizes the distortion caused by the coefficient dropping while meeting the required target rate in a frame-by-frame basis. In the rate-distortion formulation of the optimization, an algorithm which does not require memory can be employed, with such an algorithm ignoring accumulated errors caused by motion compensation and treating each picture as an intra-coded one due to its simplicity. Ignoring the accumulated errors does not much affect the quality and allows achieving essentially optimal (within 0.3 dB) performance.
  • In a given video segment and the target rate, we first assume uniform dropping that gives uniform rate reduction across different frames. Then, within a single frame, we perform the above optimal non-uniform dropping that gives different rate reductions with different breakpoints among blocks, while meeting the target rate of the given frame.
  • Unlike frame dropping, in which the reducible rates are limited to several values since the smallest unit of data which can be removed is an entire frame, coefficient dropping provides the ability to meet and available bandwidth quite accurately within the upper bound of rate reduction by adjusting the amount of dropped coefficients. Preferably, only AC DCT coefficients are dropped in order to avoid somewhat complicated syntax changes that caused by when all coefficients are dropped, and to ensure a minimum necessary quality. The upper bound of rate reduction depends on an incoming video stream. Numerous coefficient-dropping operations may be defined by specifying the percentage of rate reduction to be achieved, rather than directly specifying the dropped coefficients themselves. For example, the operation of coefficient dropping (10%) represents a 10% reduction of the bit rate of incoming video stream by coefficient dropping.
  • The combination of frame and coefficient dropping is next described For higher bit rate reduction, frame dropping or coefficient dropping alone may not be sufficient to accommodate available bandwidth. Moreover, only a few discrete points can be achievable by frame dropping, while a continuous rate adaptation is possible by using coefficient dropping. Therefore, the combination of frame dropping and coefficient dropping enables the extension of dynamic range of the reducible rate. The combination of both may also yield better perceptual quality than either technique alone, especially for large rate reductions, by optimizing the trade-off between spatial and temporal quality. For example, in order to reduce frame jerkiness at very low frame rates, temporal resolution can be traded with the spatial quality while meeting the same rate reduction.
  • Referring next to FIG. 3., a two-dimensional adaptation space defined by the combination of frame dropping and coefficient dropping is shown. Each point represents a frame dropping/coefficient dropping-combined transcoding operation. Note that the effect of the order of operations should be considered in the combination of coefficient and frame dropping. For example, there are two combinations having different orders of operation to achieve the same point 310: either 20% coefficient dropping followed by B frame dropping, or B frame dropping followed by 20% coefficient dropping. The results of both cases are the same if rate-based uniform coefficient dropping in which the same rate-reduction is applied across the frames is employed. However, in the case that different reduction ratios are assigned among frames to achieve global optimum coefficient dropping based on rate allocation, different operation orders may result in different results of reduced rate and quality. While the present disclosure is directed to the former, the present invention contemplates both scenarios.
  • The generation of a utility function will next be described. In general, the relationships among the adaptation space, resource space, and utility space shown in FIG. 1 can be modeled based on a utility function. A utility function may be defined as a media quality metric representing a user satisfaction index as a function of resources. In the context of the present invention, the adaptation space is a two-dimensional space specifying combinations of frame dropping and coefficient dropping, the resource space includes the available bandwidth varied with time, and the utility space includes the signal to noise measure of the transcoded video stream.
  • Referring next to FIG. 4, an exemplary utility function generated by a combined frame dropping/coefficient dropping transcoding method applied to previously stored MPEG-4 compressed video data, “coastguard” coded at 1.5 Mbps and adapted over a bandwidth range less than 200 kbps, is shown. FIG. 4 is a graph plotting target rate in kbits/sec against PSNR, and illustrates four curves 420, 420, 430, 440 that represent the relationship between the target rates and PSNR qualities, each for a different adaptation operation within the exemplary utility function.
  • In the example, four different frame-dropping operations and six types of coefficient dropping operations are utilized. The frame dropping operations consist of no frames dropped, one B-frame dropped in each sub-GOP, all B-frames dropped, and all B- and P-frames dropped. The six coefficient dropping operation are set at 0%, 10%, 20%, 30%, 40%, and 50% reduction of the bit rate of the original test video stream. In this way, there are 23 combined operations, which employ different combinations of the defined frame dropping and coefficient dropping operations. Those 23 operations are shown as discrete points in curves 420, 430, 440, and 450, which illustrate the set of points for the various coefficient dropping operations when no frames are dropped 420, one B-frame dropped 430, all B-frames are dropped 440, and all B- and P-frames are dropped 450, respectively.
  • FIG. 4, also illustrates a re-encoding curve 410, obtained by a cascaded full-decoding and re-encoding, and thus may be considered as a reference for performance comparison of the transcoding operations. It is important to note that for a given target bandwidth, there are multiple adaptation operations satisfying the same target rate. The optimal operation with the highest video utility is selected.
  • As shown in FIG. 4, the utility function depends on the type of video content, the chosen coding parameters of an incoming video stream, and the applied transcoding method. Given video segments which share the same content type and transcoding method, the generation of a utility function requires the repetitive computation of PSNR quality and rate for a family of defined adaptation operations by testing all possible operations.
  • Utility function generation for live video will next be described. For previously recorded video, the utility function can be generated by off-line processing in a server in which computational cost is not a concern, such as in the case of FIG. 4. However, this option is generally not an acceptable solution for live video, given the reed for such extensive repetitive computation. Accordingly, a content-based utility prediction solution may be used to predict a utility function in the case of live video.
  • In general, video can be mapped to distinctive utility distribution classes prepared in advance based on computable content features, such as motion activity and spatial activity extracted from compressed streams. Accordingly, a utility function corresponding to an expected incoming stream of video is prepared in advance for live video.
  • Forming a prediction for a live utility function is a two step process. First, an adaptive content classification loop is employed; second, a real-time estimation path is utilized. A set of utility functions that can cover entire types of content are generated and classified in the adaptive content classification loop off-line. Later, when a live video stream is received, the real-time estimation path selects an associated utility function for each video segment in order to preserve the same content type in real-time.
  • The description of utility functions will next be described. In the utility-based framework, the utility function that represents distributions of the adaptation, resource, and utility spaces, as well as the relationships among them, are delivered along with an associated video stream to an adaptation engine, e.g., located on network computer 230. The main goal of the descriptor is to describe the distributions of the three spaces (adaptation, resource, and utility) and the relationships among them to support various types of usage scenarios in an efficient way. The descriptor should provide sufficient information to the adaptation engine as to what are possible adaptation operations satisfying constrained resources and the associated utilities.
  • In order to describe a utility function such as the of FIG. 4, the range of bit rates are sampled into a finite set of points, and then all feasible frame dropping-coefficient dropping-combined operations capable of achieving the resource and the associated values of PSNR are described using the sampled resource points as indexes. In general, a finite set of points over the multi-dimensional resource. space is defined as indexes in the description.
  • Linear or non-linear sampling of the resource space can be selected depending on the characteristics of distributions of adaptation space by taking into account the efficiency of description as well as the number of sampling points. Interpolation between two consecutive points of resource and corresponding adaptation operations and utilities may also occur in a linear or non-linear manner. In the case of adaptation, however, it should be noted that interpolation is not feasible between different frame dropping operations unlike coefficient dropping.
  • By specifying a particular adaptation method, a constrained resource, and utility according to intended applications, the descriptor can support most cases of resource-constrained scenarios.
  • Some adaptation operations may not be uniquely defined in terms of quality. For example, an operation of “10% reduction of bit rate by dropping DCT coefficients in each frame (represented as coefficient dropping (10%))” does not specify the exact set of coefficients to be dropped. Different implementations may choose different sets and result in slightly different utility values. As a result, the value of utility associated to a particular operation is not reliable.
  • On the other hand, some adaptation methods do not cause the ambiguity issue due to their unambiguous representation formats in terms of adaptation. For example, some scalable compression formats, such as JPEG-2000 and MPEG-4 FGS, provide unambiguously defined scalable layers. Subsets of the layers can be truncated in a consistent way with the same resulted quality as long as the decoders are compliant with the standards.
  • Quality ranking may be employed in order to address this ambiguity issue. In some applications, the absolute values of utility of each adapted media are not important, but instead, the relative ranking of such values among different adaptation operations satisfying the same resource may be critical. In such cases, the likelihood of achieving ranking consistence is higher than absolute value consistence. In this sense, the descriptor describe the ranking instead of the utility value to provide the notion of quality even the case that the quality values are not reliable due to the ambiguity. In addition, the descriptors may include a flag to represent whether or not the ranking is consistent over different implementations. Assuming there exists some consistence among practical implementations, the empirical value of the flag may be obtained.
  • Referring next to FIGS. 5( a)-(c), the variation of utility functions resulting from different implementations of coefficient dropping to obtain the value of the consistency flag is shown. FIG. 5( a) is a reproduction of FIG. 4; FIG. 5( b) shows the same curves applied to the same data, except that macroblock optimizing is selected; and FIG. 5( c) again shows the same curves applied to the same data, except with pure rate-based uniform coefficient dropping is used, without optimization across blocks.
  • As shown in FIGS. 5( a)(c), there are noticeable variations of utility values among utility functions with different implementations. There may be several operations with different qualities achieving the same bit rate. In some parts of the bit rate range excepting the range covered by shaded box in FIG. 5( c), the rank of such equal-rate operations in terms of quality is consistent among different implementations. Even in the shaded box, there is constancy of rank depending on operations. Namely, the operation of all B frame coefficient dropping and coefficient dropping has the worst utility regardless of implementations. Based on this observation, the descriptor describes the rank and the optional flag for each operator to represent the consistency of the rank fully.
  • Referring next to FIG. 6, an exemplary utility-based descriptor is shown. The descriptor provides a set of adaptation descriptors 610, each of which describes a utility function associated with an adaptation method by including elements of resource, utility, and utility function The descriptor enables the selection of a defined adaptation method according to the intended scenario by specifying one of the enumerated ones by an attribute; such as combined frame and coefficient dropping.
  • The Resource 620 and Utility 630 descriptors define constrained resource and utility associated to the utility function 640 to be described in terms of name and unit, respectively. Especially, multiple instantiations of the Resource field 620 are allowed to accommodate the multi dimensional resource space. The UtilityFunction descriptor 640 represents a set of possible adaptation operators and associated utilities as function of resource points.
  • Referring next to FIG. 7, an exemplary UtilityFunction descriptor 640 is shown. The UtilityFunction descriptor 640 includes a set of ResourcePoints 710, each of which includes a set of AdaptationOperators 720 to describe all possible adaptation operations satisfying the sampled values of constrained resources that are described by ResourceValues 730. A particular adaptation operation of the specific adaptation method is described by selecting corresponding element. For example, FrameCoeffDropping 740 may be used for describing a particular operation of frame dropping/coefficient dropping-combined transcoding by specifying the type and number of frame to be dropped and the percentage of bit rate to be reduced by truncating coefficients. As noted above, other operations may be used, such as WaveletReduction 750 in order to describe the specific operation of wavelet reduction by specifying the number of levels and bitplanes to be truncated. The adaptation operator FGS 770 may be used to describe the specific operation of an MPEG-4 Fine Granularity Scalability (“FGS”) stream by specifying the number of bitplanes of FGS-frames, and/or the number of bitplanes of FGST-frames to be truncated from the enhancement layer.
  • In addition to adaptation operation, the associated utility value is described by the UtilityValue 760. Where adaptation method is subject to ambiguity in specifying the adaptation operation, the UtilityRankInformation 761 is instantiated instead of the UtilityValue to describe the rank of the associated operation with an optional attribute of a consistency Flag representing the consistence of the rank.
  • The foregoing merely illustrates the principles of the invention. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods which, although not explicitly shown or described herein, embody the principles of the invention and are thus within the spirit and scope of the invention.

Claims (23)

1-48. (canceled)
49. A method of delivering varying qualities of audiovisual content to one or more users, comprising:
for each user, determining a level of quality constraint;
receiving multimedia content to be transcoded;
using a computing apparatus, for each user, in real time, transcoding the multimedia content to meet the level of quality constraint for the respective user;
delivering the transcoded multimedia content to the respective user.
50. The method of claim 49, wherein transcoding includes spatial domain adaptation of the multimedia content.
51. The method of claim 49, wherein transcoding includes temporal adaptation of the multimedia content.
52. The method of claim 49, wherein transcoding includes object-based adaptation of the multimedia content.
53. The method of claim 49, wherein transcoding includes frame dropping.
54. The method of claim 49, wherein transcoding includes coefficient dropping.
55. The method of claim 49, wherein the level of constraint includes spatial resolution.
56. The method of claim 49, wherein the level of constraint includes computational capability.
57. The method of claim 49, wherein the level of constraint includes bandwidth limitations.
58. The method of claim 49, wherein transcoding the multimedia content comprises two or more transcoding operations.
59. A system for delivering varying qualities of audiovisual content to one or more users, comprising:
an application receiving, for each user, a level of quality constraint;
An application receiving multimedia content to be transcoded;
an application transcoding the multimedia content to meet the level of quality constraint for each user in real time;
a network application delivering the transcoded multimedia content to the respective user.
60. A method for adapting multimedia content to meet level of quality constraints in real-time, comprising:
receiving one or more quality constraints from a user;
receiving multimedia content to be transcoded;
determining, using a computing apparatus, in real-time, one or more adaptation operations to be applied to the multimedia content to meet the one or more quality constraints;
and applying the one or more adaptation operations to meet the one or more quality constraints.
61. The method of claim 60, wherein adaptation includes spatial domain adaptation.
62. The method of claim 60, wherein adaptation includes temporal adaptation.
63. The method of claim 60, wherein adaptation includes object-based adaptation.
64. The method of claim 60, wherein adaptation includes frame dropping.
65. The method of claim 60, wherein adaptation includes coefficient dropping.
66. The method of claim 60, wherein the level of constraint includes spatial resolution.
67. The method of claim 60, wherein the level of constraint includes computational capability.
68. The method of claim 60, wherein the level of constraint includes bandwidth limitations.
69. The method of claim 60, wherein applying the adaptation operation comprises applying two or more adaptation operations.
70. A system for adapting multimedia content to meet level of quality constraints in real-time, comprising:
an application receiving one or more quality constraints from a user;
an application receiving multimedia content to be transcoded;
an application determining, in real-time, one or more adaptation operations to be applied to the multimedia content to meet the one or more quality constraints; and
applying the one or more adaptation operations to meet the one or more quality constraints.
US12/548,199 2002-04-26 2009-08-26 Method And System For Optimal Video Transcoding Based On Utility Function Descriptors Abandoned US20090316778A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/548,199 US20090316778A1 (en) 2002-04-26 2009-08-26 Method And System For Optimal Video Transcoding Based On Utility Function Descriptors

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US37612902P 2002-04-26 2002-04-26
US38493902P 2002-05-31 2002-05-31
PCT/US2003/012858 WO2003091850A2 (en) 2002-04-26 2003-04-25 Method and system for optimal video transcoding based on utility function descriptors
US10/965,040 US8218617B2 (en) 2002-04-26 2004-10-14 Method and system for optimal video transcoding based on utility function descriptors
US12/548,199 US20090316778A1 (en) 2002-04-26 2009-08-26 Method And System For Optimal Video Transcoding Based On Utility Function Descriptors

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/965,040 Continuation US8218617B2 (en) 2002-04-26 2004-10-14 Method and system for optimal video transcoding based on utility function descriptors

Publications (1)

Publication Number Publication Date
US20090316778A1 true US20090316778A1 (en) 2009-12-24

Family

ID=29273050

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/965,040 Expired - Fee Related US8218617B2 (en) 2002-04-26 2004-10-14 Method and system for optimal video transcoding based on utility function descriptors
US12/548,199 Abandoned US20090316778A1 (en) 2002-04-26 2009-08-26 Method And System For Optimal Video Transcoding Based On Utility Function Descriptors

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/965,040 Expired - Fee Related US8218617B2 (en) 2002-04-26 2004-10-14 Method and system for optimal video transcoding based on utility function descriptors

Country Status (6)

Country Link
US (2) US8218617B2 (en)
EP (1) EP1532812A4 (en)
JP (1) JP2005525011A (en)
KR (1) KR20050007348A (en)
AU (1) AU2003231102A1 (en)
WO (1) WO2003091850A2 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100050225A1 (en) * 2008-08-25 2010-02-25 Broadcom Corporation Source frame adaptation and matching optimally to suit a recipient video device
US20110064136A1 (en) * 1997-05-16 2011-03-17 Shih-Fu Chang Methods and architecture for indexing and editing compressed video over the world wide web
US8364673B2 (en) 2008-06-17 2013-01-29 The Trustees Of Columbia University In The City Of New York System and method for dynamically and interactively searching media data
US8370869B2 (en) 1998-11-06 2013-02-05 The Trustees Of Columbia University In The City Of New York Video description system and method
US8488682B2 (en) 2001-12-06 2013-07-16 The Trustees Of Columbia University In The City Of New York System and method for extracting text captions from video and generating video summaries
US8671069B2 (en) 2008-12-22 2014-03-11 The Trustees Of Columbia University, In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US20140269938A1 (en) * 2013-03-15 2014-09-18 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
US8849058B2 (en) 2008-04-10 2014-09-30 The Trustees Of Columbia University In The City Of New York Systems and methods for image archaeology
US9060175B2 (en) 2005-03-04 2015-06-16 The Trustees Of Columbia University In The City Of New York System and method for motion estimation and mode decision for low-complexity H.264 decoder
US9071841B2 (en) 2011-05-17 2015-06-30 Microsoft Technology Licensing, Llc Video transcoding with dynamically modifiable spatial resolution
US9747013B2 (en) 2014-01-06 2017-08-29 Dropbox, Inc. Predictive caching and fetch priority

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3806118B2 (en) 2003-06-13 2006-08-09 ローム アンド ハース カンパニー Method for marking hydrocarbons with substituted anthraquinones.
US8862866B2 (en) * 2003-07-07 2014-10-14 Certicom Corp. Method and apparatus for providing an adaptable security level in an electronic communication
CA2478274C (en) 2003-08-19 2015-12-08 Certicom Corp. Method and apparatus for synchronizing an adaptable security level in an electronic communication
GB0428155D0 (en) * 2004-12-22 2005-01-26 British Telecomm Buffer underflow prevention
US8422546B2 (en) * 2005-05-25 2013-04-16 Microsoft Corporation Adaptive video encoding using a perceptual model
US8874477B2 (en) 2005-10-04 2014-10-28 Steven Mark Hoffberg Multifactorial optimization system and method
US7555715B2 (en) 2005-10-25 2009-06-30 Sonic Solutions Methods and systems for use in maintaining media data quality upon conversion to a different data format
US7995649B2 (en) 2006-04-07 2011-08-09 Microsoft Corporation Quantization adjustment based on texture level
US8503536B2 (en) 2006-04-07 2013-08-06 Microsoft Corporation Quantization adjustments for DC shift artifacts
US8059721B2 (en) 2006-04-07 2011-11-15 Microsoft Corporation Estimating sample-domain distortion in the transform domain with rounding compensation
CA2644015C (en) * 2006-04-13 2016-10-18 Certicom Corp. Method and apparatus for providing an adaptable security level in an electronic communication
US8711925B2 (en) 2006-05-05 2014-04-29 Microsoft Corporation Flexible quantization
US8250618B2 (en) * 2006-09-18 2012-08-21 Elemental Technologies, Inc. Real-time network adaptive digital video encoding/decoding
FR2907989B1 (en) * 2006-10-27 2009-01-16 Actimagine Sarl METHOD AND DEVICE FOR OPTIMIZING THE COMPRESSION OF A VIDEO STREAM
US8804829B2 (en) 2006-12-20 2014-08-12 Microsoft Corporation Offline motion description for video generation
US8238424B2 (en) 2007-02-09 2012-08-07 Microsoft Corporation Complexity-based adaptive preprocessing for multiple-pass video compression
US8498335B2 (en) 2007-03-26 2013-07-30 Microsoft Corporation Adaptive deadzone size adjustment in quantization
US20080240257A1 (en) * 2007-03-26 2008-10-02 Microsoft Corporation Using quantization bias that accounts for relations between transform bins and quantization bins
US8243797B2 (en) 2007-03-30 2012-08-14 Microsoft Corporation Regions of interest for quality adjustments
US8442337B2 (en) 2007-04-18 2013-05-14 Microsoft Corporation Encoding adjustments for animation content
US8331438B2 (en) 2007-06-05 2012-12-11 Microsoft Corporation Adaptive selection of picture-level quantization parameters for predicted video pictures
US8184715B1 (en) 2007-08-09 2012-05-22 Elemental Technologies, Inc. Method for efficiently executing video encoding operations on stream processor architectures
US8121197B2 (en) 2007-11-13 2012-02-21 Elemental Technologies, Inc. Video encoding and decoding using parallel processors
US8189933B2 (en) 2008-03-31 2012-05-29 Microsoft Corporation Classifying and controlling encoding quality for textured, dark smooth and smooth video content
WO2009133427A1 (en) * 2008-04-28 2009-11-05 Nds Limited Frame accurate switching
US8897359B2 (en) 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
JP2010062926A (en) * 2008-09-04 2010-03-18 Nec Personal Products Co Ltd Video recording apparatus and video compression program
US9002881B2 (en) * 2009-10-29 2015-04-07 Microsoft Technology Licensing, Llc Assembling streamed content for on-demand presentation
CN104272735B (en) * 2013-01-16 2018-04-13 黑莓有限公司 For the transform coefficients encoding of the context adaptive binary entropy coding of video
KR102085270B1 (en) 2013-08-12 2020-03-05 삼성전자 주식회사 Method for selecting resolution with minimum distortion value and devices performing the method
CN107808389B (en) * 2017-10-24 2020-04-17 上海交通大学 Unsupervised video segmentation method based on deep learning
US20230034162A1 (en) * 2020-01-09 2023-02-02 Sony Group Corporation Transmission apparatus and transmission method

Citations (97)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4649482A (en) * 1984-08-31 1987-03-10 Bio-Logic Systems Corp. Brain electrical activity topographical mapping
US4649380A (en) * 1983-06-15 1987-03-10 U. S. Philips Corporation Video display system comprising an index store for storing reduced versions of pictures to be displayed
US5191645A (en) * 1991-02-28 1993-03-02 Sony Corporation Of America Digital signal processing system employing icon displays
US5204706A (en) * 1990-11-30 1993-04-20 Kabushiki Kaisha Toshiba Moving picture managing device
US5208857A (en) * 1990-04-25 1993-05-04 Telediffusion De France Method and device for scrambling-unscrambling digital image data
US5408274A (en) * 1993-03-11 1995-04-18 The Regents Of The University Of California Method and apparatus for compositing compressed video data
US5488664A (en) * 1994-04-22 1996-01-30 Yeda Research And Development Co., Ltd. Method and apparatus for protecting visual information with printed cryptographic watermarks
US5493677A (en) * 1994-06-08 1996-02-20 Systems Research & Applications Corporation Generation, archiving, and retrieval of digital images with evoked suggestion-set captions and natural language interface
US5585852A (en) * 1993-06-16 1996-12-17 Intel Corporation Processing video signals for scalable video playback using independently encoded component-plane bands
US5605655A (en) * 1994-04-11 1997-02-25 Mitsubishi Jukogyo Kabushiki Kaisha Gas-liquid contacting apparatus
US5606655A (en) * 1994-03-31 1997-02-25 Siemens Corporate Research, Inc. Method for representing contents of a single video shot using frames
US5613032A (en) * 1994-09-02 1997-03-18 Bell Communications Research, Inc. System and method for recording, playing back and searching multimedia events wherein video, audio and text can be searched and retrieved
US5615112A (en) * 1993-01-29 1997-03-25 Arizona Board Of Regents Synthesized object-oriented entity-relationship (SOOER) model for coupled knowledge-base/database of image retrieval expert system (IRES)
US5623690A (en) * 1992-06-03 1997-04-22 Digital Equipment Corporation Audio/video storage and retrieval for multimedia workstations by interleaving audio and video data in data file
US5630121A (en) * 1993-02-02 1997-05-13 International Business Machines Corporation Archiving and retrieving multimedia objects using structured indexes
US5708805A (en) * 1992-10-09 1998-01-13 Matsushita Electric Industrial Co., Ltd. Image retrieving apparatus using natural language
US5713021A (en) * 1995-06-28 1998-01-27 Fujitsu Limited Multimedia data search system that searches for a portion of multimedia data using objects corresponding to the portion of multimedia data
US5721815A (en) * 1995-06-07 1998-02-24 International Business Machines Corporation Media-on-demand communication system and method employing direct access storage device
US5724484A (en) * 1991-03-20 1998-03-03 Hitachi, Ltd. Data processing methods and apparatus for supporting analysis/judgement
US5734893A (en) * 1995-09-28 1998-03-31 Ibm Corporation Progressive content-based retrieval of image and video with adaptive and iterative refinement
US5734752A (en) * 1996-09-24 1998-03-31 Xerox Corporation Digital watermarking using stochastic screen patterns
US5742283A (en) * 1993-09-27 1998-04-21 International Business Machines Corporation Hyperstories: organizing multimedia episodes in temporal and spatial displays
US5751286A (en) * 1992-11-09 1998-05-12 International Business Machines Corporation Image query system and method
US5758076A (en) * 1995-07-19 1998-05-26 International Business Machines Corporation Multimedia server system having rate adjustable data retrieval based on buffer capacity
US5870754A (en) * 1996-04-25 1999-02-09 Philips Electronics North America Corporation Video retrieval of MPEG compressed sequences using DC and motion signatures
US5873080A (en) * 1996-09-20 1999-02-16 International Business Machines Corporation Using multiple search engines to search multimedia data
US5884298A (en) * 1996-03-29 1999-03-16 Cygnet Storage Solutions, Inc. Method for accessing and updating a library of optical discs
US5887061A (en) * 1996-05-01 1999-03-23 Oki Electric Industry Co., Ltd. Compression coding device with scrambling function and expansion reproducing device with descrambling function
US5893095A (en) * 1996-03-29 1999-04-06 Virage, Inc. Similarity engine for content-based retrieval of images
US6031914A (en) * 1996-08-30 2000-02-29 Regents Of The University Of Minnesota Method and apparatus for embedding data, including watermarks, in human perceptible images
US6037984A (en) * 1997-12-24 2000-03-14 Sarnoff Corporation Method and apparatus for embedding a watermark into a digital image or image sequence
US6041079A (en) * 1998-06-30 2000-03-21 Thomson Consumer Electronics, Inc, Field/frame conversion of DCT domain mixed field/frame mode macroblocks using 1-dimensional DCT/IDCT
US6047374A (en) * 1994-12-14 2000-04-04 Sony Corporation Method and apparatus for embedding authentication information within digital data
US6058186A (en) * 1990-04-23 2000-05-02 Canon Kabushiki Kaisha Information signal transmission system
US6167084A (en) * 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US6172675B1 (en) * 1996-12-05 2001-01-09 Interval Research Corporation Indirect manipulation of data using temporally related data, with particular application to manipulation of audio or audiovisual data
US6178416B1 (en) * 1998-06-15 2001-01-23 James U. Parker Method and apparatus for knowledgebase searching
US6185329B1 (en) * 1998-10-13 2001-02-06 Hewlett-Packard Company Automatic caption text detection and processing for digital images
US6195458B1 (en) * 1997-07-29 2001-02-27 Eastman Kodak Company Method for content-based temporal segmentation of video
US6208735B1 (en) * 1997-09-10 2001-03-27 Nec Research Institute, Inc. Secure spread spectrum watermarking for multimedia data
US6208746B1 (en) * 1997-05-09 2001-03-27 Gte Service Corporation Biometric watermarks
US6222932B1 (en) * 1997-06-27 2001-04-24 International Business Machines Corporation Automatic adjustment of image watermark strength based on computed image texture
US6223183B1 (en) * 1999-01-29 2001-04-24 International Business Machines Corporation System and method for describing views in space, time, frequency, and resolution
US6339450B1 (en) * 1999-09-21 2002-01-15 At&T Corp Error resilient transcoding for video over wireless channels
US20020021828A1 (en) * 2000-08-01 2002-02-21 Arthur Papier System and method to aid diagnoses using cross-referenced knowledge and image databases
US6356309B1 (en) * 1995-08-02 2002-03-12 Matsushita Electric Industrial Co., Ltd. Video coding device and video transmission system using the same, quantization control method and average throughput calculation method used therein
US6360234B2 (en) * 1997-08-14 2002-03-19 Virage, Inc. Video cataloger system with synchronized encoders
US6366701B1 (en) * 1999-01-28 2002-04-02 Sarnoff Corporation Apparatus and method for describing the motion parameters of an object in an image sequence
US6366314B1 (en) * 1997-12-17 2002-04-02 Telediffusion De France Method and system for measuring the quality of digital television signals
US20030013951A1 (en) * 2000-09-21 2003-01-16 Dan Stefanescu Database organization and searching
US6526099B1 (en) * 1996-10-25 2003-02-25 Telefonaktiebolaget Lm Ericsson (Publ) Transcoder
US20030046018A1 (en) * 2001-04-20 2003-03-06 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandeten Forschung E.V Method for segmentation and identification of nonstationary time series
US6532541B1 (en) * 1999-01-22 2003-03-11 The Trustees Of Columbia University In The City Of New York Method and apparatus for image authentication
US6546135B1 (en) * 1999-08-30 2003-04-08 Mitsubishi Electric Research Laboratories, Inc Method for representing and comparing multimedia content
US6549911B2 (en) * 1998-11-02 2003-04-15 Survivors Of The Shoah Visual History Foundation Method and apparatus for cataloguing multimedia data
US6556695B1 (en) * 1999-02-05 2003-04-29 Mayo Foundation For Medical Education And Research Method for producing high resolution real-time images, of structure and function during medical procedures
US6556958B1 (en) * 1999-04-23 2003-04-29 Microsoft Corporation Fast clustering with sparse data
US6678389B1 (en) * 1998-12-29 2004-01-13 Kent Ridge Digital Labs Method and apparatus for embedding digital information in digital multimedia data
US6683966B1 (en) * 2000-08-24 2004-01-27 Digimarc Corporation Watermarking recursive hashes into frequency domain regions
US6700935B2 (en) * 2002-02-08 2004-03-02 Sony Electronics, Inc. Stream based bitrate transcoder for MPEG coded video
US6701309B1 (en) * 2000-04-21 2004-03-02 Lycos, Inc. Method and system for collecting related queries
US6708055B2 (en) * 1998-08-25 2004-03-16 University Of Florida Method for automated analysis of apical four-chamber images of the heart
US20040057081A1 (en) * 2002-09-20 2004-03-25 Fuji Xerox Co., Ltd. Image processing method, manipulation detection method, image processing device, manipulation detection device, image processing program, manipulation detection program, and image formation medium
US6714909B1 (en) * 1998-08-13 2004-03-30 At&T Corp. System and method for automated multimedia content indexing and retrieval
US6716175B2 (en) * 1998-08-25 2004-04-06 University Of Florida Autonomous boundary detection system for echocardiographic images
US6718047B2 (en) * 1995-05-08 2004-04-06 Digimarc Corporation Watermark embedder and reader
US6721733B2 (en) * 1997-10-27 2004-04-13 Massachusetts Institute Of Technology Information search and retrieval system
US6725372B1 (en) * 1999-12-02 2004-04-20 Verizon Laboratories Inc. Digital watermarking
US6847980B1 (en) * 1999-07-03 2005-01-25 Ana B. Benitez Fundamental entity-relationship models for the generic audio visual data signal description
US20050076055A1 (en) * 2001-08-28 2005-04-07 Benoit Mory Automatic question formulation from a user selection in multimedia content
US6886013B1 (en) * 1997-09-11 2005-04-26 International Business Machines Corporation HTTP caching proxy to filter and control display of data in a web browser
US20060026588A1 (en) * 2004-06-08 2006-02-02 Daniel Illowsky System device and method for configuring and operating interoperable device having player and engine
US7010751B2 (en) * 2000-02-18 2006-03-07 University Of Maryland, College Park Methods for the electronic annotation, retrieval, and use of electronic images
US7093028B1 (en) * 1999-12-15 2006-08-15 Microsoft Corporation User and content aware object-based data stream transmission methods and arrangements
US20070033170A1 (en) * 2000-07-24 2007-02-08 Sanghoon Sull Method For Searching For Relevant Multimedia Content
US7185049B1 (en) * 1999-02-01 2007-02-27 At&T Corp. Multimedia integration description scheme, method and system for MPEG-7
US20070047816A1 (en) * 2005-08-23 2007-03-01 Jamey Graham User Interface for Mixed Media Reality
US20070078846A1 (en) * 2005-09-30 2007-04-05 Antonino Gulli Similarity detection and clustering of images
US20070087756A1 (en) * 2005-10-04 2007-04-19 Hoffberg Steven M Multifactorial optimization system and method
US7327885B2 (en) * 2003-06-30 2008-02-05 Mitsubishi Electric Research Laboratories, Inc. Method for detecting short term unusual events in videos
US20080055479A1 (en) * 2006-09-01 2008-03-06 Texas Instruments Incorporated Color Space Appearance Model Video Processor
US20080082426A1 (en) * 2005-05-09 2008-04-03 Gokturk Salih B System and method for enabling image recognition and searching of remote content on display
US20080097939A1 (en) * 1998-05-01 2008-04-24 Isabelle Guyon Data mining platform for bioinformatics and other knowledge discovery
US7496830B2 (en) * 1999-12-07 2009-02-24 Microsoft Corporation Computer user interface architecture that saves a user's non-linear navigation history and intelligently maintains that history
US20090055094A1 (en) * 2007-06-07 2009-02-26 Sony Corporation Navigation device and nearest point search method
US7519217B2 (en) * 2004-11-23 2009-04-14 Microsoft Corporation Method and system for generating a classifier using inter-sample relationships
US7653264B2 (en) * 2005-03-04 2010-01-26 The Regents Of The University Of Michigan Method of determining alignment of images in high dimensional feature space
US7676820B2 (en) * 2003-01-06 2010-03-09 Koninklijke Philips Electronics N.V. Method and apparatus for similar video content hopping
US20100082614A1 (en) * 2008-09-22 2010-04-01 Microsoft Corporation Bayesian video search reranking
US20110025710A1 (en) * 2008-04-10 2011-02-03 The Trustees Of Columbia University In The City Of New York Systems and methods for image archeology
US7884567B2 (en) * 2006-11-16 2011-02-08 Samsung Sdi Co., Ltd. Fuel cell system and method for controlling operation of the fuel cell system
US20110064136A1 (en) * 1997-05-16 2011-03-17 Shih-Fu Chang Methods and architecture for indexing and editing compressed video over the world wide web
US20110081892A1 (en) * 2005-08-23 2011-04-07 Ricoh Co., Ltd. System and methods for use of voice mail and email in a mixed media environment
US8135221B2 (en) * 2009-10-07 2012-03-13 Eastman Kodak Company Video concept classification using audio-visual atoms
US8145677B2 (en) * 2007-03-27 2012-03-27 Faleh Jassem Al-Shameri Automated generation of metadata for mining image and text data
US20120089552A1 (en) * 2008-12-22 2012-04-12 Shih-Fu Chang Rapid image annotation via brain state decoding and visual pattern mining
US20140064091A1 (en) * 2012-08-29 2014-03-06 International Business Machines Corporation Sliced routing table management with replication

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996008095A1 (en) * 1994-09-08 1996-03-14 Virtex Communications, Inc. Method and apparatus for electronic distribution of digital multi-media information
JP3609488B2 (en) 1995-05-17 2005-01-12 株式会社日立製作所 Information processing system
US5953506A (en) * 1996-12-17 1999-09-14 Adaptive Media Technologies Method and apparatus that provides a scalable media delivery system
US7145946B2 (en) * 2001-07-27 2006-12-05 Sony Corporation MPEG video drift reduction

Patent Citations (102)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4649380A (en) * 1983-06-15 1987-03-10 U. S. Philips Corporation Video display system comprising an index store for storing reduced versions of pictures to be displayed
US4649482A (en) * 1984-08-31 1987-03-10 Bio-Logic Systems Corp. Brain electrical activity topographical mapping
US6058186A (en) * 1990-04-23 2000-05-02 Canon Kabushiki Kaisha Information signal transmission system
US5208857A (en) * 1990-04-25 1993-05-04 Telediffusion De France Method and device for scrambling-unscrambling digital image data
US5204706A (en) * 1990-11-30 1993-04-20 Kabushiki Kaisha Toshiba Moving picture managing device
US5191645A (en) * 1991-02-28 1993-03-02 Sony Corporation Of America Digital signal processing system employing icon displays
US5724484A (en) * 1991-03-20 1998-03-03 Hitachi, Ltd. Data processing methods and apparatus for supporting analysis/judgement
US5623690A (en) * 1992-06-03 1997-04-22 Digital Equipment Corporation Audio/video storage and retrieval for multimedia workstations by interleaving audio and video data in data file
US5708805A (en) * 1992-10-09 1998-01-13 Matsushita Electric Industrial Co., Ltd. Image retrieving apparatus using natural language
US5751286A (en) * 1992-11-09 1998-05-12 International Business Machines Corporation Image query system and method
US5615112A (en) * 1993-01-29 1997-03-25 Arizona Board Of Regents Synthesized object-oriented entity-relationship (SOOER) model for coupled knowledge-base/database of image retrieval expert system (IRES)
US5630121A (en) * 1993-02-02 1997-05-13 International Business Machines Corporation Archiving and retrieving multimedia objects using structured indexes
US5408274A (en) * 1993-03-11 1995-04-18 The Regents Of The University Of California Method and apparatus for compositing compressed video data
US5585852A (en) * 1993-06-16 1996-12-17 Intel Corporation Processing video signals for scalable video playback using independently encoded component-plane bands
US5742283A (en) * 1993-09-27 1998-04-21 International Business Machines Corporation Hyperstories: organizing multimedia episodes in temporal and spatial displays
US5606655A (en) * 1994-03-31 1997-02-25 Siemens Corporate Research, Inc. Method for representing contents of a single video shot using frames
US5605655A (en) * 1994-04-11 1997-02-25 Mitsubishi Jukogyo Kabushiki Kaisha Gas-liquid contacting apparatus
US5488664A (en) * 1994-04-22 1996-01-30 Yeda Research And Development Co., Ltd. Method and apparatus for protecting visual information with printed cryptographic watermarks
US5617119A (en) * 1994-06-08 1997-04-01 Systems Research & Applications Corporation Protection of an electronically stored image in a first color space by the alteration of a digital component in a second color space
US5493677A (en) * 1994-06-08 1996-02-20 Systems Research & Applications Corporation Generation, archiving, and retrieval of digital images with evoked suggestion-set captions and natural language interface
US5613032A (en) * 1994-09-02 1997-03-18 Bell Communications Research, Inc. System and method for recording, playing back and searching multimedia events wherein video, audio and text can be searched and retrieved
US6047374A (en) * 1994-12-14 2000-04-04 Sony Corporation Method and apparatus for embedding authentication information within digital data
US6718047B2 (en) * 1995-05-08 2004-04-06 Digimarc Corporation Watermark embedder and reader
US5721815A (en) * 1995-06-07 1998-02-24 International Business Machines Corporation Media-on-demand communication system and method employing direct access storage device
US5713021A (en) * 1995-06-28 1998-01-27 Fujitsu Limited Multimedia data search system that searches for a portion of multimedia data using objects corresponding to the portion of multimedia data
US5758076A (en) * 1995-07-19 1998-05-26 International Business Machines Corporation Multimedia server system having rate adjustable data retrieval based on buffer capacity
US6356309B1 (en) * 1995-08-02 2002-03-12 Matsushita Electric Industrial Co., Ltd. Video coding device and video transmission system using the same, quantization control method and average throughput calculation method used therein
US5734893A (en) * 1995-09-28 1998-03-31 Ibm Corporation Progressive content-based retrieval of image and video with adaptive and iterative refinement
US5884298A (en) * 1996-03-29 1999-03-16 Cygnet Storage Solutions, Inc. Method for accessing and updating a library of optical discs
US5893095A (en) * 1996-03-29 1999-04-06 Virage, Inc. Similarity engine for content-based retrieval of images
US5870754A (en) * 1996-04-25 1999-02-09 Philips Electronics North America Corporation Video retrieval of MPEG compressed sequences using DC and motion signatures
US5887061A (en) * 1996-05-01 1999-03-23 Oki Electric Industry Co., Ltd. Compression coding device with scrambling function and expansion reproducing device with descrambling function
US6031914A (en) * 1996-08-30 2000-02-29 Regents Of The University Of Minnesota Method and apparatus for embedding data, including watermarks, in human perceptible images
US5873080A (en) * 1996-09-20 1999-02-16 International Business Machines Corporation Using multiple search engines to search multimedia data
US5734752A (en) * 1996-09-24 1998-03-31 Xerox Corporation Digital watermarking using stochastic screen patterns
US6526099B1 (en) * 1996-10-25 2003-02-25 Telefonaktiebolaget Lm Ericsson (Publ) Transcoder
US6172675B1 (en) * 1996-12-05 2001-01-09 Interval Research Corporation Indirect manipulation of data using temporally related data, with particular application to manipulation of audio or audiovisual data
US6208746B1 (en) * 1997-05-09 2001-03-27 Gte Service Corporation Biometric watermarks
US20110064136A1 (en) * 1997-05-16 2011-03-17 Shih-Fu Chang Methods and architecture for indexing and editing compressed video over the world wide web
US6222932B1 (en) * 1997-06-27 2001-04-24 International Business Machines Corporation Automatic adjustment of image watermark strength based on computed image texture
US6195458B1 (en) * 1997-07-29 2001-02-27 Eastman Kodak Company Method for content-based temporal segmentation of video
US6360234B2 (en) * 1997-08-14 2002-03-19 Virage, Inc. Video cataloger system with synchronized encoders
US6208735B1 (en) * 1997-09-10 2001-03-27 Nec Research Institute, Inc. Secure spread spectrum watermarking for multimedia data
US6886013B1 (en) * 1997-09-11 2005-04-26 International Business Machines Corporation HTTP caching proxy to filter and control display of data in a web browser
US6721733B2 (en) * 1997-10-27 2004-04-13 Massachusetts Institute Of Technology Information search and retrieval system
US6366314B1 (en) * 1997-12-17 2002-04-02 Telediffusion De France Method and system for measuring the quality of digital television signals
US6037984A (en) * 1997-12-24 2000-03-14 Sarnoff Corporation Method and apparatus for embedding a watermark into a digital image or image sequence
US20080097939A1 (en) * 1998-05-01 2008-04-24 Isabelle Guyon Data mining platform for bioinformatics and other knowledge discovery
US6178416B1 (en) * 1998-06-15 2001-01-23 James U. Parker Method and apparatus for knowledgebase searching
US6041079A (en) * 1998-06-30 2000-03-21 Thomson Consumer Electronics, Inc, Field/frame conversion of DCT domain mixed field/frame mode macroblocks using 1-dimensional DCT/IDCT
US7184959B2 (en) * 1998-08-13 2007-02-27 At&T Corp. System and method for automated multimedia content indexing and retrieval
US6714909B1 (en) * 1998-08-13 2004-03-30 At&T Corp. System and method for automated multimedia content indexing and retrieval
US6716175B2 (en) * 1998-08-25 2004-04-06 University Of Florida Autonomous boundary detection system for echocardiographic images
US6708055B2 (en) * 1998-08-25 2004-03-16 University Of Florida Method for automated analysis of apical four-chamber images of the heart
US6167084A (en) * 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US6185329B1 (en) * 1998-10-13 2001-02-06 Hewlett-Packard Company Automatic caption text detection and processing for digital images
US6549911B2 (en) * 1998-11-02 2003-04-15 Survivors Of The Shoah Visual History Foundation Method and apparatus for cataloguing multimedia data
US6678389B1 (en) * 1998-12-29 2004-01-13 Kent Ridge Digital Labs Method and apparatus for embedding digital information in digital multimedia data
US6532541B1 (en) * 1999-01-22 2003-03-11 The Trustees Of Columbia University In The City Of New York Method and apparatus for image authentication
US6366701B1 (en) * 1999-01-28 2002-04-02 Sarnoff Corporation Apparatus and method for describing the motion parameters of an object in an image sequence
US6223183B1 (en) * 1999-01-29 2001-04-24 International Business Machines Corporation System and method for describing views in space, time, frequency, and resolution
US7185049B1 (en) * 1999-02-01 2007-02-27 At&T Corp. Multimedia integration description scheme, method and system for MPEG-7
US6556695B1 (en) * 1999-02-05 2003-04-29 Mayo Foundation For Medical Education And Research Method for producing high resolution real-time images, of structure and function during medical procedures
US6556958B1 (en) * 1999-04-23 2003-04-29 Microsoft Corporation Fast clustering with sparse data
US6847980B1 (en) * 1999-07-03 2005-01-25 Ana B. Benitez Fundamental entity-relationship models for the generic audio visual data signal description
US6546135B1 (en) * 1999-08-30 2003-04-08 Mitsubishi Electric Research Laboratories, Inc Method for representing and comparing multimedia content
US6339450B1 (en) * 1999-09-21 2002-01-15 At&T Corp Error resilient transcoding for video over wireless channels
US6725372B1 (en) * 1999-12-02 2004-04-20 Verizon Laboratories Inc. Digital watermarking
US7496830B2 (en) * 1999-12-07 2009-02-24 Microsoft Corporation Computer user interface architecture that saves a user's non-linear navigation history and intelligently maintains that history
US7093028B1 (en) * 1999-12-15 2006-08-15 Microsoft Corporation User and content aware object-based data stream transmission methods and arrangements
US7010751B2 (en) * 2000-02-18 2006-03-07 University Of Maryland, College Park Methods for the electronic annotation, retrieval, and use of electronic images
US6701309B1 (en) * 2000-04-21 2004-03-02 Lycos, Inc. Method and system for collecting related queries
US20110093492A1 (en) * 2000-07-24 2011-04-21 Sanghoon Sull System and Method for Indexing, Searching, Identifying, and Editing Multimedia Files
US20070044010A1 (en) * 2000-07-24 2007-02-22 Sanghoon Sull System and method for indexing, searching, identifying, and editing multimedia files
US20070033170A1 (en) * 2000-07-24 2007-02-08 Sanghoon Sull Method For Searching For Relevant Multimedia Content
US20070038612A1 (en) * 2000-07-24 2007-02-15 Sanghoon Sull System and method for indexing, searching, identifying, and editing multimedia files
US20020021828A1 (en) * 2000-08-01 2002-02-21 Arthur Papier System and method to aid diagnoses using cross-referenced knowledge and image databases
US6683966B1 (en) * 2000-08-24 2004-01-27 Digimarc Corporation Watermarking recursive hashes into frequency domain regions
US20030013951A1 (en) * 2000-09-21 2003-01-16 Dan Stefanescu Database organization and searching
US20030046018A1 (en) * 2001-04-20 2003-03-06 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandeten Forschung E.V Method for segmentation and identification of nonstationary time series
US20050076055A1 (en) * 2001-08-28 2005-04-07 Benoit Mory Automatic question formulation from a user selection in multimedia content
US6700935B2 (en) * 2002-02-08 2004-03-02 Sony Electronics, Inc. Stream based bitrate transcoder for MPEG coded video
US20040057081A1 (en) * 2002-09-20 2004-03-25 Fuji Xerox Co., Ltd. Image processing method, manipulation detection method, image processing device, manipulation detection device, image processing program, manipulation detection program, and image formation medium
US7676820B2 (en) * 2003-01-06 2010-03-09 Koninklijke Philips Electronics N.V. Method and apparatus for similar video content hopping
US7327885B2 (en) * 2003-06-30 2008-02-05 Mitsubishi Electric Research Laboratories, Inc. Method for detecting short term unusual events in videos
US20060026588A1 (en) * 2004-06-08 2006-02-02 Daniel Illowsky System device and method for configuring and operating interoperable device having player and engine
US7519217B2 (en) * 2004-11-23 2009-04-14 Microsoft Corporation Method and system for generating a classifier using inter-sample relationships
US7653264B2 (en) * 2005-03-04 2010-01-26 The Regents Of The University Of Michigan Method of determining alignment of images in high dimensional feature space
US20080082426A1 (en) * 2005-05-09 2008-04-03 Gokturk Salih B System and method for enabling image recognition and searching of remote content on display
US20110081892A1 (en) * 2005-08-23 2011-04-07 Ricoh Co., Ltd. System and methods for use of voice mail and email in a mixed media environment
US20070047816A1 (en) * 2005-08-23 2007-03-01 Jamey Graham User Interface for Mixed Media Reality
US20070078846A1 (en) * 2005-09-30 2007-04-05 Antonino Gulli Similarity detection and clustering of images
US20070087756A1 (en) * 2005-10-04 2007-04-19 Hoffberg Steven M Multifactorial optimization system and method
US20080055479A1 (en) * 2006-09-01 2008-03-06 Texas Instruments Incorporated Color Space Appearance Model Video Processor
US7884567B2 (en) * 2006-11-16 2011-02-08 Samsung Sdi Co., Ltd. Fuel cell system and method for controlling operation of the fuel cell system
US8145677B2 (en) * 2007-03-27 2012-03-27 Faleh Jassem Al-Shameri Automated generation of metadata for mining image and text data
US20090055094A1 (en) * 2007-06-07 2009-02-26 Sony Corporation Navigation device and nearest point search method
US20110025710A1 (en) * 2008-04-10 2011-02-03 The Trustees Of Columbia University In The City Of New York Systems and methods for image archeology
US20100082614A1 (en) * 2008-09-22 2010-04-01 Microsoft Corporation Bayesian video search reranking
US20120089552A1 (en) * 2008-12-22 2012-04-12 Shih-Fu Chang Rapid image annotation via brain state decoding and visual pattern mining
US8135221B2 (en) * 2009-10-07 2012-03-13 Eastman Kodak Company Video concept classification using audio-visual atoms
US20140064091A1 (en) * 2012-08-29 2014-03-06 International Business Machines Corporation Sliced routing table management with replication

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110064136A1 (en) * 1997-05-16 2011-03-17 Shih-Fu Chang Methods and architecture for indexing and editing compressed video over the world wide web
US9330722B2 (en) 1997-05-16 2016-05-03 The Trustees Of Columbia University In The City Of New York Methods and architecture for indexing and editing compressed video over the world wide web
US8370869B2 (en) 1998-11-06 2013-02-05 The Trustees Of Columbia University In The City Of New York Video description system and method
US8488682B2 (en) 2001-12-06 2013-07-16 The Trustees Of Columbia University In The City Of New York System and method for extracting text captions from video and generating video summaries
US9060175B2 (en) 2005-03-04 2015-06-16 The Trustees Of Columbia University In The City Of New York System and method for motion estimation and mode decision for low-complexity H.264 decoder
US8849058B2 (en) 2008-04-10 2014-09-30 The Trustees Of Columbia University In The City Of New York Systems and methods for image archaeology
US8364673B2 (en) 2008-06-17 2013-01-29 The Trustees Of Columbia University In The City Of New York System and method for dynamically and interactively searching media data
US8793749B2 (en) * 2008-08-25 2014-07-29 Broadcom Corporation Source frame adaptation and matching optimally to suit a recipient video device
US20100050225A1 (en) * 2008-08-25 2010-02-25 Broadcom Corporation Source frame adaptation and matching optimally to suit a recipient video device
US9665824B2 (en) 2008-12-22 2017-05-30 The Trustees Of Columbia University In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US8671069B2 (en) 2008-12-22 2014-03-11 The Trustees Of Columbia University, In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US9071841B2 (en) 2011-05-17 2015-06-30 Microsoft Technology Licensing, Llc Video transcoding with dynamically modifiable spatial resolution
US9578333B2 (en) * 2013-03-15 2017-02-21 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
US20170078678A1 (en) * 2013-03-15 2017-03-16 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
US20140269938A1 (en) * 2013-03-15 2014-09-18 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
US9787999B2 (en) * 2013-03-15 2017-10-10 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
US9747013B2 (en) 2014-01-06 2017-08-29 Dropbox, Inc. Predictive caching and fetch priority
US9766791B2 (en) * 2014-01-06 2017-09-19 Dropbox, Inc. Predictive caching and fetch priority
US9952753B2 (en) 2014-01-06 2018-04-24 Dropbox, Inc. Predictive caching and fetch priority

Also Published As

Publication number Publication date
AU2003231102A1 (en) 2003-11-10
AU2003231102A8 (en) 2003-11-10
WO2003091850A2 (en) 2003-11-06
US8218617B2 (en) 2012-07-10
WO2003091850A3 (en) 2005-03-24
KR20050007348A (en) 2005-01-17
EP1532812A2 (en) 2005-05-25
US20090290635A1 (en) 2009-11-26
EP1532812A4 (en) 2007-10-10
JP2005525011A (en) 2005-08-18

Similar Documents

Publication Publication Date Title
US8218617B2 (en) Method and system for optimal video transcoding based on utility function descriptors
JP4650868B2 (en) Compressed video transcoding method
US6925120B2 (en) Transcoder for scalable multi-layer constant quality video bitstreams
US6490320B1 (en) Adaptable bitstream video delivery system
JP4601889B2 (en) Apparatus and method for converting a compressed bitstream
Kim et al. Content-adaptive utility-based video adaptation
US6542546B1 (en) Adaptable compressed bitstream transcoder
US9749713B2 (en) Budget encoding
US9774848B2 (en) Efficient compression and transport of video over a network
EP1289301B1 (en) Method and device for encoding image
Seeling et al. Video traces for network performance evaluation: a comprehensive overview and guide on video traces and their utilization in networking research
KR100848310B1 (en) Method and apparatus for scalable video adaptation using adaptation operators for scalable video
JP2005323353A (en) High-fidelity transcoding
US20110246673A1 (en) Method and System for Optimizing the Content and Transfer of Media Files
Kim et al. An optimal framework of video adaptation and its application to rate adaptation transcoding
Auli-Llinas et al. Enhanced JPEG2000 quality scalability through block-wise layer truncation
CN100366077C (en) Method and system for optimal video transcoding based on utility function descriptors
NAKAJIMA et al. MPEG bit rate and format conversions for heterogeneous network/storage applications
Alsrehin Toward a general parametric model for assessing the impact of video transcoding on objective video quality
Mukherjee et al. Terminal and Network Quality of Service
Kang et al. MPEG-21 DIA-based video adaptation framework and its application to rate adaptation
Suchomski et al. Format independence of audio and video in multimedia database systems
Notebaert et al. Digital recording of performing arts: formats and conversion
Zhang et al. Intermedia: system and application for video adaptation

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION