US20020178149A1 - Content -based similarity retrieval system for image data - Google Patents

Content -based similarity retrieval system for image data Download PDF

Info

Publication number
US20020178149A1
US20020178149A1 US09/833,818 US83381801A US2002178149A1 US 20020178149 A1 US20020178149 A1 US 20020178149A1 US 83381801 A US83381801 A US 83381801A US 2002178149 A1 US2002178149 A1 US 2002178149A1
Authority
US
United States
Prior art keywords
right arrow
arrow over
content
retrieval method
salient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US09/833,818
Other versions
US6834288B2 (en
Inventor
Jiann-Jone Chen
Chen-Chin Chiang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial Technology Research Institute ITRI
Original Assignee
Industrial Technology Research Institute ITRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial Technology Research Institute ITRI filed Critical Industrial Technology Research Institute ITRI
Priority to US09/833,818 priority Critical patent/US6834288B2/en
Assigned to INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE reassignment INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHEN, JIANN-JONE, CHIANG, CHEN-CHIN
Priority to TW090123293A priority patent/TW511010B/en
Publication of US20020178149A1 publication Critical patent/US20020178149A1/en
Application granted granted Critical
Publication of US6834288B2 publication Critical patent/US6834288B2/en
Adjusted expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5854Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99932Access augmentation or optimizing
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99944Object-oriented database structure
    • Y10S707/99945Object-oriented database structure processing
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99948Application of database or data structure, e.g. distributed, multimedia, or image

Definitions

  • the present invention is generally directed to image retrieval systems, and more particularly, to content-based retrieval of similar-looking images by employing multi-instance or relevance feedback to query images.
  • MPEG-7 uses contour-based and region-based descriptors. Although describing shape contour by Fourier descriptors (FDs) can provide size, rotation and transition invariants for indexing, FDs are sensitive to noises and are more suitable for describing closed contour of shapes. For region-based descriptors, zernike and pseudo-zernike moments (ZMs and PZMs) are efficient features for retrieving similar shapes. Specific features such as edge orientation, aspect ratio, or complexity can be extracted for different databases and applications. Statistics, such as histogram or probability distribution model for the above-described features, are computed and considered as matching criteria for similarity measurement.
  • FDs Fourier descriptors
  • ZMs and PZMs pseudo-zernike moments
  • Shape descriptors have also been used to extract shape boundaries which are grouped into families on the basis of perceptual similarity. Visually salient feature is determined using probabilistic distribution model of trademarks in database and then trademarks with similar shape were retrieved according to this salient feature. A Multi-resolution description can be obtained by computing the mean and variance of each wavelet sub-band as to provide a concise description of the shape's texture and shape. Shape features as discussed above can also be brought up according to specific databases and user requirements. Although efficient in retrieving similar shapes for one application, one set of universal descriptors cannot satisfy all specific requirements. Indeed, one set of descriptors may perform well for one database but not the other, and each user may need specific combination of features for their specific retrieval target.
  • an object of the present invention is to provide a content-based retrieval method and apparatus which retrieves statistically salient common features among sample query images having different feature sets.
  • UQM universal query mechanism
  • each set of sample query images is constructed by finding similar images shapes in the database.
  • the resulting sample query images are statistically similar to query input, i.e. relative instead of absolute similarity.
  • a probability distribution model for the feature vectors is used to dynamically adjust weights such that most common ones among sample query images dominates feedback query.
  • new feature sets are devised, they could be acquired by the query system.
  • the query unit searches from all feature sets such that the statistically common features become the new query vector. Accordingly, the UQM accommodates new feature sets easily and adjusts weights for various features dynamically according to a user's query and statistics of the database.
  • each user has his definition for shape similarity and no one universal set of shape descriptors could satisfy all specific requirements.
  • the indexing system provides relevance feedback to learn what user's intention is and generating a new feature vector for next query.
  • MMV mean of feature vectors
  • the UQM of the present invention finds statistically salient common features among sample query images with different feature sets.
  • FIG. 1 is a schematic diagram showing a content-based retrieval method according to a preferred embodiment of the present invention
  • FIG. 2 is a schematic diagram showing a UQM according to the preferred embodiment of the present invention.
  • FIG. 3 is a schematic chart showing the measurement of feature saliency
  • FIGS. 4 A-C and 5 A-B show results of the content-based retrieval method according to a preferred embodiment of the present invention
  • FIG. 1 One embodiment of content-based similarity retrieval process for database images is shown in FIG. 1.
  • first phase common features of multimedia content in a multimedia database are extracted by feature extraction units 12 and 14 and stored in feature databases 16 and 18 .
  • second phase i.e., similarity retrieval
  • a user selects an initial query image either through an on-line editing interface 20 or from one or more sample images stored in the database 10 . If a new image is created from the on-line interface 20 , features in the new image must be extracted by both feature extraction units 12 and 14 before searching in the feature databases 16 and 18 .
  • ⁇ right arrow over (F) ⁇ s ( ⁇ right arrow over ( ⁇ ) ⁇ 1 , ⁇ right arrow over ( ⁇ ) ⁇ 2 , . . . , ⁇ right arrow over ( ⁇ ) ⁇ n ),
  • ⁇ right arrow over (N) ⁇ t ⁇ ij
  • the weight, w j , of j-th feature, ⁇ j , for the next query are adjusted according to the following relation: ⁇ j ⁇ F D ⁇ ( m j P , ⁇ j P , m j N , ⁇ j N ) F C ⁇ ( m j P , ⁇ j P ) ⁇ F C ⁇ ( m j N , ⁇ j N ) ( 6 )
  • ⁇ i c and ⁇ i c are standard deviation of ith features from relevant and all database samples, respectively.
  • Results of the content-based retrieval method according to a preferred embodiment of the present invention are shown in FIGS. 4 and 5.
  • Taiwan Bell Trademark Company Thirty thousand registered Taiwan trademarks were collected from Taiwan Bell Trademark Company. They contain text pattern, animal, regular geometrical shape and shapes with text et al. These shape patterns were segmented manually from scanned documents in JPEG format. Each one is pre-processed by locating MBC of shapes before further feature extraction. For efficient feature extraction, magnitude of ZM and PZM were computed for each sample by the lookup-table method with order, n, up to ten. The numbers of ZM and PZM are 36 and 66 , respectively.
  • FIG. 4A is the retrieval results by one query image 42 .
  • Image 43 is selected as the relevance feedback for the next retrieval.
  • Mean of feature vectors of the two instances (images 42 and 43 ) is first used as the input for the second query.
  • FIG. 4B shows the second query results and two visually similar shapes, i.e., images 44 and 45 , other than those in FIG. 4A, are retrieved if features were not weighted by the UQM method.
  • FIG. 4C three more similar shapes in images 46 , 47 and 48 are retrieved when the proposed UQM are used.
  • FIGS. 5A and 5B Another retrieval results of query by three-instances are demonstrated in FIGS. 5A and 5B.
  • the proposed UQM searches out two more visually similar images 50 and 51 that are not presented in FIG. 5B in which features are not weighted by UQM.

Abstract

An image retrieval system for retrieving image similarities from a database is proposed. The image retrieval system uses a universal query mechanism (UQM) to locate statistically silent common features among sample query images from different feature sets. The UQM also adjusts the weight factor for each feature to meet a user's query demand.

Description

    BACKGROUND OF THE INVENTION
  • 1. Field of the Invention [0001]
  • The present invention is generally directed to image retrieval systems, and more particularly, to content-based retrieval of similar-looking images by employing multi-instance or relevance feedback to query images. [0002]
  • 2. Description of the Related Art [0003]
  • The importance of content-based retrieval techniques for multimedia is assured in view of the widely used and distributed international coding standards, such as JPEG and MPEG, on the Internet. A particular multimedia content description interface, MPEG-7, is currently used to provide normal numerical descriptors for database search engine as matching criteria. For 2D shapes, MPEG-7 uses contour-based and region-based descriptors. Although describing shape contour by Fourier descriptors (FDs) can provide size, rotation and transition invariants for indexing, FDs are sensitive to noises and are more suitable for describing closed contour of shapes. For region-based descriptors, zernike and pseudo-zernike moments (ZMs and PZMs) are efficient features for retrieving similar shapes. Specific features such as edge orientation, aspect ratio, or complexity can be extracted for different databases and applications. Statistics, such as histogram or probability distribution model for the above-described features, are computed and considered as matching criteria for similarity measurement. [0004]
  • Shape descriptors have also been used to extract shape boundaries which are grouped into families on the basis of perceptual similarity. Visually salient feature is determined using probabilistic distribution model of trademarks in database and then trademarks with similar shape were retrieved according to this salient feature. A Multi-resolution description can be obtained by computing the mean and variance of each wavelet sub-band as to provide a concise description of the shape's texture and shape. Shape features as discussed above can also be brought up according to specific databases and user requirements. Although efficient in retrieving similar shapes for one application, one set of universal descriptors cannot satisfy all specific requirements. Indeed, one set of descriptors may perform well for one database but not the other, and each user may need specific combination of features for their specific retrieval target. [0005]
  • SUMMARY OF THE INVENTION
  • Accordingly, an object of the present invention is to provide a content-based retrieval method and apparatus which retrieves statistically salient common features among sample query images having different feature sets. [0006]
  • It is another object of the present invention to update sample query images by multi-instance or relevance feedback. [0007]
  • It is still another object of the present invention to provide a universal query mechanism (UQM) or process that is plural and flexible in selecting proper features as to meet a user's intent or requirement. [0008]
  • These and other objects of the present invention are achieved by providing a content-based retrieval method and apparatus which finds the most common features among each set of sample query images from multi-instance or relevance feedback. In particular, each set of sample query images is constructed by finding similar images shapes in the database. The resulting sample query images are statistically similar to query input, i.e. relative instead of absolute similarity. A probability distribution model for the feature vectors is used to dynamically adjust weights such that most common ones among sample query images dominates feedback query. Whenever new feature sets are devised, they could be acquired by the query system. The query unit then searches from all feature sets such that the statistically common features become the new query vector. Accordingly, the UQM accommodates new feature sets easily and adjusts weights for various features dynamically according to a user's query and statistics of the database. [0009]
  • For similarity retrieval, each user has his definition for shape similarity and no one universal set of shape descriptors could satisfy all specific requirements. Usually, the indexing system provides relevance feedback to learn what user's intention is and generating a new feature vector for next query. Instead of using mean of feature vectors (MFV) from sample query images as the new query vector, the UQM of the present invention finds statistically salient common features among sample query images with different feature sets. [0010]
  • Related aspects and advantages of the invention will become apparent and more readily appreciated from the following detailed description of the invention, taken in conjunction with the accompanying drawings.[0011]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a schematic diagram showing a content-based retrieval method according to a preferred embodiment of the present invention; [0012]
  • FIG. 2 is a schematic diagram showing a UQM according to the preferred embodiment of the present invention; [0013]
  • FIG. 3 is a schematic chart showing the measurement of feature saliency; and [0014]
  • FIGS. [0015] 4A-C and 5A-B show results of the content-based retrieval method according to a preferred embodiment of the present invention
  • DETAILED DESCRIPTION OF THE INVENTION
  • The present invention will now be described by way of preferred embodiments with references to the accompanying drawings. Like numerals refer to corresponding parts of various drawings. [0016]
  • One embodiment of content-based similarity retrieval process for database images is shown in FIG. 1. During the first phase, common features of multimedia content in a multimedia database are extracted by [0017] feature extraction units 12 and 14 and stored in feature databases 16 and 18. In the second phase (i.e., similarity retrieval), a user selects an initial query image either through an on-line editing interface 20 or from one or more sample images stored in the database 10. If a new image is created from the on-line interface 20, features in the new image must be extracted by both feature extraction units 12 and 14 before searching in the feature databases 16 and 18. If one of the sample images stored in the database 10 were used as the initial query image, then only an identification code will be sent to the feature databases 16 and 18, which sends associated feature of the initial query image to a UQM 22 such that both query units 24 and 26 in the UQM can generate a representative feature vector for retrieving a set of sample query images.
  • Since more than one feature set may be created and new features can be plugged into the UQM based on a user's requirement, statistically common features are retrieved among sample query images through multi-instance or by relevance feedback. The weight factor for each feature is adjusted by UQM to meet a user's query requirement. The query results are the same when non-related features are involved or excluded in the retrieval process. [0018]
  • Assume n feature sets are created, then the feature vector for sample s can be represented as [0019]
  • {right arrow over (F)}s=({right arrow over (ƒ)}1, {right arrow over (ƒ)}2, . . . , {right arrow over (ƒ)}n),
  • where the i-th feature set is {right arrow over (ƒ)}[0020] i=(ƒi1, ƒi2, . . . ƒiN 1 ). Let {right arrow over (N)}t={ij|i=1, . . . , n, j=1, . . . , Ni}, and the feature vector can be represented as {right arrow over (F)}s={ƒk|k∈{right arrow over (N)}t}. Let the probability distribution function of feature ƒi be p(ƒi), then the probability density function P(ƒ) would be: P ( f ) = 0 f p ( f i ) f i ( 1 )
    Figure US20020178149A1-20021128-M00001
  • Through multi-instance or relevance feedback, the UQM as shown in FIG. 2 seeks to find common features among sample query images. Weight factors are adjusted such that statistically common features dominate in the representative feature vector for next sample query images. Let there be q sample query images, either for multi-instance or relevance feedback, and the q feature vectors available are {right arrow over (F)}[0021] s where s=1, . . . , q. Mean and variance of these vectors could be represented as: { m j } N t = E [ F s ] = { E [ f j ] } j N t = { E { f ij i = 1 , , q } } j N t and ( 2 ) { σ j 2 } N t = { E [ ( f j - m j ) 2 ] } j N t = { E { f ij 2 i = 1 , , q } - m j 2 } j N t . ( 3 )
    Figure US20020178149A1-20021128-M00002
  • For similarity measurement in statistics as shown in FIG. 3, positive and negative feedback samples are provided and feature compactness F[0022] C(•) and feature diversity FD(•) in probability are defined as:
  • F C(m,σ)=P(m+σ)−P(m−σ); and  (4)
  • F D(m 11 ,m 22)=P(m 2−σ2)−P(m 11).  (5)
  • F[0023] D(•)=1 is set so that if the number of negative feedback samples is zero. The weight, wj, of j-th feature,ƒj, for the next query are adjusted according to the following relation: ω j F D ( m j P , σ j P , m j N , σ j N ) F C ( m j P , σ j P ) · F C ( m j N , σ j N ) ( 6 )
    Figure US20020178149A1-20021128-M00003
  • For one feature, say j-th feature, which is statistically similar among query sample images, it would yield a smaller F[0024] C(•), and this feature would get high credit because of larger wj is assigned. Whenever new feature sets are created, statistically common features among all feature sets would dominate the query process no matter how many feature sets are specified. In short, feature sets could be integrated or excluded from the query according to a user's requirement without affecting query results, i.e., only statistically similar features dominate the query.
  • For applications that provide positive and negative samples, the query unit intends to enhance a positive feature while excluding the negative ones. Since F[0025] D(mj Pj P,mj Nj N) evaluates the depletion region between features to be discriminated, features to be excluded needs not be statistically common hence the following is set: FC(mj Nj N)=1. In addition, to make common features dominate and to exclude unrelated features, the following is set: Wj=1 when the ratio in equation (6) is larger than a predefined threshold and 0 for others.
  • In short, the control steps of the invention are summarized as follows: [0026]
  • (1) the feature vector (ƒ[0027] 12, . . . , ƒN) is extracted for each sample image in the database;
  • (2) the probability distribution function is computed and stored for each feature either by {(m[0028] ii)}i=1, . . . , N or {p(ƒij)|j=1, . . . , M}i=1, . . . , N;
  • (3) for each query, compute mean (m) and standard deviation (σ) for each feature among query image; [0029]
  • (4) find the feature commonality for each feature i.e., F[0030] C(m,σ)=P(m+σ)−P(m−σ) and generate new weight by the following condition: iƒ(Fc<PT) ωi=1 else ωi=0; and
  • (5) perform new retrieval by new feature vector (m[0031] 1,m2, . . . , mN) and new weight vector (ω12, . . . , ωN).
  • Statistical characteristics of features in the database are exploited for similarity retrieval. In cases that most feature values of the same family diverse while they are similar in probability space, the retrieval results may not converge from human perception. It happens when there are very few similar images in the database or too many noises in the images. For this, w[0032] i=0 is set to keep the common feature dominating the query process when σ i c > 1 c · σ i s ,
    Figure US20020178149A1-20021128-M00004
  • where σ[0033] i c and σi c are standard deviation of ith features from relevant and all database samples, respectively.
  • Results of the content-based retrieval method according to a preferred embodiment of the present invention are shown in FIGS. 4 and 5. Thirty thousand registered Taiwan trademarks were collected from Taiwan Bell Trademark Company. They contain text pattern, animal, regular geometrical shape and shapes with text et al. These shape patterns were segmented manually from scanned documents in JPEG format. Each one is pre-processed by locating MBC of shapes before further feature extraction. For efficient feature extraction, magnitude of ZM and PZM were computed for each sample by the lookup-table method with order, n, up to ten. The numbers of ZM and PZM are [0034] 36 and 66, respectively.
  • FIG. 4A is the retrieval results by one [0035] query image 42. Image 43 is selected as the relevance feedback for the next retrieval. Mean of feature vectors of the two instances (images 42 and 43) is first used as the input for the second query. FIG. 4B shows the second query results and two visually similar shapes, i.e., images 44 and 45, other than those in FIG. 4A, are retrieved if features were not weighted by the UQM method. In FIG. 4C, three more similar shapes in images 46, 47 and 48 are retrieved when the proposed UQM are used.
  • Another retrieval results of query by three-instances are demonstrated in FIGS. 5A and 5B. In FIG. 5A, the proposed UQM searches out two more visually [0036] similar images 50 and 51 that are not presented in FIG. 5B in which features are not weighted by UQM.
  • The invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Scope on the invention is thus indicated by the claims rather than by the forgoing description, and all changes which come within the meanings and range of equivalency of the claims are therefore intended to be embraced herein. [0037]

Claims (20)

What is claimed is:
1. A content-based retrieval method for retrieving one or more images from a multimedia database, the retrieval method comprising the steps of:
constructing a sample query image;
extracting a salient and common feature vector from said sample query image;
generating an adjusted weight factor based on said salient and common feature vector;
generating a new feature vector based on said adjusted weight factor; and
generating a new query image based on said new feature vector.
2. The content-based retrieval method of claim 1, wherein said step of extracting salient and common feature vector comprises:
extracting a representative feature vector for each of said sample query images; and
computing and storing a probability distribution function, a mean value and a variance value for said representative feature vector.
3. The image retrieval method of claim 2 wherein said mean value and said variance value are respectively represented by:
{m j}{right arrow over (N)} t =E[{right arrow over (F)} s ]={E[f j]}jε{right arrow over (N)} t ={E{f ij |i=1, . . . ,q}} jεN{right arrow over (N)} t ; {σj 2{{right arrow over (N)}t ={E[(f j −m j)2]}jε{right arrow over (N)} t ={E{f 1j 2 |i=1, . . . , q}−m j 2}jε{right arrow over (N)} i.
4. The content-based image retrieval method of claim 2, wherein said probability distribution function is represented by:
{(m ii)}i=1, . . . , N or {pij)|j=1, . . . , M} i=1, . . . , N.
5. The content-based image retrieval method of claim 2, wherein said salient and common feature vector is measured by:
F C(m,σ)=P(m+σ)−P(m−σ).
6. The content-based image retrieval method of claim 1, wherein said adjusted weight factor is generated according to:
iƒ(F C <P Ti=1 else ωi=0.
7. A content-based retrieval method for retrieving one or more images from a multimedia database, the retrieval method comprising the steps of:
inputting one or more initial query images by a user;
extracting feature vectors for said initial query images based on one or more requirements of said user;
extracting salient and common features based on said feature vectors;
generating adjusted weight factors based on said salient and common features;
generating new feature vectors based on said adjusted weight factors; and
generating new query images based on said new feature vectors.
8. The content-based retrieval method of claim 7, wherein said extracted feature vectors are stored in one or more feature databases.
9. The content-based retrieval method of claim 7, wherein said step of extracting said salient and common feature vectors comprises:
extracting a representative feature vector for each of said sample query images; and
computing and storing a probability distribution function, a mean value and a variance for said representative feature vector.
10. The content-based image retrieval method of claim 9 wherein said mean value and said variance value are respectively represented by:
{m j}{right arrow over (N)} t =E[{right arrow over (F)} s ]={E[f j]}jε{right arrow over (N)} t ={E{f ij |i=1, . . . ,q}} jεN{right arrow over (N)} t ; {σj 2{{right arrow over (N)}t ={E[(f j −m j)2]}jε{right arrow over (N)} t ={E{f 1j 2 |i=1, . . . , q}−m j 2}jε{right arrow over (N)} i .
11. The content-based image retrieval method of claim 9, wherein said probability distribution function is represented by:
{(m ii)}i=1, . . . , N or {pij)|j=1, . . . , M} i=1, . . . , N.
12. The content-based image retrieval method of claim 9, wherein said salient and common feature vector is measured by:
F C(m,σ)=P(m+σ)−P(m−σ).
13. The content-based image retrieval method of claim 7, wherein said adjusted weight factor is generated according to:
iƒ(Fc <P Ti=1 else ωi=0.
14. A content-based retrieval method for retrieving one or more images from a multimedia database, the retrieval method comprising the steps of:
selecting one or more initial query images from said database;
retrieving corresponding feature vectors for said initial query images;
extracting salient and common features based on said feature vectors;
generating new weighting factors based on said salient and common features;
generating new feature vectors based on said new weighting factors; and
generating new query images based on said new feature vectors.
15. The content-based retrieval method of claim 14, wherein said step of extracting salient and common feature vectors comprises:
extracting a representative feature vector for each of said sample query images; and
computing and storing a probability distribution function, a mean value and a variance value for said representative feature vector.
16. The image retrieval method of claim 15, wherein said mean value and said variance value are respectively represented by:
{m j}{right arrow over (N)} t =E[{right arrow over (F)} s ]={E[f j]}jε{right arrow over (N)} t ={E{f ij |i=1, . . . ,q}} jεN{right arrow over (N)} t ; {σj 2{{right arrow over (N)}t ={E[(f j −m j)2]}jε{right arrow over (N)} t ={E{f 1j 2 |i=1, . . . , q}−m j 2}jε{right arrow over (N)} i.
17. The content-based image retrieval method of claim 15, wherein said probability distribution function is represented by:
{(m ii)}i=1, . . . , N or {pij)|j=1, . . . , M} i=1, . . . , N.
18. The content-based image retrieval method of claim 15, wherein said salient and common feature vector is measured by:
F C(m,σ)=P(m+σ)−P(m−σ).
19. The content-based image retrieval method of claim 15, wherein said adjusted weight factor is generated according to:
iƒ(F C<PTi=1 else ωi=0.
20. A content-based retrieval apparatus for retrieving one or more images from a multimedia database, comprising:
a multimedia database for storing sample query images;
one or more feature extraction unit for extracting salient and common feature vectors from said sample query images;
one or more feature databases for storing said salient and common feature vectors; and
one or more universal query units for generating adjusted weight factors based on said salient and common feature vectors, generating new feature vectors based on said adjusted weight factors, and generating new query images based on said new feature vectors.
US09/833,818 2001-04-13 2001-04-13 Content-based similarity retrieval system for image data Expired - Lifetime US6834288B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US09/833,818 US6834288B2 (en) 2001-04-13 2001-04-13 Content-based similarity retrieval system for image data
TW090123293A TW511010B (en) 2001-04-13 2001-09-21 A database search system and method for resemblance comparison of image data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09/833,818 US6834288B2 (en) 2001-04-13 2001-04-13 Content-based similarity retrieval system for image data

Publications (2)

Publication Number Publication Date
US20020178149A1 true US20020178149A1 (en) 2002-11-28
US6834288B2 US6834288B2 (en) 2004-12-21

Family

ID=25265349

Family Applications (1)

Application Number Title Priority Date Filing Date
US09/833,818 Expired - Lifetime US6834288B2 (en) 2001-04-13 2001-04-13 Content-based similarity retrieval system for image data

Country Status (2)

Country Link
US (1) US6834288B2 (en)
TW (1) TW511010B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030179213A1 (en) * 2002-03-18 2003-09-25 Jianfeng Liu Method for automatic retrieval of similar patterns in image databases
US20040258311A1 (en) * 2001-10-11 2004-12-23 Siemens Aktiengesellschaft Method for generating geometric models for optical partial recognition
GB2415562A (en) * 2004-06-23 2005-12-28 Hewlett Packard Development Co Generating saliency paths for viewing images
US20060193536A1 (en) * 2005-01-31 2006-08-31 Maurizio Pilu Image processing method and apparatus
WO2007137352A1 (en) * 2006-05-29 2007-12-06 University Of Wollongong Content based image retrieval
WO2022147049A1 (en) * 2021-01-04 2022-07-07 Alibaba Group Holding Limited Method, apparatus, and electronic device for obtaining trademark similarity
US20220365636A1 (en) * 2019-06-26 2022-11-17 Radius5 Inc. Image display system and program

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7130466B2 (en) * 2000-12-21 2006-10-31 Cobion Ag System and method for compiling images from a database and comparing the compiled images with known images
US7266545B2 (en) * 2001-08-07 2007-09-04 International Business Machines Corporation Methods and apparatus for indexing in a database and for retrieving data from a database in accordance with queries using example sets
US20030103673A1 (en) * 2001-11-30 2003-06-05 Nainesh Rathod Shape searcher
JP2006119723A (en) * 2004-10-19 2006-05-11 Canon Inc Device and method for image processing
CA2510644A1 (en) * 2005-06-23 2006-12-23 Cognos Incorporated Quality of service feedback for technology-neutral data reporting
JP4655824B2 (en) * 2005-08-24 2011-03-23 富士ゼロックス株式会社 Image recognition apparatus, image recognition method, and image recognition program
JP2008165424A (en) * 2006-12-27 2008-07-17 Sony Corp Image retrieval device and method, imaging device and program
US8396331B2 (en) * 2007-02-26 2013-03-12 Microsoft Corporation Generating a multi-use vocabulary based on image data
US8050503B2 (en) * 2007-07-05 2011-11-01 General Electric Company Systems and methods for computer vision using curvelets
US8429173B1 (en) 2009-04-20 2013-04-23 Google Inc. Method, system, and computer readable medium for identifying result images based on an image query
US8571330B2 (en) * 2009-09-17 2013-10-29 Hewlett-Packard Development Company, L.P. Video thumbnail selection
CN102193934B (en) * 2010-03-11 2013-05-29 株式会社理光 System and method for searching representative image of image set
US8401282B2 (en) * 2010-03-26 2013-03-19 Mitsubishi Electric Research Laboratories, Inc. Method for training multi-class classifiers with active selection and binary feedback
ES2384928B1 (en) * 2010-06-25 2013-05-20 Telefónica, S.A. METHOD AND SYSTEM FOR THE QUICK AND ROBUST IDENTIFICATION OF SPECIFIC PRODUCTS IN IMAGES.
US9412020B2 (en) 2011-11-09 2016-08-09 Board Of Regents Of The University Of Texas System Geometric coding for billion-scale partial-duplicate image search
US10013436B1 (en) 2014-06-17 2018-07-03 Google Llc Image annotation based on label consensus

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5579471A (en) * 1992-11-09 1996-11-26 International Business Machines Corporation Image query system and method
US5794237A (en) * 1995-11-13 1998-08-11 International Business Machines Corporation System and method for improving problem source identification in computer systems employing relevance feedback and statistical source ranking
US5819288A (en) * 1996-10-16 1998-10-06 Microsoft Corporation Statistically based image group descriptor particularly suited for use in an image classification and retrieval system
US5893905A (en) * 1996-12-24 1999-04-13 Mci Communications Corporation Automated SLA performance analysis monitor with impact alerts on downstream jobs
US5911139A (en) * 1996-03-29 1999-06-08 Virage, Inc. Visual image database search engine which allows for different schema
US5924105A (en) * 1997-01-27 1999-07-13 Michigan State University Method and product for determining salient features for use in information searching
US5963670A (en) * 1996-02-12 1999-10-05 Massachusetts Institute Of Technology Method and apparatus for classifying and identifying images
US5987457A (en) * 1997-11-25 1999-11-16 Acceleration Software International Corporation Query refinement method for searching documents
US6072904A (en) * 1997-12-31 2000-06-06 Philips Electronics North America Corp. Fast image retrieval using multi-scale edge representation of images
US20010003185A1 (en) * 1999-12-02 2001-06-07 Lg Electronics, Inc. Method for updating multimedia feature information
US20020174120A1 (en) * 2001-03-30 2002-11-21 Hong-Jiang Zhang Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (CBIR)

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5579471A (en) * 1992-11-09 1996-11-26 International Business Machines Corporation Image query system and method
US5794237A (en) * 1995-11-13 1998-08-11 International Business Machines Corporation System and method for improving problem source identification in computer systems employing relevance feedback and statistical source ranking
US5963670A (en) * 1996-02-12 1999-10-05 Massachusetts Institute Of Technology Method and apparatus for classifying and identifying images
US5911139A (en) * 1996-03-29 1999-06-08 Virage, Inc. Visual image database search engine which allows for different schema
US5819288A (en) * 1996-10-16 1998-10-06 Microsoft Corporation Statistically based image group descriptor particularly suited for use in an image classification and retrieval system
US5893905A (en) * 1996-12-24 1999-04-13 Mci Communications Corporation Automated SLA performance analysis monitor with impact alerts on downstream jobs
US5924105A (en) * 1997-01-27 1999-07-13 Michigan State University Method and product for determining salient features for use in information searching
US5987457A (en) * 1997-11-25 1999-11-16 Acceleration Software International Corporation Query refinement method for searching documents
US6072904A (en) * 1997-12-31 2000-06-06 Philips Electronics North America Corp. Fast image retrieval using multi-scale edge representation of images
US20010003185A1 (en) * 1999-12-02 2001-06-07 Lg Electronics, Inc. Method for updating multimedia feature information
US20020174120A1 (en) * 2001-03-30 2002-11-21 Hong-Jiang Zhang Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (CBIR)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040258311A1 (en) * 2001-10-11 2004-12-23 Siemens Aktiengesellschaft Method for generating geometric models for optical partial recognition
US20030179213A1 (en) * 2002-03-18 2003-09-25 Jianfeng Liu Method for automatic retrieval of similar patterns in image databases
GB2415562A (en) * 2004-06-23 2005-12-28 Hewlett Packard Development Co Generating saliency paths for viewing images
US20050286739A1 (en) * 2004-06-23 2005-12-29 Maurizio Pilu Image processing
GB2415562B (en) * 2004-06-23 2007-11-21 Hewlett Packard Development Co Image processing
US7844075B2 (en) * 2004-06-23 2010-11-30 Hewlett-Packard Development Company, L.P. Image processing
US20060193536A1 (en) * 2005-01-31 2006-08-31 Maurizio Pilu Image processing method and apparatus
US7606442B2 (en) * 2005-01-31 2009-10-20 Hewlett-Packard Development Company, L.P. Image processing method and apparatus
WO2007137352A1 (en) * 2006-05-29 2007-12-06 University Of Wollongong Content based image retrieval
US20220365636A1 (en) * 2019-06-26 2022-11-17 Radius5 Inc. Image display system and program
US11698715B2 (en) * 2019-06-26 2023-07-11 Radius5 Inc. Image display system and program
WO2022147049A1 (en) * 2021-01-04 2022-07-07 Alibaba Group Holding Limited Method, apparatus, and electronic device for obtaining trademark similarity

Also Published As

Publication number Publication date
TW511010B (en) 2002-11-21
US6834288B2 (en) 2004-12-21

Similar Documents

Publication Publication Date Title
US6834288B2 (en) Content-based similarity retrieval system for image data
Chen et al. A region-based fuzzy feature matching approach to content-based image retrieval
Jing et al. Learning in region-based image retrieval
US7457825B2 (en) Generating search requests from multimodal queries
US9424277B2 (en) Methods and apparatus for automated true object-based image analysis and retrieval
Zhao et al. Bridging the semantic gap in image retrieval
Rui et al. Image retrieval: Past, present, and future
US6704725B1 (en) Method of searching multimedia data
Deselaers et al. Fire–flexible image retrieval engine: Imageclef 2004 evaluation
Taycher et al. Image digestion and relevance feedback in the ImageRover WWW search engine
Vadivel et al. Performance comparison of distance metrics in content-based image retrieval applications
GB2439842A (en) Search engine
Jan et al. Region of interest-based image retrieval techniques: a review
Di Sciascio et al. Query by sketch and relevance feedback for content-based image retrieval over the web
Vadivel et al. Characteristics of weighted feature vector in content-based image retrieval applications
Li et al. Introduction to Content‐Based Image Retrieval—Overview of Key Techniques
Costache et al. Categorization based relevance feedback search engine for earth observation images repositories
Simpson et al. Text-and Content-based Approaches to Image Retrieval for the ImageCLEF 2009 Medical Retrieval Track.
Park et al. Web image retrieval using majority-based ranking approach
Paredes et al. A probabilistic model for user relevance feedback on image retrieval
Di Sciascio et al. DrawSearch: a tool for interactive content-based image retrieval over the Internet
Wang Looking beyond region boundaries: Region-based image retrieval using fuzzy feature matching
Dobrescu et al. Fractal dimension based technique for database image retrieval
Salamah Efficient content based image retrieval
Sun et al. HIRBIR: A hierarchical approach to region-based image retrieval

Legal Events

Date Code Title Description
AS Assignment

Owner name: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE, TAIWAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, JIANN-JONE;CHIANG, CHEN-CHIN;REEL/FRAME:011702/0463;SIGNING DATES FROM 20010323 TO 20010329

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

REMI Maintenance fee reminder mailed
FPAY Fee payment

Year of fee payment: 8

FPAY Fee payment

Year of fee payment: 12