US5586197A - Image searching method and apparatus thereof using color information of an input image - Google Patents

Image searching method and apparatus thereof using color information of an input image Download PDF

Info

Publication number
US5586197A
US5586197A US08/300,488 US30048894A US5586197A US 5586197 A US5586197 A US 5586197A US 30048894 A US30048894 A US 30048894A US 5586197 A US5586197 A US 5586197A
Authority
US
United States
Prior art keywords
color information
image
luminance
color
image searching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US08/300,488
Inventor
Kazuko Tsujimura
Yuichi Bannai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Assigned to CANON KABUSHIKI KAISHA reassignment CANON KABUSHIKI KAISHA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BANNAI, YUICHI, TSUJIMURA, KAZUKO
Application granted granted Critical
Publication of US5586197A publication Critical patent/US5586197A/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour

Definitions

  • the present invention relates to an image processing method and apparatus and, more particularly, to a method to search for an image in an image database, and the apparatus capable of employing the method.
  • each contribution factor of the three attributes of color differs in accordance with values of the three attributes of colors. For instance, in a color which has low chromaticity that is close to black or white, and low luminance, a contribution factor of hue which is an attribute to distinguish the difference of color, such as red, blue, and yellow, is also low. If the degree of similarity is calculated on the assumption that the contribution factors of the attributes are the same for all the colors in spite of the aforementioned fact that the contribution factors actually differ, the searching quality becomes lower since low contribution values are included in the calculation of similarity.
  • the present invention has been made in consideration of the above situation, and has as its object to provide an image processing method and apparatus which make the quality of search higher by referring to color information when an image is searched out of the image database, and which make it easier for an operator to search for the desired image.
  • an image searching method for searching an image database by using color information containing a plurality of color components comprising a contribution factor calculation step of calculating a contribution factor of each color component of the color information on the basis of the color information, a similarity calculation step of calculating similarity between the color information and a plurality of predetermined color information based on the contribution factor of each color component of color information, and a sorting step of sorting color information corresponding to similarity in order of similarity, and of displaying images.
  • an image database searching method which searches for an image from the image database by using color information, comprising a generating step of generating a sample image including color information, a contribution factor calculation step of calculating a contribution factor of color information based on the color information of the sample image, a similarity calculation step of calculating similarity between the sample image containing the color information and respective images from an image database, according to the contribution factor, and a displaying step of displaying images from the image database on a display means in order of similarity.
  • an image database searching apparatus comprising contribution factor calculation means for calculating a contribution factor of each color component of color information on the basis of the color information, degree of similarity calculation means for calculating similarity between the color information and a plurality of predetermined color information based on the contribution factor of each color component of color information, and display means for sorting color information corresponding to similarity in the order of similarity, and displaying images, wherein the apparatus searches for images by using color information containing a plurality of color components.
  • an image searching apparatus comprising generating means for generating a sample image including color information, contribution factor calculation means for calculating a contribution factor of color information based on the color information of the sample image, degree of similarity calculation means for calculating similarity between the sample image containing the color information and respective images from an image database, according to the contribution factor, and displaying means for displaying images from the image database in order of similarity, wherein the apparatus searches for the image from the image database by using color information.
  • FIG. 1 is a block diagram showing a block diagram of an image searching apparatus in first and second embodiments
  • FIG. 2 is a flowchart of a searching processing sequence
  • FIG. 3 is a flowchart showing a procedure for making a sample image
  • FIG. 4 illustrates a structure of a display for designating colors when making the sample image
  • FIG. 5 is a flowchart showing a process to determine similarity
  • FIG. 6 is a flowchart showing a process for calculating each contribution factor of the three attributes of color data
  • FIG. 7 illustrates a shape of a membership function of luminance
  • FIG. 8 illustrates a shape of a membership function of chromaticity
  • FIG. 9 is a flowchart showing a process for calculating the contribution factor using the membership functions.
  • FIG. 10 is a flowchart showing another process of calculating the contribution factor using the membership functions.
  • a HSV H stands for hue, S for chromaticity, and V for luminance
  • reference numeral 10 denotes a stored image data input unit for inputting image data to be stored in database
  • 20 denotes an image storage unit for storing the image data input at the stored image data input unit
  • 30 denotes a sample image input unit for inputting a sample image made by an operator to obtain the desired image, where the sample image input unit 30 has a pointing input device 31.
  • an image color information extraction unit 40 extracts color information out of the image data stored in the image storage unit 20 by using the HSV color space
  • a sample color information extraction unit 50 extracts color information out of the sample image input at the sample image input unit 30 by using the HSV color space
  • a contribution factors deciding unit 60 decides the contribution factors of hue H, chromaticity S, and luminance V of color data in the HSV color space.
  • Unit 70 is a candidate image deciding unit to decide the searched candidate images by calculating the similarity
  • unit 80 is a control unit to control the entire apparatus in this embodiment
  • unit 90 is a display unit which displays the searched candidate images.
  • FIG. 2 is a flowchart of an image searching process performed by the apparatus in FIG. 1. An image searching process is explained below with reference to FIG. 2.
  • step S1 a sample image of a desired image is made and input by the operator at the sample image input unit 30, and the sample image is transmitted to the control unit 80.
  • the process of making the sample image is explained later in detail with reference to the flowchart in FIG. 3 and the illustration of the display in FIG. 4.
  • the sample color information extraction unit 50 extracts the color information, namely, the values of the three attributes in the HSV color space, H1, S1, and V1, out of the input sample image.
  • the representative value of the designated portion is defined by a mode color, however, it can be defined by a mean or a median in statistics, for instance.
  • step S4 the similarity between the sample image and the stored images are calculated based on two different sets of the extracted color information, that is, (H1, S1, V1) and (H2i, S2i, V2i), and also on each contribution factor decided in the contribution factors deciding unit 60. This process is explained later in detail with reference to FIG. 5.
  • the sorted images are displayed in order of the degree of the similarity on the display unit 90.
  • the desired image is searched for from the stored images based on the color information of the sample image made by the operator by following the aforementioned procedure.
  • step S2 the sample image input process of step S2 is explained with reference to the flowchart of FIG. 3.
  • the operator makes the sample image in a window which is open on the screen of the display unit 90 in the image processing apparatus by following the procedure shown in FIG. 3.
  • the operator designates a portion which represents an object image in the desired image by the pointing input device 31.
  • a color is designated in the portion.
  • An example of color designation by using the screen of the display unit 90 is illustrated in FIG. 4.
  • Reference numeral 41 denotes six color chips including the three primary colors (red, green, blue) and the intermediate colors of the three primary colors (yellow, cyan, magenta), 42 denotes slide-bars to adjust each value of the HSV.
  • the operator designates a color which is most similar to the color of portion by using the pointing input device 31. Then the indicated color is considered as standard, and each values of the HSV is adjusted so that the color becomes more similar to the color of the object in the desired image by moving the slide-bars with the pointing input device 31.
  • the color under adjustment is displayed in a box 43 on the screen, so that the operator can see what shade of the color the operator designated. After the adjustment is finished, the color designation is completed by clicking a "quit" display using the pointing input device 31. Thus, the color of the portion of the sample image is decided.
  • step S12 the portion which is selected at step S10 is filled with the color designated at step S11.
  • step S13 the operator checks whether or not there is a need to designate another portion, and if not, the process of making the sample image is completed. Further, if another portion needs to be designated, the process returns to step 10, and the process described above is repeated.
  • the sample image is made by performing the aforementioned steps.
  • each contribution factor, RH, RS, RV, for normalizing the three attributes of the color information of the sample image, H1, S1, and V1 is decided by using the contribution factors deciding unit 60.
  • the process in the contribution factors deciding unit 60 is described later with reference to FIG. 6.
  • step S31 the value differences of the three attributes between the color information of the sample image, H1, S1, and V1, and the color information of the stored images, H2i, S2i, and V2i, are calculated.
  • ), ⁇ Si( ⁇ Si
  • S1-S2i), and ⁇ Vi ( ⁇ Vi
  • ), where i 1, 2, 3, . . . ,n: where n is the total number of stored images.
  • step S30 the process to decide the contribution factor of step S30 is described below in detail with reference to a flowchart in FIG. 6.
  • the flowchart shows the process performed by the contribution factors deciding unit 60.
  • each contribution factor of the three attributes is denoted by RH, RS and RV, and each of the factors ranges from 0.0 to 1.0.
  • the normalized values of the three attributes, that is the normalized contribution factor are denoted by RH, RS and RV.
  • Values of the normalized contribution factors, which are output from the contribution factors deciding unit 60, are RH, RS and RV which are obtained after the normalization process described below.
  • step S20 the color information of the sample image which is extracted at step S2 in FIG. 2 is input to the contribution factors deciding unit 60 from the sample image input unit 30 via the control unit 80.
  • the desired image is searched for in of the stored images in accordance with the color information of the sample image as determined by the operator.
  • the image can be drawn on paper or the like, and read by a scanner instead of drawing the image on a display.
  • Input values of color data such as RGB and HSV, in the form of a number.
  • RGB color data is used as the sample image data
  • the RGB color data needs to be converted into HSV space data, since the image data in the image storage unit 20 is HSV space data.
  • This conversion from the RGB data to the HSV data is performed by using the following conversion algorithm. The conversion algorithm is fully explained in Fundamentals of Interactive Computer Graphics(J. D. Foley, A. Vandam, pp613-620).
  • the HSV color space is employed as a color perception space which is defined by hue, chromaticity, and luminance.
  • hue, chromaticity, and luminance are defined by hue, chromaticity, and luminance.
  • other color perception spaces such as a Munsell color space.
  • thresholds Tv and Ts are predetermined to obtain the contribution factors. There is another method to determine the contribution factor by using predetermined membership functions. In a second embodiment, another example of the operation of the contribution factors deciding unit 60 is described.
  • FIGS. 7 and 8 are examples of the shapes of membership functions of luminance and chromaticity, respectively.
  • a horizontal axis shows the value of luminance
  • a vertical axis shows the membership grade of the corresponding luminance.
  • a horizontal axis shows the value of chromaticity
  • a vertical axis shows the membership grade of the corresponding chromaticity.
  • the membership functions in FIG. 7 illustrates a membership function 101 which indicates that the luminance is low, and a membership function 100 which indicates that the luminance is not low.
  • the membership functions in FIG. 8 illustrates two membership functions 103 and 102, which indicates that the chromaticity is low, and that the chromaticity is not low, respectively.
  • the membership functions 101 and 100 are denoted by M1v(i) and Mhv(i), respectively, hereinafter.
  • i in the parenthesis it indicates luminance.
  • M1v(i) and Mhv(i) are the membership grades relating to the luminance i.
  • the membership functions 103 and 102 are denoted by M1s(j) and Mhs(j), respectively, hereinafter.
  • j in the parenthesis it indicates chromaticity. Therefore, M1s(j) and Mhs(j) are the membership grades relating to the chromaticity j.
  • contribution factors i.e., RH, RS and RV
  • contribution factors i.e., RH, RS and RV
  • membership functions The followings are the processes at each step.
  • step S40 color information of a sample image (H1, S1, V1) which is extracted at step S2 in FIG. 2, is input to the contribution factors deciding unit 60 from the sample image input unit 30 via the control unit 80.
  • membership grades, M1v(V1) and Mhv(V1) which correspond to luminance V1 are obtained by using the membership functions 101 and 100 in FIG. 7.
  • membership grades, M1s(S1) and Mhv(S1) corresponding to chromaticity S1 are obtained by referring to the membership functions 103 and 102 in FIG. 8.
  • the contribution factors, C1(x), C2(x), and C3(x) corresponding to three cases of the color data x are obtained based on the respective membership grades which are obtained at steps S41 and S42, where the three cases are case 1 (low luminance V), case 2 (high luminance V and low chromaticity S), and case 3 (high luminance V and high chromaticity S).
  • the values of C1(x), C2(x) and C3(x) are given by the following equations:
  • step S44 the contribution factors of the three attributes of colors, Rh, Rs and Rv are obtained by performing a matrix operation represented by S44 in FIG. 10.
  • the present invention can be applied to a system constituted by a plurality of devices, or to an apparatus comprising a single device. Furthermore, the invention is also applicable to a case where the object of the invention is attained by supplying a program to a system or apparatus.
  • the contribution factor of an image color component of better quality is obtained by using the membership functions, such that performance of the image search is enhanced.
  • the quality of search using color information is improved when an image is searched from an image database.

Abstract

A control unit controls an image searching system which performs a high quality search for a desired image in an image database by using color information. A sample image determined by an operator is used as a key to the search, and is input to a sample image input unit. A sample color information extraction unit extracts HSV color space data from the sample image, and an image color information extraction unit extracts HSV color image data from a plurality of images stored in the image database. A contribution factors deciding unit decides a contribution factor for each component in the HSV color space, and a candidate image deciding unit calculates a similarity between the plurality of images stored in the database and the sample image. Image candidates are determined based on the calculated similarity, and the image candidates are displayed on a display unit. The desired image is then selected from the displayed image candidates.

Description

BACKGROUND OF THE INVENTION
The present invention relates to an image processing method and apparatus and, more particularly, to a method to search for an image in an image database, and the apparatus capable of employing the method.
As one of the conventional methods to search for a necessary image out of an image database, there is a method in which an operator makes an example of the desired image, and that the search is performed with reference to color information of the image. The color information is extracted from the desired image by using a color perception space which is composed of three elements, namely, hue, chromaticity, and luminance (altogether, called `the three attributes of colors`, hereinafter). Then, similarity between the extracted color information of the desired image and color information extracted from an image stored in the image database by employing the same color perception space, is examined. Thereby, the order of image candidates, which can be the same as, or which can be similar to the desired image made by the operator, is decided based on the similarity. The image candidate which has a higher similarity is more likely similar to the desired image. Then the images are displayed in order of similarity.
Regarding color data, each contribution factor of the three attributes of color differs in accordance with values of the three attributes of colors. For instance, in a color which has low chromaticity that is close to black or white, and low luminance, a contribution factor of hue which is an attribute to distinguish the difference of color, such as red, blue, and yellow, is also low. If the degree of similarity is calculated on the assumption that the contribution factors of the attributes are the same for all the colors in spite of the aforementioned fact that the contribution factors actually differ, the searching quality becomes lower since low contribution values are included in the calculation of similarity.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above situation, and has as its object to provide an image processing method and apparatus which make the quality of search higher by referring to color information when an image is searched out of the image database, and which make it easier for an operator to search for the desired image.
According to the present invention, the foregoing object is attained by providing an image searching method for searching an image database by using color information containing a plurality of color components, comprising a contribution factor calculation step of calculating a contribution factor of each color component of the color information on the basis of the color information, a similarity calculation step of calculating similarity between the color information and a plurality of predetermined color information based on the contribution factor of each color component of color information, and a sorting step of sorting color information corresponding to similarity in order of similarity, and of displaying images.
According to the present invention, another object is attained by providing an image database searching method which searches for an image from the image database by using color information, comprising a generating step of generating a sample image including color information, a contribution factor calculation step of calculating a contribution factor of color information based on the color information of the sample image, a similarity calculation step of calculating similarity between the sample image containing the color information and respective images from an image database, according to the contribution factor, and a displaying step of displaying images from the image database on a display means in order of similarity.
According to the present invention, a further object is attained by providing an image database searching apparatus comprising contribution factor calculation means for calculating a contribution factor of each color component of color information on the basis of the color information, degree of similarity calculation means for calculating similarity between the color information and a plurality of predetermined color information based on the contribution factor of each color component of color information, and display means for sorting color information corresponding to similarity in the order of similarity, and displaying images, wherein the apparatus searches for images by using color information containing a plurality of color components.
According to the present invention, a further object is attained by providing an image searching apparatus comprising generating means for generating a sample image including color information, contribution factor calculation means for calculating a contribution factor of color information based on the color information of the sample image, degree of similarity calculation means for calculating similarity between the sample image containing the color information and respective images from an image database, according to the contribution factor, and displaying means for displaying images from the image database in order of similarity, wherein the apparatus searches for the image from the image database by using color information.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
FIG. 1 is a block diagram showing a block diagram of an image searching apparatus in first and second embodiments;
FIG. 2 is a flowchart of a searching processing sequence;
FIG. 3 is a flowchart showing a procedure for making a sample image;
FIG. 4 illustrates a structure of a display for designating colors when making the sample image;
FIG. 5 is a flowchart showing a process to determine similarity;
FIG. 6 is a flowchart showing a process for calculating each contribution factor of the three attributes of color data;
FIG. 7 illustrates a shape of a membership function of luminance;
FIG. 8 illustrates a shape of a membership function of chromaticity;
FIG. 9 is a flowchart showing a process for calculating the contribution factor using the membership functions; and
FIG. 10 is a flowchart showing another process of calculating the contribution factor using the membership functions.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Preferred embodiments of the present invention will be described in detail in accordance with the accompanying drawings.
(First Embodiment)
In the first embodiment, a HSV (H stands for hue, S for chromaticity, and V for luminance) color space, for instance, is employed as a color perception space for indicating color data by hue, chromaticity, and luminance. In FIG. 1, reference numeral 10 denotes a stored image data input unit for inputting image data to be stored in database, 20 denotes an image storage unit for storing the image data input at the stored image data input unit 10, and 30 denotes a sample image input unit for inputting a sample image made by an operator to obtain the desired image, where the sample image input unit 30 has a pointing input device 31. Further, an image color information extraction unit 40 extracts color information out of the image data stored in the image storage unit 20 by using the HSV color space, a sample color information extraction unit 50 extracts color information out of the sample image input at the sample image input unit 30 by using the HSV color space, and a contribution factors deciding unit 60 decides the contribution factors of hue H, chromaticity S, and luminance V of color data in the HSV color space. Unit 70 is a candidate image deciding unit to decide the searched candidate images by calculating the similarity, unit 80 is a control unit to control the entire apparatus in this embodiment, and unit 90 is a display unit which displays the searched candidate images.
FIG. 2 is a flowchart of an image searching process performed by the apparatus in FIG. 1. An image searching process is explained below with reference to FIG. 2.
At step S1, a sample image of a desired image is made and input by the operator at the sample image input unit 30, and the sample image is transmitted to the control unit 80. The process of making the sample image is explained later in detail with reference to the flowchart in FIG. 3 and the illustration of the display in FIG. 4.
At step S2, the sample color information extraction unit 50 extracts the color information, namely, the values of the three attributes in the HSV color space, H1, S1, and V1, out of the input sample image.
At step S3, the image color information extraction unit 40 find a mode color in pixel units (the "mode" here is a term in statistics) in a designated portion of each image i (i=1,2, . . . n: n is a total number of stored images) which is stored in the image storage unit 20, where the designated portion of each image i corresponds to the portion designated by the operator in the sample image, and the pixel mode color is defined as a representative value of the portion. Then each value of three attributes in the HSV color space, H2i, S2i, and V2i, are extracted.
It should be noted that in the aforementioned embodiment the representative value of the designated portion is defined by a mode color, however, it can be defined by a mean or a median in statistics, for instance.
At step S4, the similarity between the sample image and the stored images are calculated based on two different sets of the extracted color information, that is, (H1, S1, V1) and (H2i, S2i, V2i), and also on each contribution factor decided in the contribution factors deciding unit 60. This process is explained later in detail with reference to FIG. 5.
At step S5, the stored images are sorted in order of similarity in accordance with the similarity Ri (i=1,2,3, n). A stored image which has the higher similarity is considered as more similar image to the sample image than the others. Note that the processes 4 and 5 are performed at the candidate image deciding unit 70.
At step 6, the sorted images are displayed in order of the degree of the similarity on the display unit 90.
Thus, the desired image is searched for from the stored images based on the color information of the sample image made by the operator by following the aforementioned procedure.
Next, the sample image input process of step S2 is explained with reference to the flowchart of FIG. 3. The operator makes the sample image in a window which is open on the screen of the display unit 90 in the image processing apparatus by following the procedure shown in FIG. 3.
At step S10, the operator designates a portion which represents an object image in the desired image by the pointing input device 31.
At step S11, a color is designated in the portion. An example of color designation by using the screen of the display unit 90 is illustrated in FIG. 4. Reference numeral 41 denotes six color chips including the three primary colors (red, green, blue) and the intermediate colors of the three primary colors (yellow, cyan, magenta), 42 denotes slide-bars to adjust each value of the HSV. The operator designates a color which is most similar to the color of portion by using the pointing input device 31. Then the indicated color is considered as standard, and each values of the HSV is adjusted so that the color becomes more similar to the color of the object in the desired image by moving the slide-bars with the pointing input device 31. The color under adjustment is displayed in a box 43 on the screen, so that the operator can see what shade of the color the operator designated. After the adjustment is finished, the color designation is completed by clicking a "quit" display using the pointing input device 31. Thus, the color of the portion of the sample image is decided.
At step S12, the portion which is selected at step S10 is filled with the color designated at step S11.
At step S13, the operator checks whether or not there is a need to designate another portion, and if not, the process of making the sample image is completed. Further, if another portion needs to be designated, the process returns to step 10, and the process described above is repeated.
Thus, the sample image is made by performing the aforementioned steps.
Next, the process to obtain the respective similarity of step S4 between the sample image of step S2 and the stored images of step S3 is described in detail with reference to the flowchart in FIG. 5. Note that it is assumed that only one portion of the image is designated by the operator. In this process, the similarity is calculated based on the color information of the sample image (H1, S1, V1) and the color information of each stored image (H2i, S2i, V2i, where i=1, 2, 3, . . . ,n: n is a total number of the stored images).
At step S30, first, each contribution factor, RH, RS, RV, for normalizing the three attributes of the color information of the sample image, H1, S1, and V1 is decided by using the contribution factors deciding unit 60. The process in the contribution factors deciding unit 60 is described later with reference to FIG. 6.
At step S31, the value differences of the three attributes between the color information of the sample image, H1, S1, and V1, and the color information of the stored images, H2i, S2i, and V2i, are calculated. The respective differences are described as, ΔHi(ΔHi=|H1-H2i|), ΔSi(ΔSi=|S1-S2i), and ΔVi (ΔVi=|V1-V2i|), where i=1, 2, 3, . . . ,n: where n is the total number of stored images.
At step S32, color differences, in which the contribution factors are considered, DHi(DHi=RH*Δhi), DSi(DSi=RS*ΔSi), and Dvi(Dvi=RV*ΔVi), where i=1, 2, 3, . . . ,n: n n: n is the total number of the stored images, are calculated on the basis of RH, RS, and RV which are obtained at step S30, and of ΔHi, ΔSi, and ΔVi which are obtained at step S31.
At step S33, DHi, DSi, and DVi are added, and similarity Ri (i=1, 2, 3, . . . , n: n is the total number of the stored images) is defined by the negative value of the result of the addition (-(DHi+DSi+DVi)). Likewise, similarity Ri(i=1, 2, 3, . . . ,n) of the each stored image are calculated. In a case where a plurality of portion are designated, the similarity Rij (i=1, 2, 3, . . . ,n:j=1, 2, 3, . . . ,m: m is a total number of a portion.) are calculated for each image. Then a representative degree of similarity of each image is defined by, for instance, the total value of the degree of similarity relating the images, namely, the result of addition Ri=Ri1+Ri2+. . . +Rim.
Next, the process to decide the contribution factor of step S30 is described below in detail with reference to a flowchart in FIG. 6. The flowchart shows the process performed by the contribution factors deciding unit 60.
Note that each contribution factor of the three attributes is denoted by RH, RS and RV, and each of the factors ranges from 0.0 to 1.0. Further, the normalized values of the three attributes, that is the normalized contribution factor, are denoted by RH, RS and RV. Values of the normalized contribution factors, which are output from the contribution factors deciding unit 60, are RH, RS and RV which are obtained after the normalization process described below.
At step S20, the color information of the sample image which is extracted at step S2 in FIG. 2 is input to the contribution factors deciding unit 60 from the sample image input unit 30 via the control unit 80.
At step S21, the color information of the sample image which is read at step S20 is examined, as to whether or not the luminance V1 is low, more specifically, whether or not the value of the luminance V1 is greater or equal to a predetermined threshold Tv. If the value of luminance V1 is less than the threshold Tv, the process proceeds to step S26 and the contribution factors of hue H, chromaticity S, and luminance V are assigned as Rh=0, Rs=0, and Rv=1, respectively. In other words, the low luminance results in a low contribution factor of chromaticity. Then the process moves to step S24.
At step S24, since the contribution factors, Rh, Rs, and Rv are decided, these three values are normalized to obtain the normalized contribution factors RH, RS and RV. The following are the equations for the normalizing calculation:
RH =Rh/(Rh+Rs+Rv)
RS =Rs/(Rh+Rs+Rv)
RV =Rv/(Rh+Rs+Rv)
Next, at step S22, whether or not the chromaticity S1 is low is , more specifically, whether or not the value of the chromaticity S1 is greater or equal to a predetermined threshold Ts. If the value of chromaticity S1 is less than the threshold Ts, the process proceeds to step S27, and the contribution factors of hue H, chromaticity S, and luminance V are assigned as Rh=0, Rs=1, and Rv=1, respectively. Then the process moves to step S24. On the other hand, if the value of chromaticity S1 is larger than the threshold Ts, the process proceeds to step S23, and contribution factors of hue H, chromaticity S, and luminance V are assigned as Rh=1, Rs=0, and Rv=0, respectively. Then the process moves to step S24 to perform a normalizing procedure.
Thus, the normalized contribution factors of the three attributes, RH, RS and RV are decided by following the above described procedure.
As explained above, the desired image is searched for in of the stored images in accordance with the color information of the sample image as determined by the operator.
Note that as an example of a sample image input means, the image can be drawn on paper or the like, and read by a scanner instead of drawing the image on a display.
Further, various ways to designate a color of a portion during the process of making a sample image are as follows:
1. Choose a color out of a limited number of colors which are displayed on a screen,
2. Designate a color by using color chips, and input a discrimination number of the designated color chip, and
3. Input values of color data, such as RGB and HSV, in the form of a number.
Further, in a case where RGB color data is used as the sample image data, the RGB color data needs to be converted into HSV space data, since the image data in the image storage unit 20 is HSV space data. This conversion from the RGB data to the HSV data is performed by using the following conversion algorithm. The conversion algorithm is fully explained in Fundamentals of Interactive Computer Graphics(J. D. Foley, A. Vandam, pp613-620).
______________________________________                                    
max = The maximum value of R, G, and B                                    
min = The minimum value of R, G, and B                                    
if(max ≠ 0)                                                         
then S = (max-min)/max                                                    
else S = 0                                                                
if S = 0 then H = undefined                                               
else                                                                      
begin   rc = (max-R)/(max-min)                                            
        gc = (max-G)/(max-min)                                            
        bc = (max-B)/(max-min)                                            
if(R = max) then                                                          
H = bc-gc                                                                 
else if(G = max) then                                                     
H = 2+rc-bc                                                               
else if(B = max) then                                                     
H = 4+gc-rc                                                               
if(H ≦ 0) then H = H+360                                           
end                                                                       
______________________________________                                    
Note that in the first embodiment the HSV color space is employed as a color perception space which is defined by hue, chromaticity, and luminance. However, needless to say, it is possible to employ other color perception spaces, such as a Munsell color space.
As explained above, according to this embodiment, it is easier for an operator to search for a desired image in an image database by improving the quality of the search by using color information when searching for similar images to the desired image.
(Second Embodiment)
As a method for deciding contribution factors of the three attributes of color data in the first embodiment, thresholds Tv and Ts are predetermined to obtain the contribution factors. There is another method to determine the contribution factor by using predetermined membership functions. In a second embodiment, another example of the operation of the contribution factors deciding unit 60 is described.
FIGS. 7 and 8 are examples of the shapes of membership functions of luminance and chromaticity, respectively. In FIG. 7, a horizontal axis shows the value of luminance, and a vertical axis shows the membership grade of the corresponding luminance. Similarly, in FIG. 8, a horizontal axis shows the value of chromaticity, and a vertical axis shows the membership grade of the corresponding chromaticity. The membership functions in FIG. 7 illustrates a membership function 101 which indicates that the luminance is low, and a membership function 100 which indicates that the luminance is not low. Likewise, the membership functions in FIG. 8 illustrates two membership functions 103 and 102, which indicates that the chromaticity is low, and that the chromaticity is not low, respectively. The membership functions 101 and 100 are denoted by M1v(i) and Mhv(i), respectively, hereinafter. Regarding i in the parenthesis, it indicates luminance. Further, M1v(i) and Mhv(i) are the membership grades relating to the luminance i. In the same manner, the membership functions 103 and 102 are denoted by M1s(j) and Mhs(j), respectively, hereinafter. Regarding j in the parenthesis, it indicates chromaticity. Therefore, M1s(j) and Mhs(j) are the membership grades relating to the chromaticity j.
The process for deciding contribution factors of the three attributes of color data is explained below with reference to FIGS. 9 and 10. Here, contribution factors (i.e., RH, RS and RV) of color data X(H1, S1, V1) of a sample image illustrated by using the HSV color space, RH, RS and RV are calculated by using membership functions. The followings are the processes at each step.
At step S40, color information of a sample image (H1, S1, V1) which is extracted at step S2 in FIG. 2, is input to the contribution factors deciding unit 60 from the sample image input unit 30 via the control unit 80.
At step S41, membership grades, M1v(V1) and Mhv(V1) which correspond to luminance V1, are obtained by using the membership functions 101 and 100 in FIG. 7.
At step S42, membership grades, M1s(S1) and Mhv(S1) corresponding to chromaticity S1, are obtained by referring to the membership functions 103 and 102 in FIG. 8.
At step S43, the contribution factors, C1(x), C2(x), and C3(x) corresponding to three cases of the color data x are obtained based on the respective membership grades which are obtained at steps S41 and S42, where the three cases are case 1 (low luminance V), case 2 (high luminance V and low chromaticity S), and case 3 (high luminance V and high chromaticity S). The values of C1(x), C2(x) and C3(x) are given by the following equations:
C1(X)=M1v(V1)
C2(X)=min(Mhv(V1), M1s(S1))
C3(X)=min(Mhv(V1), Mhs(S1))
At step S44, the contribution factors of the three attributes of colors, Rh, Rs and Rv are obtained by performing a matrix operation represented by S44 in FIG. 10.
At step S45, these three values are normalized, and the normalized contribution factors RH, RS and RV are obtained. The normalized contribution factors are given by the following equations:
RH=Rh/(Rh+Rs+Rv)
RS=Rs/(Rh+Rs+Rv)
RV=Rv/(Rh+Rs+Rv)
The present invention can be applied to a system constituted by a plurality of devices, or to an apparatus comprising a single device. Furthermore, the invention is also applicable to a case where the object of the invention is attained by supplying a program to a system or apparatus.
As described above, according to this embodiment, the contribution factor of an image color component of better quality is obtained by using the membership functions, such that performance of the image search is enhanced.
Furthermore, according to the embodiments of the invention the quality of search using color information is improved when an image is searched from an image database.
As many apparently widely different embodiments of the present invention can be made without departing from the spirit and scope thereof, it is to be understood that the invention is not limited to the specific embodiments thereof except as defined in the appended claims.

Claims (22)

What is claimed is:
1. A computer-implemented image searching method of searching for a desired image in an image database which stores a plurality of images each having predetermined color information signals containing a plurality of color components, by using an input color information signal containing a plurality of color components, comprising:
an image searching computer-implemented first step of calculating a contribution factor of each color component of color information on the basis of the input color information signals;
an image searching computer-implemented second step of calculating a similarity between the input color information signal and the plurality of predetermined color information signals based on said contribution factor of each color component of the color information signals, as calculated at said image searching computer-implemented first step; and
an image searching computer-implemented third step of sorting said plurality of predetermined color information signals based on an order of similarity as calculated at said image searching computer-implemented second step, and of displaying the sorted predetermined color information signals on a display means; and
an image searching computer-implemental fourth step of selecting one of the displayed sorted predetermined color information signals as the desired image.
2. The image searching method according to claim 1, wherein said each color component of color information is hue, chromaticity, and luminance.
3. The image searching method according to claim 1, wherein, at said image searching computer-implemented first step, the contribution factor of each color component of color information is calculated based on the comparison between each color component of the input color information and a predetermined threshold for each color.
4. The image searching method according to claim 2, wherein at said image searching computer-implemented first step, if luminance of said input color information is lower than a predetermined threshold, then a contribution factor corresponding to the luminance is set to be high, and wherein if the luminance is higher and chromaticity of the input color information is higher than the predetermined thresholds, then a contribution factor corresponding to hue is set to be high, further wherein if the luminance of said input color information is higher and the chromaticity of said input color information is lower than the predetermined thresholds, then the contribution factor of the chromaticity of the input color information and the contribution factor of the luminance of the input color information are set to be high.
5. The image searching method according to claim 2, wherein, at said image searching computer-implemented first step, the calculation is performed based on a plurality of membership functions which show membership grades of chromaticity and luminance of the input color information.
6. The image searching method according to claim 1, wherein the input color information is generated at an image searching computer-implemented fourth step of generating color information.
7. A computer-implemented image searching method of searching for a desired image in an image database which stores a plurality of images by using color information signals, comprising:
an image searching computer-implemented first step of generating a sample image including color information signals;
an image searching computer-implemented second step of calculating contribution factors of color information based on the color information signals of the sample image generated at said image searching computer-implemented first step;
an image searching computer-implemented third step of calculating similarity between the sample image and the plurality of images stored in said image database, based on the contribution factors calculated at said image searching computer-implemented second step;
an image searching computer-implemented fourth step of displaying the plurality of images from said image database on a display means in an order of the similarity; and
an image searching computer-implemented fourth step of selecting one of the displayed images as the desired image.
8. The image searching method according to claim 7, wherein color components of said color information are hue, chromaticity, and luminance.
9. The image searching method according to claim 8, wherein, at said image searching computer-implemented second step, the contribution factors corresponding to said color components of the color information of the sample image are calculated based on a comparison between said color components of the color information of the sample image and predetermined thresholds.
10. The image searching method according to claim 8, wherein, at said image searching computer-implemented second step,
if luminance of the color information is lower than a predetermined threshold, then the luminance is set to be high,
and wherein if the luminance is higher and chromaticity of the color information is higher than the predetermined thresholds, then the contribution factor corresponding to hue of the color information is set to be high,
further wherein if the luminance of said color information is higher and the chromaticity of the color information is lower than the predetermined thresholds, then the contribution factor of the chromaticity of the color information and the contribution factor of the luminance of the color information are set to be high.
11. The image searching method according to claim 8, wherein, at said image searching computer-implemented second step, the calculation is performed based on a plurality of membership functions which show membership grades of chromaticity and luminance of the color information.
12. An image searching apparatus for searching for a desired image in an image database which stores a plurality of images each having predetermined signals containing a plurality of color components by using an input color information signal containing a plurality of color components, comprising:
contribution factor calculation means for calculating a contribution factor of each color component of color information on the basis of the input color information signal;
similarity calculations means for calculating similarity between the input color information signal and the plurality of predetermined color information signals based on said contribution factor of each color component of the color information signals as calculated by the contribution factor calculation means; and
displaying means for sorting said plurality of predetermined color information signals based on an order of similarity as calculated by said similarity calculation means, and of displaying the sorted predetermined color information on a display means; and
selection means for selecting one of the displayed sorted predetermined color information signals as the desired image.
13. The image searching apparatus according to claim 12, wherein said each color component of color information is hue, chromaticity, and luminance.
14. The image searching method according to claim 12, wherein said contribution factor calculation means calculates the contribution factor corresponding to said each color component of color information based on the comparison between said each color component of color information and a predetermined threshold for each color.
15. The image searching apparatus according to claim 13, wherein
said contribution factor calculation means sets a contribution factor corresponding to the luminance to be high, if luminance of said input color information is lower than a predetermined threshold,
and wherein a contribution factor corresponding to hue is set to be high, if the luminance is higher and chromaticity of the input color information is higher than the predetermined thresholds,
further wherein the contribution factor of the chromaticity of the input color information and the contribution factor of the luminance of the input color information is set to be high, if the luminance of said input color information is higher and the chromaticity of said input color information is lower than the predetermined thresholds.
16. The image searching apparatus according to claim 13, wherein said contribution factor calculation method calculates based on a plurality of membership functions which show membership grades of chromaticity and luminance of the input color information.
17. The image searching apparatus according to claim 12, wherein the input color information is generated by a color information generating means.
18. An image searching apparatus for searching for a desired image in an image database which stores a plurality of images by using color information, comprising:
generating means for generating a sample image including color information signal;
contribution factor calculation means for calculating contribution factors of color information based on the color information signals of the sample image generated by said generating means;
similarity calculation means for calculating similarity between the sample image generated by said generating means and the plurality of images stored in said image database, based on the contribution factors;
displaying means for displaying the plurality of images from said image database on a display means in an order of the similarity; and
selection means for selecting one of the displayed images as the desired image.
19. The image searching apparatus according to claim 18, wherein color components of said color information are hue, chromaticity, and luminance.
20. The image searching apparatus according to claim 18, wherein said contribution factor calculation means calculates the contribution factors corresponding to said color components of the color information of the sample image based on a comparison between said color components of the color information of the sample image and predetermined thresholds.
21. The image searching apparatus according to claim 19, wherein said contribution factor calculation means sets the luminance to the high, if luminance of the color information is lower than a predetermined threshold,
and wherein the contribution factor corresponding to hue of the color information is set to be high, if the luminance is higher and chromaticity of the color information is higher than the predetermined thresholds,
further wherein the contribution factor of the chromaticity of the color information and the contribution factor of the luminance of the color information is set to be high, if the luminance of said color information is higher and the chromaticity of the color information is lower than the predetermined thresholds.
22. The image searching apparatus according to claim 19, wherein said contribution factor calculation means calculates based on a plurality of membership functions which show membership grades of chromaticity and luminance of the color information.
US08/300,488 1993-09-02 1994-09-02 Image searching method and apparatus thereof using color information of an input image Expired - Lifetime US5586197A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP5-218462 1993-09-02
JP21846293A JP3234064B2 (en) 1993-09-02 1993-09-02 Image retrieval method and apparatus

Publications (1)

Publication Number Publication Date
US5586197A true US5586197A (en) 1996-12-17

Family

ID=16720288

Family Applications (1)

Application Number Title Priority Date Filing Date
US08/300,488 Expired - Lifetime US5586197A (en) 1993-09-02 1994-09-02 Image searching method and apparatus thereof using color information of an input image

Country Status (4)

Country Link
US (1) US5586197A (en)
EP (1) EP0643358B1 (en)
JP (1) JP3234064B2 (en)
DE (1) DE69433373T2 (en)

Cited By (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5644765A (en) * 1993-12-09 1997-07-01 Canon Kabushiki Kaisha Image retrieving method and apparatus that calculates characteristic amounts of data correlated with and identifying an image
US5652881A (en) * 1993-11-24 1997-07-29 Hitachi, Ltd. Still picture search/retrieval method carried out on the basis of color information and system for carrying out the same
WO1997048068A1 (en) * 1996-06-14 1997-12-18 Sarnoff Corporation Method for indexing image information using a reference model
US5751286A (en) * 1992-11-09 1998-05-12 International Business Machines Corporation Image query system and method
US5950189A (en) * 1997-01-02 1999-09-07 At&T Corp Retrieval system and method
US6026411A (en) * 1997-11-06 2000-02-15 International Business Machines Corporation Method, apparatus, and computer program product for generating an image index and for internet searching and querying by image colors
US6130962A (en) * 1997-06-06 2000-10-10 Matsushita Electric Industrial Co., Ltd. Information retrieval apparatus for enabling information retrieval with ambiguous retrieval key
US6181818B1 (en) 1994-11-15 2001-01-30 Canon Kabushiki Kaisha Image retrieval method and apparatus
US6246790B1 (en) 1997-12-29 2001-06-12 Cornell Research Foundation, Inc. Image indexing using color correlograms
US6256409B1 (en) 1998-10-19 2001-07-03 Sony Corporation Method for determining a correlation between images using multi-element image descriptors
US6332037B1 (en) 1999-04-09 2001-12-18 Board Of Regents Of The University Of Nebraska Invariant, Eigenvalue based, non-degenerate data structure characterization, storage and retrieval indexing method
US6393147B2 (en) * 1998-04-13 2002-05-21 Intel Corporation Color region based recognition of unidentified objects
US20020102020A1 (en) * 1999-02-01 2002-08-01 Richard Qian Method for image characterization using color and texture statistics with embedded spatial information
US20020106122A1 (en) * 2000-12-01 2002-08-08 Dean Messing Method for image description using color and local spatial information
US6445834B1 (en) 1998-10-19 2002-09-03 Sony Corporation Modular image query system
US6445835B1 (en) 1998-10-29 2002-09-03 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US6480624B1 (en) * 1997-09-30 2002-11-12 Minolta Co., Ltd. Color discrimination apparatus and method
US6519360B1 (en) * 1997-09-17 2003-02-11 Minolta Co., Ltd. Image processing apparatus for comparing images based on color feature information and computer program product in a memory
US6535652B2 (en) * 1998-02-05 2003-03-18 Canon Kabushiki Kaisha Image retrieval apparatus and method, and computer-readable memory therefor
US20030086627A1 (en) * 2001-10-10 2003-05-08 Berriss William P. Method and apparatus for searching for and retrieving colour images
US20030103673A1 (en) * 2001-11-30 2003-06-05 Nainesh Rathod Shape searcher
US6611609B1 (en) 1999-04-09 2003-08-26 The Board Of Regents Of The University Of Nebraska Method of tracking changes in a multi-dimensional data structure
US6625311B1 (en) 1999-04-09 2003-09-23 The Board Of Regents Of The University Of Nebraska Methodology for data structure characterization, indexing, storage and retrieval
US20030198380A1 (en) * 1999-05-17 2003-10-23 Samsung Electronics Co., Ltd.., University Of , Calif. At Santa Barbara Color image processing method
US20030231806A1 (en) * 1999-07-30 2003-12-18 Vlad Troyanker Perceptual similarity image retrieval
US6674915B1 (en) 1999-10-07 2004-01-06 Sony Corporation Descriptors adjustment when using steerable pyramid to extract features for content based search
US20040073543A1 (en) * 2002-10-14 2004-04-15 Samsung Electronics Co., Ltd. Image retrieval method and apparatus using iterative matching
US6731789B1 (en) * 1999-01-29 2004-05-04 Canon Kabushiki Kaisha Image processing apparatus and method, and storage medium
US20040190793A1 (en) * 2002-03-01 2004-09-30 Rorvig Mark E. Method for retrieving images by content measure metadata encoding
US6807298B1 (en) 1999-03-12 2004-10-19 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US20050108099A1 (en) * 2001-06-29 2005-05-19 Volkswagen Aktiengesellschaft Manufacturing device
US6941323B1 (en) 1999-08-09 2005-09-06 Almen Laboratories, Inc. System and method for image comparison and retrieval by enhancing, defining, and parameterizing objects in images
US20050220338A1 (en) * 2002-01-31 2005-10-06 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US7015931B1 (en) 1999-04-29 2006-03-21 Mitsubishi Denki Kabushiki Kaisha Method and apparatus for representing and searching for color images
US20060137363A1 (en) * 2004-12-24 2006-06-29 Oxford Instruments Superconductivity Limited Cryostat assembly
CN100462978C (en) * 2007-04-18 2009-02-18 北京北大方正电子有限公司 Image searching method and system
US20090106192A1 (en) * 2001-02-09 2009-04-23 Harris Scott C Visual database for online transactions
US20090259567A1 (en) * 2008-04-11 2009-10-15 Kimberly Watts System and Apparatus for Creating Customized Artwork at Point-of-Purchase
US7961938B1 (en) * 2006-06-30 2011-06-14 Adobe Systems Incorporated Finding and structuring images based on a color search
US20110184950A1 (en) * 2010-01-26 2011-07-28 Xerox Corporation System for creative image navigation and exploration
US20120057032A1 (en) * 2010-09-03 2012-03-08 Pantech Co., Ltd. Apparatus and method for providing augmented reality using object list
US20120121193A1 (en) * 1997-10-27 2012-05-17 Massachusetts Institute Of Technology Image comparison and processing
US8532377B2 (en) 2010-12-22 2013-09-10 Xerox Corporation Image ranking based on abstract concepts
US20140074453A1 (en) * 2012-09-13 2014-03-13 International Business Machines Corporation Computer implemented method, program, and system for identifying non-text element suitable for communication in multi-language environment
US8732739B2 (en) 2011-07-18 2014-05-20 Viggle Inc. System and method for tracking and rewarding media and entertainment usage including substantially real time rewards
US20140355874A1 (en) * 2012-01-30 2014-12-04 Rakuten, Inc. Image processing system, image processing device, image processing method, program, and information storage medium
US9020415B2 (en) 2010-05-04 2015-04-28 Project Oda, Inc. Bonus and experience enhancement system for receivers of broadcast media
US9235777B2 (en) 2013-08-12 2016-01-12 Apollo Japan Co., Ltd. Code conversion device for image information, a code conversion method for the image information, a system for providing image related information using an image, and a non-transitory computer readable medium for the image information
US20160092445A1 (en) * 2013-05-16 2016-03-31 Beijing Jingdong Shangke Information Technology Co., Ltd. Image provision method, server device, and terminal device
US9977990B1 (en) * 2017-01-08 2018-05-22 International Business Machines Corporation Cognitive method for visual classification of very similar planar objects
CN113440843A (en) * 2021-06-25 2021-09-28 咪咕互动娱乐有限公司 Cloud game starting control method and device, cloud server and terminal equipment
US20220187809A1 (en) * 2019-03-26 2022-06-16 Jfe Steel Corporation Process state monitoring device and process state monitoring method

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0916618A (en) * 1995-06-30 1997-01-17 Fujitsu Ltd Data base system
US5933823A (en) * 1996-03-01 1999-08-03 Ricoh Company Limited Image database browsing and query using texture analysis
US5870754A (en) * 1996-04-25 1999-02-09 Philips Electronics North America Corporation Video retrieval of MPEG compressed sequences using DC and motion signatures
US6400853B1 (en) 1997-03-19 2002-06-04 Canon Kabushiki Kaisha Image retrieval apparatus and method
US6345274B1 (en) * 1998-06-29 2002-02-05 Eastman Kodak Company Method and computer program product for subjective image content similarity-based retrieval
JP2000207405A (en) * 1999-01-12 2000-07-28 Dainippon Screen Mfg Co Ltd Method for classifying plural images, classifying device and recording medium
JP2001022932A (en) * 1999-07-07 2001-01-26 Shijin Kogyo Sakushinkai Automatic image comparing method
JP3528172B2 (en) * 1999-12-14 2004-05-17 日本電気株式会社 Image search system using draw tool on WWW
JP4063486B2 (en) * 2000-09-06 2008-03-19 日本電気株式会社 Image search apparatus and method, and storage medium storing image search program
JP3658761B2 (en) * 2000-12-12 2005-06-08 日本電気株式会社 Image search system, image search method, and storage medium storing image search program
KR100499130B1 (en) 2001-11-27 2005-07-04 삼성전자주식회사 Image retrieval method and apparatus independent of the illumination change
JP2003242162A (en) * 2002-02-18 2003-08-29 Nec Soft Ltd Feature point extracting method, image searching method, feature point extracting apparatus, image searching system, and program
US7466856B2 (en) 2002-09-26 2008-12-16 Samsung Electronics Co., Ltd. Image retrieval method and apparatus independent of illumination change
JP4405831B2 (en) 2003-05-20 2010-01-27 キヤノン株式会社 Image processing apparatus, control method therefor, and program
JP2007241370A (en) * 2006-03-06 2007-09-20 Nikon Corp Portable device and imaging device
CN105224619B (en) * 2015-09-18 2018-06-05 中国科学院计算技术研究所 A kind of spatial relationship matching process and system suitable for video/image local feature
WO2022249491A1 (en) * 2021-05-28 2022-12-01 日本電気株式会社 Color assessment device, color assessment method, and non-transitory computer-readable medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4685068A (en) * 1985-08-20 1987-08-04 The Singer Company Generic database generator system and method
JPS62279468A (en) * 1986-05-29 1987-12-04 Matsushita Electric Ind Co Ltd Document image electronic filing device
US4931929A (en) * 1985-01-22 1990-06-05 Search & Source, Incorporated Design component selection computer with specification of product characteristics and of color by machine readable device
JPH05158999A (en) * 1991-12-02 1993-06-25 Mitsubishi Electric Corp Image retrieval system
EP0558054A2 (en) * 1992-02-28 1993-09-01 Mitsubishi Denki Kabushiki Kaisha Image filing method
JPH05274372A (en) * 1992-03-25 1993-10-22 Mitsubishi Electric Corp Automatic feature color adding device for image
JPH05290094A (en) * 1992-04-08 1993-11-05 Mitsubishi Electric Corp Retrieval device for similar tone of picture
US5335084A (en) * 1991-06-03 1994-08-02 Hitachi, Ltd. Document image filing system having input color mode selecting capability

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4931929A (en) * 1985-01-22 1990-06-05 Search & Source, Incorporated Design component selection computer with specification of product characteristics and of color by machine readable device
US4685068A (en) * 1985-08-20 1987-08-04 The Singer Company Generic database generator system and method
JPS62279468A (en) * 1986-05-29 1987-12-04 Matsushita Electric Ind Co Ltd Document image electronic filing device
US5335084A (en) * 1991-06-03 1994-08-02 Hitachi, Ltd. Document image filing system having input color mode selecting capability
JPH05158999A (en) * 1991-12-02 1993-06-25 Mitsubishi Electric Corp Image retrieval system
EP0558054A2 (en) * 1992-02-28 1993-09-01 Mitsubishi Denki Kabushiki Kaisha Image filing method
JPH05274372A (en) * 1992-03-25 1993-10-22 Mitsubishi Electric Corp Automatic feature color adding device for image
JPH05290094A (en) * 1992-04-08 1993-11-05 Mitsubishi Electric Corp Retrieval device for similar tone of picture

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Fundamentals of Interactive Computer Graphics, James D. Foley and Andries Van Dam, Intensity and Color, "Color Models For Raster Graphics", pp. 612-620, Addison-Wesley Publishing Company, 1982.
Fundamentals of Interactive Computer Graphics, James D. Foley and Andries Van Dam, Intensity and Color, Color Models For Raster Graphics , pp. 612 620, Addison Wesley Publishing Company, 1982. *

Cited By (83)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5751286A (en) * 1992-11-09 1998-05-12 International Business Machines Corporation Image query system and method
US5652881A (en) * 1993-11-24 1997-07-29 Hitachi, Ltd. Still picture search/retrieval method carried out on the basis of color information and system for carrying out the same
US5644765A (en) * 1993-12-09 1997-07-01 Canon Kabushiki Kaisha Image retrieving method and apparatus that calculates characteristic amounts of data correlated with and identifying an image
US6181818B1 (en) 1994-11-15 2001-01-30 Canon Kabushiki Kaisha Image retrieval method and apparatus
US6246804B1 (en) * 1994-11-15 2001-06-12 Canon Kabushiki Kaisha Image retrieval method and apparatus using a compound image formed from a plurality of detected regions
WO1997048068A1 (en) * 1996-06-14 1997-12-18 Sarnoff Corporation Method for indexing image information using a reference model
US5768447A (en) * 1996-06-14 1998-06-16 David Sarnoff Research Center, Inc. Method for indexing image information using a reference model
US5950189A (en) * 1997-01-02 1999-09-07 At&T Corp Retrieval system and method
US6130962A (en) * 1997-06-06 2000-10-10 Matsushita Electric Industrial Co., Ltd. Information retrieval apparatus for enabling information retrieval with ambiguous retrieval key
US6519360B1 (en) * 1997-09-17 2003-02-11 Minolta Co., Ltd. Image processing apparatus for comparing images based on color feature information and computer program product in a memory
US6480624B1 (en) * 1997-09-30 2002-11-12 Minolta Co., Ltd. Color discrimination apparatus and method
US20120121193A1 (en) * 1997-10-27 2012-05-17 Massachusetts Institute Of Technology Image comparison and processing
US6026411A (en) * 1997-11-06 2000-02-15 International Business Machines Corporation Method, apparatus, and computer program product for generating an image index and for internet searching and querying by image colors
US6430312B1 (en) 1997-12-29 2002-08-06 Cornell Research Foundation, Inc. Image subregion querying using color correlograms
US6246790B1 (en) 1997-12-29 2001-06-12 Cornell Research Foundation, Inc. Image indexing using color correlograms
US6535652B2 (en) * 1998-02-05 2003-03-18 Canon Kabushiki Kaisha Image retrieval apparatus and method, and computer-readable memory therefor
US6393147B2 (en) * 1998-04-13 2002-05-21 Intel Corporation Color region based recognition of unidentified objects
US6687416B2 (en) 1998-10-19 2004-02-03 Sony Corporation Method for determining a correlation between images using multi-element image descriptors
US6445834B1 (en) 1998-10-19 2002-09-03 Sony Corporation Modular image query system
US6256409B1 (en) 1998-10-19 2001-07-03 Sony Corporation Method for determining a correlation between images using multi-element image descriptors
US6445835B1 (en) 1998-10-29 2002-09-03 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US6731789B1 (en) * 1999-01-29 2004-05-04 Canon Kabushiki Kaisha Image processing apparatus and method, and storage medium
US6542632B1 (en) 1999-02-01 2003-04-01 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US6965706B2 (en) 1999-02-01 2005-11-15 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US20020102020A1 (en) * 1999-02-01 2002-08-01 Richard Qian Method for image characterization using color and texture statistics with embedded spatial information
US7697754B2 (en) 1999-03-12 2010-04-13 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US7106900B2 (en) 1999-03-12 2006-09-12 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US20060147112A1 (en) * 1999-03-12 2006-07-06 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US6807298B1 (en) 1999-03-12 2004-10-19 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US20040240734A1 (en) * 1999-03-12 2004-12-02 Electronics And Telecommunications Research Institute Method for generating a block-based image histogram
US6611609B1 (en) 1999-04-09 2003-08-26 The Board Of Regents Of The University Of Nebraska Method of tracking changes in a multi-dimensional data structure
US6625311B1 (en) 1999-04-09 2003-09-23 The Board Of Regents Of The University Of Nebraska Methodology for data structure characterization, indexing, storage and retrieval
US6332037B1 (en) 1999-04-09 2001-12-18 Board Of Regents Of The University Of Nebraska Invariant, Eigenvalue based, non-degenerate data structure characterization, storage and retrieval indexing method
US7015931B1 (en) 1999-04-29 2006-03-21 Mitsubishi Denki Kabushiki Kaisha Method and apparatus for representing and searching for color images
CN1322457C (en) * 1999-04-29 2007-06-20 三菱电机株式会社 Method and apparatus for representing and searching for colour images
US7636094B2 (en) 1999-04-29 2009-12-22 Mitsubishi Denki Kabushiki Kaisha Method and apparatus for representing and searching for colour images
US20030198380A1 (en) * 1999-05-17 2003-10-23 Samsung Electronics Co., Ltd.., University Of , Calif. At Santa Barbara Color image processing method
US7245762B2 (en) * 1999-05-17 2007-07-17 Samsung Electronics Co., Ltd. Color image processing method
US7212667B1 (en) * 1999-05-17 2007-05-01 Samsung Electronics Co., Ltd. Color image processing method for indexing an image using a lattice structure
US7031555B2 (en) * 1999-07-30 2006-04-18 Pixlogic Llc Perceptual similarity image retrieval
US20030231806A1 (en) * 1999-07-30 2003-12-18 Vlad Troyanker Perceptual similarity image retrieval
US8775451B2 (en) 1999-08-09 2014-07-08 Almen Laboratories, Inc. Object based image retrieval
US7483919B2 (en) 1999-08-09 2009-01-27 Almen Laboratories Object based image retrieval
US6941323B1 (en) 1999-08-09 2005-09-06 Almen Laboratories, Inc. System and method for image comparison and retrieval by enhancing, defining, and parameterizing objects in images
US6674915B1 (en) 1999-10-07 2004-01-06 Sony Corporation Descriptors adjustment when using steerable pyramid to extract features for content based search
US20040234159A1 (en) * 1999-10-07 2004-11-25 Lizhi Wang Descriptors adjustment when using steerable pyramid to extract features for content based search
US7062084B2 (en) 2000-12-01 2006-06-13 Sharp Laboratories Of America, Inc. Method for image description using color and local spatial information
US20020106122A1 (en) * 2000-12-01 2002-08-08 Dean Messing Method for image description using color and local spatial information
US20090106192A1 (en) * 2001-02-09 2009-04-23 Harris Scott C Visual database for online transactions
US20050108099A1 (en) * 2001-06-29 2005-05-19 Volkswagen Aktiengesellschaft Manufacturing device
US7409359B2 (en) * 2001-06-29 2008-08-05 Volkswagen Aktiengesellschaft Configurator for configuring and displaying the appearance of an article to be configured
US20030086627A1 (en) * 2001-10-10 2003-05-08 Berriss William P. Method and apparatus for searching for and retrieving colour images
US20070122031A1 (en) * 2001-10-10 2007-05-31 Mitsubishi Denki Kabushiki Kaisha Method and apparatus for searching for and retrieving colour images
US20030103673A1 (en) * 2001-11-30 2003-06-05 Nainesh Rathod Shape searcher
US7440611B2 (en) 2002-01-31 2008-10-21 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US20050220338A1 (en) * 2002-01-31 2005-10-06 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US20040190793A1 (en) * 2002-03-01 2004-09-30 Rorvig Mark E. Method for retrieving images by content measure metadata encoding
US7298931B2 (en) * 2002-10-14 2007-11-20 Samsung Electronics Co., Ltd. Image retrieval method and apparatus using iterative matching
US20040073543A1 (en) * 2002-10-14 2004-04-15 Samsung Electronics Co., Ltd. Image retrieval method and apparatus using iterative matching
US20060137363A1 (en) * 2004-12-24 2006-06-29 Oxford Instruments Superconductivity Limited Cryostat assembly
US8699826B2 (en) 2006-06-30 2014-04-15 Adobe Systems Incorporated Finding and structuring images based on a color search
US7961938B1 (en) * 2006-06-30 2011-06-14 Adobe Systems Incorporated Finding and structuring images based on a color search
US20110170770A1 (en) * 2006-06-30 2011-07-14 Adobe Systems Incorporated Finding and structuring images based on a color search
CN100462978C (en) * 2007-04-18 2009-02-18 北京北大方正电子有限公司 Image searching method and system
US20090259567A1 (en) * 2008-04-11 2009-10-15 Kimberly Watts System and Apparatus for Creating Customized Artwork at Point-of-Purchase
US20110184950A1 (en) * 2010-01-26 2011-07-28 Xerox Corporation System for creative image navigation and exploration
US8775424B2 (en) 2010-01-26 2014-07-08 Xerox Corporation System for creative image navigation and exploration
US9026034B2 (en) 2010-05-04 2015-05-05 Project Oda, Inc. Automatic detection of broadcast programming
US9020415B2 (en) 2010-05-04 2015-04-28 Project Oda, Inc. Bonus and experience enhancement system for receivers of broadcast media
US20120057032A1 (en) * 2010-09-03 2012-03-08 Pantech Co., Ltd. Apparatus and method for providing augmented reality using object list
US8532377B2 (en) 2010-12-22 2013-09-10 Xerox Corporation Image ranking based on abstract concepts
US8732739B2 (en) 2011-07-18 2014-05-20 Viggle Inc. System and method for tracking and rewarding media and entertainment usage including substantially real time rewards
US20140355874A1 (en) * 2012-01-30 2014-12-04 Rakuten, Inc. Image processing system, image processing device, image processing method, program, and information storage medium
US9367764B2 (en) * 2012-01-30 2016-06-14 Rakuten, Inc. Image processing system, image processing device, image processing method, program, and information storage medium for providing an aid that makes it easy to grasp color of an image
US20140074453A1 (en) * 2012-09-13 2014-03-13 International Business Machines Corporation Computer implemented method, program, and system for identifying non-text element suitable for communication in multi-language environment
US9514127B2 (en) * 2012-09-13 2016-12-06 International Business Machines Corporation Computer implemented method, program, and system for identifying non-text element suitable for communication in multi-language environment
US20160092445A1 (en) * 2013-05-16 2016-03-31 Beijing Jingdong Shangke Information Technology Co., Ltd. Image provision method, server device, and terminal device
US9734164B2 (en) * 2013-05-16 2017-08-15 Beijing Jingdong Shangke Information Technology Co, Ltd. Method, server, device, and terminal device for providing an image search
US9235777B2 (en) 2013-08-12 2016-01-12 Apollo Japan Co., Ltd. Code conversion device for image information, a code conversion method for the image information, a system for providing image related information using an image, and a non-transitory computer readable medium for the image information
US9977990B1 (en) * 2017-01-08 2018-05-22 International Business Machines Corporation Cognitive method for visual classification of very similar planar objects
US20220187809A1 (en) * 2019-03-26 2022-06-16 Jfe Steel Corporation Process state monitoring device and process state monitoring method
CN113440843A (en) * 2021-06-25 2021-09-28 咪咕互动娱乐有限公司 Cloud game starting control method and device, cloud server and terminal equipment
CN113440843B (en) * 2021-06-25 2023-12-08 咪咕互动娱乐有限公司 Cloud game starting control method and device, cloud server and terminal equipment

Also Published As

Publication number Publication date
JPH0773195A (en) 1995-03-17
EP0643358A3 (en) 1995-06-07
DE69433373D1 (en) 2004-01-15
DE69433373T2 (en) 2004-10-14
JP3234064B2 (en) 2001-12-04
EP0643358A2 (en) 1995-03-15
EP0643358B1 (en) 2003-12-03

Similar Documents

Publication Publication Date Title
US5586197A (en) Image searching method and apparatus thereof using color information of an input image
CN100446542C (en) Systems and methods for organizing image data into regions
KR100649338B1 (en) Color quantization and similarity measure for content based image retrieval
JP3607059B2 (en) Color feature extraction apparatus, color feature extraction method, and program storage medium
US8594420B2 (en) Color naming, color categorization and describing color composition of images
US5781899A (en) Image index production method and image index production system for image storage and management system
US20100271513A1 (en) Selection of decorative picture suitable for input picture
KR20060080586A (en) Method and apparatus for indexing and searching graphic elements
US5742520A (en) Color picture processing method and color picture processing apparatus
JPH09163161A (en) Picture processor
US20040013297A1 (en) Method for performing color gamut compression
US20050125368A1 (en) Apparatus and program for learning data and apparatus for deciding meaning of image
US6836563B2 (en) Computer-readable medium and program for quantizing a data set, method and apparatus for quantizing a data set
CN101925921A (en) Image signal processing device and image signal processing program
JP2004192555A (en) Information management method, device and program
JPH07160725A (en) Picture retrieval device
Karatzas et al. Two approaches for text segmentation in web images
US10084941B2 (en) Generating color similarity measures
US20150026013A1 (en) System and methods for cognitive visual product search
JPH0822532A (en) Color picture processor for subtractive color processing
JP4974303B2 (en) Color conversion information generation apparatus, color conversion information generation method and program
JP5104528B2 (en) Image processing apparatus and image processing program
JP6584573B2 (en) Color profile providing apparatus and image forming apparatus
KR100357055B1 (en) method of color expression considering background contrast and device of color feature extraction
JPH06274618A (en) Area identifying device

Legal Events

Date Code Title Description
AS Assignment

Owner name: CANON KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TSUJIMURA, KAZUKO;BANNAI, YUICHI;REEL/FRAME:007140/0315

Effective date: 19940830

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
FEPP Fee payment procedure

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: PETITION RELATED TO MAINTENANCE FEES FILED (ORIGINAL EVENT CODE: PMFP); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

REMI Maintenance fee reminder mailed
FEPP Fee payment procedure

Free format text: PETITION RELATED TO MAINTENANCE FEES GRANTED (ORIGINAL EVENT CODE: PMFG); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 8

SULP Surcharge for late payment
PRDP Patent reinstated due to the acceptance of a late maintenance fee

Effective date: 20050401

FPAY Fee payment

Year of fee payment: 12