US20080309963A1 - Print image matching parameter extraction and rendering on display devices - Google Patents
Print image matching parameter extraction and rendering on display devices Download PDFInfo
- Publication number
- US20080309963A1 US20080309963A1 US11/761,981 US76198107A US2008309963A1 US 20080309963 A1 US20080309963 A1 US 20080309963A1 US 76198107 A US76198107 A US 76198107A US 2008309963 A1 US2008309963 A1 US 2008309963A1
- Authority
- US
- United States
- Prior art keywords
- avg
- pim
- image
- setting
- saturation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/46—Colour picture communication systems
- H04N1/56—Processing of colour picture signals
- H04N1/60—Colour correction or control
- H04N1/6027—Correction or control of colour gradation or colour contrast
Definitions
- the present disclosure relates generally to imaging rendering, and more specifically to techniques for automatically extracting print image matching (PIM) parameters that can be automatically adjusted to produce a better looking (visually enhanced) image.
- PIM print image matching
- the gamut curve for a typical color gamut is denoted as 2 and has a generally horseshoe shape.
- the horseshoe shaped gamut curve 2 represents the entire range of possible colors.
- Overlaid in the horseshoe shaped gamut curve 2 is the gamut curve denoted as 4 for an image taken by a digital still camera image.
- the gamut curve denoted as 6 A is for a standard red, green blue (sRGB) display device, such as a CRT or other typical computer monitor, and has a generally triangular shape. The corners of the triangle represent the primary colors red (R), green (G) and blue (B) of the gamut.
- a gamut curve denoted as 6 B is overlaid which is the gamut curve for an EPSON® 6-color inkjet printer.
- the color gamut is device-dependent.
- a digital still camera device may be able to capture more colors than what a sRGB display device can render.
- a multi-ink printing device has a wider color gamut than the sRGB display.
- PIM Print Image Matching
- EPSON® is technology introduced by EPSON® to allow a camera/user to specify settings that are later used by the printer to process an image at print time.
- the PIM technology enables conveying the image “as captured” to the printer along with control information used to instruct the printer to perform certain operations.
- the PIM technology creates information outside of the visible color space of a sRGB display that could be used to print on a printer's wider color space.
- the PIM technology also provides active control of image print quality by a camera's manufacturer or user.
- PIM technology allows a camera's subsystem to correct for light/color imbalances occurring at the time of taking a picture without processing the image; the parameters for processing the image are included in the EXIF header (PIM tag) and are later used by the printer to apply picture specific processing.
- the camera user can also potentially specify types of pictures (i.e., portrait, landscape scenery, etc.) which can be interpreted by the camera subsystem information PIM parameters.
- the PIM technology supports multiple parameters with often overlapping scope.
- An image with the PIM correction ON is corrected for light/color imbalances when rendered on or by a display device.
- the same image with the PIM correction OFF may appear lighter or darker due to light/color imbalances.
- the PIM parameters are used to make the image more balanced. For example if the image is too bright, the PIM parameters can make the image darker. However, if the image is too dark, the PIM parameters can make the image brighter.
- a system comprising an image capturing device operable to capture an original image and extract PIM parameter data automatically based on specifics of the original image.
- the system includes a rendering device which is PIM-enabled.
- the system has a processor operable to calculate automatically at least one automated PIM parameter from the PIM parameter data.
- the processor also inserts the at least one automated PIM parameter in PIM header information for communication to the rendering device to modify the original image when rendered.
- an apparatus which comprises a processor operable to calculate automatically at least one automated PIM parameter based on automatically extracted PIM parameter data based on specifics of an original image.
- the processor also inserts the at least one automated PIM parameter in PIM header information for communication to a PIM-enabled rendering device to modify the original image when rendered.
- the apparatus also includes memory coupled to the processor.
- a further aspect includes a wireless device comprising an image capturing module operable to capture an original image.
- the device also include a processor operable to automatically extract PIM parameter data automatically based on specifics of the original image and insert the PIM parameter data in a header slot of a header appended to the original image.
- the device creates an image data file with the header and the original image for communications.
- a communication module of the device communicates the image data file, the image data file being used to calculate automatically at least one automated PIM parameter.
- FIG. 1A shows gamut curves for a typical color gamut, a standard RGB display and a digital still camera.
- FIG. 1B shows gamut curves for a typical color gamut, an Epson 6-color inkjet printer and a digital still camera.
- FIG. 2 illustrates a general diagram of a system for automatic PIM parameter extraction and rendering on a display device wherein the image is captured by a image capturing device.
- FIG. 3 illustrates a general diagram of a system for automatic PIM parameter extraction and rendering on a display device wherein the image is captured by a camera wireless phone device with still imaging capturing or video capability.
- FIG. 4 illustrates a general block diagram of the automatic PIM parameter extraction and rendering.
- FIG. 5 illustrates a list of PIM parameters in the PIM parameters module.
- FIG. 6 illustrates the general flowchart for the PIM parameter setting method.
- FIG. 7 illustrates a plot or curve for contrast setting.
- FIGS. 8A-8B illustrates plots or curves for brightness setting.
- FIG. 9 illustrates a plot or curve for highlight point setting.
- FIG. 10 illustrates a flowchart of an automatic PIM parameter extraction method.
- FIG. 11 illustrates a block diagram of a PIM image equalization processing process in a rendering device.
- FIGS. 12A-12B illustrate a flowchart of the histogram equalization process.
- FIG. 13 illustrates a flowchart of the shadow and highlight point setting process.
- FIGS. 14A-14C illustrate a flowchart for a memory color saturation process.
- FIG. 15A illustrates a plot of an original image luma histogram.
- FIG. 15B illustrates a plot of a transformed image luma histogram.
- the system 10 includes an apparatus in the form of a computer 14 with a display 14 A, such as a CRT, LCD, etc., a processor 15 and memory 16 (shown in phantom), and keyboard 18 .
- a display 14 A such as a CRT, LCD, etc.
- processor 15 and memory 16 shown in phantom
- keyboard 18 In lieu of or in addition to the keyboard 18 , other data input and/or computer navigational devices such as a mouse, voice-responsive assemblies may be included.
- the computer 14 is coupled via a wire or wireless connection to an image capturing device 12 and a printing device 35 .
- the printing device 35 is a PIM-enabled printing device 35 .
- the memory 16 includes machine readable medium for storing program instructions therein.
- the apparatus is not limited to the computer 14 , but may be any other type of general purpose computing device.
- FIG. 3 illustrates a system, generally designated at 10 ′, for automatic PIM parameters extraction and rendering on a display device using a wireless camera phone device 12 ′.
- the system 10 ′ is essentially the same as system 10 except that in lieu of an image capturing device 12 , a wireless camera phone device 12 ′ is used.
- the PIM parameter extraction and rendering module 20 includes an automatic PIM extracting sub-module 22 A and an extracted PIM parameter header inserting sub-module 22 B.
- the PIM parameter extraction and rendering module 20 further includes a PIM parameter calculator 24 .
- the PIM parameter calculator 24 includes a histogram equalize image sub-module 24 A to correct for brightness, contrast and gamma value setting.
- the operation (process S 101 ) of the histogram equalize image sub-module 24 A is set forth in FIGS. 12A and 12B .
- the shadow and highlight point setting sub-module 24 B stretches the image.
- the operation (process S 141 ) of the shadow and highlight point setting sub-module 24 B is set forth in FIG. 13 .
- the memory color saturation sub-module 24 C saturates a plurality of colors.
- the operation (process S 161 ) of the memory color saturation sub-module 24 C is set forth in FIGS. 14A-14C .
- the program instructions of sub-modules 22 A and 22 B and the program-instructions of sub-modules 24 A- 24 C may reside on different machines.
- the automatic PIM extracting sub-module 22 A and an extracted PIM parameter header inserting sub-module 22 B may reside on the image capturing device 12 or wireless camera phone device 12 ′.
- the a histogram equalize image sub-module 24 A, the shadow and highlight point setting sub-module 24 B and the memory color saturation sub-module 24 C may reside on computer 14 .
- the entire PIM parameter extraction and rendering module 20 may reside on a single machine or computer 14 .
- Program instructions may be used to cause a general-purpose or special-purpose processing system that is programmed with the instructions to perform the methods described herein. Alternatively, the methods may be performed by specific hardware components that contain hardwired logic for performing the methods, or by any combination of programmed computer components and custom hardware components.
- the methods described herein may be provided as a computer program product that may include a machine readable medium having stored thereon instructions that may be used to program a processing system or other electronic device to perform the methods.
- the term “machine readable medium” or “machine accessible medium” used herein shall include any medium that is capable of storing or encoding a sequence of instructions for execution by the machine and that causes the machine to perform any one of the methods described herein.
- machine readable medium and “machine accessible medium” shall accordingly include, but not be limited to, solid-state memories, optical and magnetic disks, and a carrier wave that encodes a data signal.
- machine readable medium and “machine accessible medium” shall accordingly include, but not be limited to, solid-state memories, optical and magnetic disks, and a carrier wave that encodes a data signal.
- software in one form or another (e.g., program, procedure, process, application, module, logic, and so on) as taking an action or causing a result. Such expressions are merely a shorthand way of stating the execution of the software by a processing system to cause the processor to perform an action or produce a result.
- the PIM parameter extraction and rendering module 20 is operable to display on any of printer 35 , (computer) display 14 A, or display of image capturing device 12 .
- printer 35 (computer) display 14 A
- image capturing device 12 A general overview of the operation for system 10 will now be described.
- a user takes a picture or captures a still image with the image capturing device 12 .
- the user transfers, downloads or otherwise saves the picture or still image (hereinafter referred to as “image data” in memory or a machine readable medium.
- image data file will contain the actual picture or image captured plus some header information.
- the captured image may be encoded, and a header is appended thereto.
- the jpeg format has a header with a PIM parameters slot.
- the PIM parameters slot is empty.
- the still jpeg image and related header is modified to include the extracted PIM parameters.
- the automatic PIM extracting sub-module 22 A extracts data related to the PIM parameters while the extracted PIM parameter header inserting sub-module 22 B inserts the extracted PIM parameters in the header information to create the image data file.
- the extracted PIM parameters include the PIM parameter data necessary for the histogram equalize image sub-module 24 A, the shadow and highlight point setting sub-module 24 B and the memory color saturation sub-module 24 C to perform their operations by computer 14 .
- the PIM header to printer 35 is modified to include the adjusted parameters of the a histogram equalize image sub-module 24 A, the shadow and highlight point setting sub-module 24 B and the memory color saturation sub-module 24 C.
- the printer 35 receives the original image data with a set of PIM parameters in a PIM header.
- the original image data is not changed or modified in any way.
- the PIM parameters in the printer header are used by the PIM-enabled printer 35 to render the original image with enhanced color and lighting balance via the PIM header.
- the non-PIM-enabled printer prints only the original image without any modification.
- PIM parameters can be interpreted not only by printer 35 which produces a hardcopy display of the image, but by other display devices.
- Some special monitors may be a rendering device (which can be a printer or a monitor); display rendering is not actually seen until it is printed.
- the wireless camera phone device 12 ′ is operable to take or capture a picture or still image (hereinafter referred to as “image data”).
- image data a picture or still image
- the wireless camera phone device 12 ′ is also operable to send or transmit the image data.
- the wireless camera phone device 12 ′ will encode image data and append a jpeg header to form a jpeg image file.
- the jpeg image file can be transmitted to different devices via wireless communications.
- the wireless camera phone device 12 ′ transmits the jpeg image file to computer 14 .
- the wireless camera phone device 12 ′ includes the automatic PIM extracting sub-module 22 A and the extracted PIM parameter header inserting sub-module 22 B.
- the header is developed. For a jpeg picture, the header is a jpeg header, but in the jpeg header there is small space reserved for PIM parameters which in the regular mode would have been empty.
- the wireless camera phone device 12 ′ includes the automatic PIM parameter extracting sub-module 22 A and the extracted PIM parameter header inserting sub-module 22 B, then certain PIM parameter data based on the image specifics is extracted and inserted in the jpeg header.
- the image file can be sent to different places. If the image file is opened on a display 14 A, it is not going to be different than the original image, the PIM parameters are not seen by regular jpeg decoders. However, if original image is printed, that is, if the original image was sent to the printer 35 , usually the printer 35 will decode the image and it will print it out. If the printer does not support PIM, then the printer will just decode the mage and then print whatever is in the image (whatever was encoded) exactly as it was taken. If there is PIM information, and if the printer 35 is PIM-enabled, the printer 35 prints the image with the automatically calculated PIM parameters and other PIM parameters set by the user or device.
- the original image is not modified.
- the original image is process to automatically note (without user or device intervention) whether the original image is too dark. While keeping the original image and knowing that the image was dark, the automatically calculated PIM parameters advise the printer to do correct the image during rendering without changing the original image.
- the first element is a device that generates the original image or picture.
- the second element extracts the PIM parameter data and inserts them in the image header.
- the third element automatically calculates a set of PIM parameters automatically based on the extracted PIM parameter data and creates a PIM header which includes at least the set of PIM parameters calculated.
- the fourth element includes the rendering device.
- the wireless camera phone device 12 ′ includes both the first and second elements.
- the wireless camera phone device 12 ′ includes a module for capturing a still image and includes the automatic PIM extracting sub-module 22 A and the extracted PIM parameter header inserting sub-module 22 B.
- Other image capturing devices may not include both of the first and second elements.
- a parameter setting procedure 30 for system 10 begins with step S 32 where an image file is created by the image file creator module 16 .
- Examples of an image file include JPEG and TIFF formats. While only two examples of image files are provided, currently there are many other image files that can be used.
- the image file is created after the user shoots or captures the picture or scene with the image capturing device 12 .
- Step S 32 is followed by step S 34 where the PIM parameters are calculated according to the methods described in detail below via the PIM extraction module 20 .
- the PIM extraction module 20 includes the program code or instructions executable by processor 15 .
- Step S 34 is followed by step S 36 where the PIM parameters are set.
- Step S 36 is followed by step S 38 where the image is printed using the PIM-enabled printing device 35 with PIM support.
- the user can then visually inspect or perform a sensory evaluation to determine if any further adjustments are necessary using the display 14 A on the computer 14 .
- the computer 14 is shown as a personal computer having stored therein the computer program code or instructions for carrying out the automatic PIM parameter extraction method 100 .
- the computer 14 may be a Laptop, Notebook, Tablet or other computing device with printing capability and a port for connecting to the printing device 35 .
- the PIM parameters module 28 includes a gamma value setting, releasing RGB clipping, color space setting, shadow point setting, highlight point setting, contrast setting, brightness setting, RGB color balance setting, saturation setting, sharpness setting, memory color correction setting, HSB (hue, saturation, brightness) correction setting, tone curve setting and channel mix setting.
- FIG. 7 A plot or curve of the contrast setting is shown in FIG. 7 .
- the contrast setting plot is shown increasing at the point of the denoted circle.
- FIGS. 8A and 8B plots or curves of the brightness setting for the output level verses the input level are shown.
- the first plot FIG. 8A
- the brightness setting plot is shown increasing.
- FIG. 9 plots of the highlight point setting and shadow point setting are shown. The arrows at the top indicate the direction of the highlight setting while the bottom arrows indicate the direction of the shadow setting.
- the memory color correction setting provides RGB offsets for green, sky blue, flesh color and red.
- the automatic PIM parameter extraction method 100 first, applies or uses the gamma value, brightness, and contrast settings to histogram equalize the image (build histogram equalization function with the three curves) via a histogram equalization process S 101 .
- An exemplary histogram equalization process S 101 is shown in FIGS. 12A-12B .
- the method 100 uses a shadow and highlight point setting process S 141 to stretch the image to adjust the image in the range of 0-255.
- An exemplary shadow and highlight point setting process S 141 is shown in FIG. 13 .
- the method 100 saturates the memory color for a plurality of colors in a memory color saturation process S 161 . For example, all colors except the flesh color can be saturated in the memory color saturation process S 161 .
- An exemplary memory color saturation process S 161 is shown in FIGS. 14A-14C .
- the method 100 analyzes an image histogram and designs a transformation which will correct some image deficiencies detectable from the histogram, i.e., too bright or too dark image.
- the designed transformation is then to be implemented using a selected set of PIM tools (parameters).
- PIM tools Parameters
- Three PIM parameters gamma value, contrast, and brightness are used to construct the transformation. Those are specified in the PIM header as scalars but are expanded by the printing device 35 to full curves according to the PIM specification. Knowing the scalar-curve mapping allows for “constructing” those curves and using them to construct the desired transformation.
- Another parameter, sharpness was also considered. It was concluded that the way sharpness is being defined in the PIM specification (i.e., filtering operation triggered based on exceeding a threshold value) does not represent a reasonable addition to histogram processing nor it has use for edge enhancement.
- a PIM image equalization processing process 40 in a rendering device 35 such as a PIM-enabled printing device 35 .
- the process 40 begins with a transport modified input or a gamma corrected R′G′B′ image at step S 42 .
- Step S 42 is followed by step S 44 where the gamma having a gamma value ( ⁇ ) is removed according to equation Eq.(1)
- gamma correction is applied by a rendering device because the rendering device will have a certain response.
- the rendering device will have a modify the image.
- gamma correction which effectively removes that response or modification.
- After the gamma correction the image is back to its original state.
- the gamma correction transformation may be device specific.
- step S 44 is followed by step S 46 A where the first transformation (H 1 (x)) is calculated according to equation Eq.(2)
- Step S 46 A is followed by the intermediate steps for calculating subsequent transformations up to transform N (HN(x)) at step S 46 N according to equation Eq.(3)
- Step S 46 N is followed by step S 48 where the rendering device (printer 35 or a display monitor) applies the display gamma according to equation Eq.(4)
- the steps S 46 A-S 46 N relate to the PIM parameters.
- the transformations F 1 , F 2 , . . . FN relate to the PIM parameters.
- transformations F 1 , F 2 and F 3 can be for the highlight point setting, gamma correction and memory color settings, respectively. All other transformations F 4 -FN support additional PIM parameters.
- histogram equalization will produce an image that will exhibit the greatest dynamic range. However sometimes histogram equalization does not produce the desired contrast in the areas of interest (too dark image with small bright element or too bright image with a small dark element). In those cases histogram equalization is too aggressive (shift dark to grey in the first case and white to grey in the second case).
- CAP CAP
- x 0-255; and a is chosen to be 2/3.
- a luma histogram plot is shown on an image before and after the transformation for ⁇ equal to 2/3.
- the histogram equalization process S 101 begins at step S 102 where the histogram h(i) of L′ (gamma corrected luma) is computed.
- Step S 102 is followed by step S 104 where the cumulative density function H(i) is computed.
- step S 104 is followed by step S 106 where the cumulative density function H(i) is applied according to equation Eq.(7)
- Step S 106 is followed by step S 108 where the transform function (T(i)) is computed according to equation Eq.(6) rewritten (by substituting i or x) into equation (8)
- Step S 108 is followed by step S 110 where gamma ⁇ is found that minimizes equation Eq.(9)
- Step S 110 is followed by step S 112 where the difference between the transform function and the component to remove gamma is computed according to equation Eq.(10)
- i 0-255 and where i ⁇ is the means (raising to the power) to remove gamma.
- step S 112 is followed by step S 114 where the contrast C setting is computed according to equation Eq.(11)
- Step S 114 is followed by step S 116 where the brightness B setting is computed according to equation Eq.(12)
- Step S 116 is followed by step S 1 18 where the gamma correction value ( ⁇ ) is computed according to equation Eq.(13)
- Step S 118 is followed by step S 120 where the gamma correction value is inserted in the PIM header.
- RGB color balance settings are an operation that is already performed on the image (white balance) via a matrix multiplication and addition.
- the exact same degrees of freedom are supported in PIM header which makes this correction unnecessary.
- the shadow and highlight setting process S 141 is basically a linear point-to-point mapping and is required when the image capturing device 12 or the wireless camera phone device 12 ′ uses a RGB format that is other than [0:255]. Commonly, a wireless camera phone device may use a RGB format that is [19:238]. Other RGB formats may be used. Thus, the best use of the shadow and highlight setting process S 141 is “stretching” the captured image if it is other than [0:255].
- step S 141 the shadow and highlight setting process S 141 begins with step S 142 where the gamma corrected R′G′B′ format image is obtained having a range [P:Q].
- step S 142 is followed by step S 144 where the R′G′B′ format image is converted to a linear RGB formatted image such as in the range [P′:Q′] in accordance with equations Eq.(14A) and Eq. (14B).
- step S 144 is followed by step S 146 where the shadow point is set to P′.
- step S 148 the highlight point is set to 255-Q′.
- the image at step S 148 has been effectively stretched to the range of [0:255].
- the captured image or snapshot from wireless camera phone device 12 ′ has a 219 R′G′B′ format (e.g. the gamma corrected RGB values lie [19:238]).
- the range is [1:219]. Therefore, the image can be stretched to [0:255] by setting shadow point to 1 and highlight point to 36.
- the shadow and highlight setting process S 141 is only required for those devices which produce an image that requires stretching to [0:255].
- the memory color saturation process S 161 modifies colors that lie within a certain range to obtain more vivid color representation for colors that are remembered when looked at (such as foliage green, sky blue, skin, water etc.). According to some psychophysical experiments, it has been found that the general public likes green, red, and blue to be saturated and flesh color untouched (flesh color is dependent on cultural preferences and that is the reason it is not modified).
- the memory color saturation process S 161 allows the memory colors to be modified independently in the PIM header.
- the modifiable memory colors include green, sky, blue, flesh color and red.
- the memory color saturation process S 161 saturates green, sky, blue and red.
- step S 161 the memory color saturation process S 161 begins with step S 162 where the R′G′B′ are normalized according to equations Eq.(15A), (15B) and (15C).
- R n represents the normalized value of red R′
- G n represents the normalized value of green G′
- B n represents the normalized value of blue B′.
- R′G′B′ indicates that the image in a RGB format has been subjected to gamma correction.
- Step S 162 is followed by step S 164 where the mean color value or average of the R′G′B′ is computed around the color of interest (the actual color coordinate for green, blue, and red are given below and are subject to normalization).
- Steps S 164 is followed by step S 166 where a determination is made regarding equation Eq.(16) defined as
- R In , and G In are the normalized color of interest components.
- the value of K is chosen to be 20 for green and red and 12 for sky blue because there are less color variations in the sky color.
- step S 166 determines whether the gamma correction is removed from the average or mean color value denoted as R AVG G AVG B AVG .
- Step S 170 is followed by step S 172 where the mean color value R AVG G AVG B AVG is transformed into hue (H), saturation (S) and intensity (I) space (hereinafter referred to as “HSI space”) according to equation Eq.(17)
- ⁇ is a memory transformation angle
- Steps S 176 and S 178 are followed by step S 180 where the saturation (S) is computed in accordance with equation Eq.(18)
- Step S 180 is followed by step S 182 where I is computed in accordance with equation Eq.(19)
- Step S 182 is followed by step S 184 where the saturation S is increased by a percentage or fraction ( ⁇ ) without exceeding a maximum saturation of 1. More specifically, at accordance with equation Eq.(20)
- Step S 184 is followed by the process to transform the HSI space back to RGB space as set forth in FIG. 15C .
- steps S 186 A, 186 B and 186 C are shown following step S 184 .
- Equation Eq.(20A) is the transformation for blue B O and is computed at step S 190 A;
- Eq.(20B) is the transformation for red R O and is computed at step S 192 A;
- Eq.(20C) is the transformation for green G O and is computed at step S 194 A.
- the subscript “O” is used to denote an output.
- equation Eq.(21A) offsets the hue (H) by 120 at step S 188 B; equation Eq.(21B) is the transformation for red R O computed at step S 190 B; equation Eq.(21C) is the transformation for green G O computed at step S 192 B; and equation Eq.(21D) is the transformation for blue B O computed at step S 194 B.
- equation Eq.(22A) offsets the hue (H) by 240 at step S 188 C
- equation Eq.(22B) is the transformation for green G O computed at step S 190 C
- equation Eq.(22C) is the transformation for blue B O computed at step S 192 C
- equation Eq.(22D) is the transformation for red R O computed at step S 194 C.
- the saturated mean color difference is the difference between the input RGB which is represented by R′G′B′ and the output RGB or R O G O B O .
Abstract
Description
- I. Field
- The present disclosure relates generally to imaging rendering, and more specifically to techniques for automatically extracting print image matching (PIM) parameters that can be automatically adjusted to produce a better looking (visually enhanced) image.
- II. Background
- Referring to
FIGS. 1A and 1B , the gamut curve for a typical color gamut is denoted as 2 and has a generally horseshoe shape. The horseshoe shapedgamut curve 2 represents the entire range of possible colors. Overlaid in the horseshoe shapedgamut curve 2 is the gamut curve denoted as 4 for an image taken by a digital still camera image. The gamut curve denoted as 6A is for a standard red, green blue (sRGB) display device, such as a CRT or other typical computer monitor, and has a generally triangular shape. The corners of the triangle represent the primary colors red (R), green (G) and blue (B) of the gamut. InFIG. 1B , in lieu of thegamut curve 6A, a gamut curve denoted as 6B is overlaid which is the gamut curve for an EPSON® 6-color inkjet printer. - As can be readily seen, the color gamut is device-dependent. A digital still camera device may be able to capture more colors than what a sRGB display device can render. Moreover, a multi-ink printing device has a wider color gamut than the sRGB display.
- Print Image Matching (hereinafter, PIM) is technology introduced by EPSON® to allow a camera/user to specify settings that are later used by the printer to process an image at print time. The PIM technology enables conveying the image “as captured” to the printer along with control information used to instruct the printer to perform certain operations. The PIM technology creates information outside of the visible color space of a sRGB display that could be used to print on a printer's wider color space. The PIM technology also provides active control of image print quality by a camera's manufacturer or user.
- For example, PIM technology allows a camera's subsystem to correct for light/color imbalances occurring at the time of taking a picture without processing the image; the parameters for processing the image are included in the EXIF header (PIM tag) and are later used by the printer to apply picture specific processing. The camera user can also potentially specify types of pictures (i.e., portrait, landscape scenery, etc.) which can be interpreted by the camera subsystem information PIM parameters. The PIM technology supports multiple parameters with often overlapping scope.
- An image with the PIM correction ON is corrected for light/color imbalances when rendered on or by a display device. The same image with the PIM correction OFF may appear lighter or darker due to light/color imbalances. Specifically, the PIM parameters are used to make the image more balanced. For example if the image is too bright, the PIM parameters can make the image darker. However, if the image is too dark, the PIM parameters can make the image brighter.
- However, the ease of use of PIM parameters has generally been unacceptable.
- Techniques to automatically extract PIM parameters that can be automatically adjusted to produce a better looking (visually enhanced) image are described herein. In an embodiment, a system is disclosed comprising an image capturing device operable to capture an original image and extract PIM parameter data automatically based on specifics of the original image. The system includes a rendering device which is PIM-enabled. Furthermore, the system has a processor operable to calculate automatically at least one automated PIM parameter from the PIM parameter data. The processor also inserts the at least one automated PIM parameter in PIM header information for communication to the rendering device to modify the original image when rendered.
- In another embodiment, an apparatus is disclosed which comprises a processor operable to calculate automatically at least one automated PIM parameter based on automatically extracted PIM parameter data based on specifics of an original image. The processor also inserts the at least one automated PIM parameter in PIM header information for communication to a PIM-enabled rendering device to modify the original image when rendered. The apparatus also includes memory coupled to the processor.
- A further aspect includes a wireless device comprising an image capturing module operable to capture an original image. The device also include a processor operable to automatically extract PIM parameter data automatically based on specifics of the original image and insert the PIM parameter data in a header slot of a header appended to the original image. The device creates an image data file with the header and the original image for communications. A communication module of the device communicates the image data file, the image data file being used to calculate automatically at least one automated PIM parameter.
- Various aspects and embodiments of the disclosure are described in further detail below.
- Aspects and embodiments of the disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference characters identify correspondingly throughout.
-
FIG. 1A shows gamut curves for a typical color gamut, a standard RGB display and a digital still camera. -
FIG. 1B shows gamut curves for a typical color gamut, an Epson 6-color inkjet printer and a digital still camera. -
FIG. 2 illustrates a general diagram of a system for automatic PIM parameter extraction and rendering on a display device wherein the image is captured by a image capturing device. -
FIG. 3 illustrates a general diagram of a system for automatic PIM parameter extraction and rendering on a display device wherein the image is captured by a camera wireless phone device with still imaging capturing or video capability. -
FIG. 4 illustrates a general block diagram of the automatic PIM parameter extraction and rendering. -
FIG. 5 illustrates a list of PIM parameters in the PIM parameters module. -
FIG. 6 illustrates the general flowchart for the PIM parameter setting method. -
FIG. 7 illustrates a plot or curve for contrast setting. -
FIGS. 8A-8B illustrates plots or curves for brightness setting. -
FIG. 9 illustrates a plot or curve for highlight point setting. -
FIG. 10 illustrates a flowchart of an automatic PIM parameter extraction method. -
FIG. 11 illustrates a block diagram of a PIM image equalization processing process in a rendering device. -
FIGS. 12A-12B illustrate a flowchart of the histogram equalization process. -
FIG. 13 illustrates a flowchart of the shadow and highlight point setting process. -
FIGS. 14A-14C illustrate a flowchart for a memory color saturation process. -
FIG. 15A illustrates a plot of an original image luma histogram. -
FIG. 15B illustrates a plot of a transformed image luma histogram. - The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or designs.
- Referring now to the drawings in detail, and more specifically to
FIG. 2 , an exemplary system, generally designated at 10, for automatic PIM parameters extraction and rendering on a display device is shown. Thesystem 10 includes an apparatus in the form of acomputer 14 with adisplay 14A, such as a CRT, LCD, etc., aprocessor 15 and memory 16 (shown in phantom), andkeyboard 18. In lieu of or in addition to thekeyboard 18, other data input and/or computer navigational devices such as a mouse, voice-responsive assemblies may be included. Thecomputer 14 is coupled via a wire or wireless connection to animage capturing device 12 and aprinting device 35. In the exemplary embodiment, theprinting device 35 is a PIM-enabledprinting device 35. Additionally, thememory 16 includes machine readable medium for storing program instructions therein. The apparatus is not limited to thecomputer 14, but may be any other type of general purpose computing device. -
FIG. 3 illustrates a system, generally designated at 10′, for automatic PIM parameters extraction and rendering on a display device using a wirelesscamera phone device 12′. Thesystem 10′ is essentially the same assystem 10 except that in lieu of animage capturing device 12, a wirelesscamera phone device 12′ is used. - With specific reference to
FIG. 4 , a PIM parameter extraction andrendering module 20 for performing the operations described herein is shown. The PIM parameter extraction andrendering module 20 includes an automatic PIM extracting sub-module 22A and an extracted PIM parameter header inserting sub-module 22B. The PIM parameter extraction andrendering module 20 further includes aPIM parameter calculator 24. ThePIM parameter calculator 24 includes a histogram equalize image sub-module 24A to correct for brightness, contrast and gamma value setting. The operation (process S101) of the histogram equalize image sub-module 24A is set forth inFIGS. 12A and 12B . The shadow and highlight point setting sub-module 24B stretches the image. The operation (process S141) of the shadow and highlight point setting sub-module 24B is set forth inFIG. 13 . The memorycolor saturation sub-module 24C saturates a plurality of colors. The operation (process S161) of the memorycolor saturation sub-module 24C is set forth inFIGS. 14A-14C . - The program instructions of
sub-modules image capturing device 12 or wirelesscamera phone device 12′. Alternately, the a histogram equalize image sub-module 24A, the shadow and highlight point setting sub-module 24B and the memorycolor saturation sub-module 24C may reside oncomputer 14. In another arrangement, if an image capturing device is coupled to or integrated with thecomputer 14, the entire PIM parameter extraction andrendering module 20 may reside on a single machine orcomputer 14. - Program instructions may be used to cause a general-purpose or special-purpose processing system that is programmed with the instructions to perform the methods described herein. Alternatively, the methods may be performed by specific hardware components that contain hardwired logic for performing the methods, or by any combination of programmed computer components and custom hardware components. The methods described herein may be provided as a computer program product that may include a machine readable medium having stored thereon instructions that may be used to program a processing system or other electronic device to perform the methods. The term “machine readable medium” or “machine accessible medium” used herein shall include any medium that is capable of storing or encoding a sequence of instructions for execution by the machine and that causes the machine to perform any one of the methods described herein. The terms “machine readable medium” and “machine accessible medium” shall accordingly include, but not be limited to, solid-state memories, optical and magnetic disks, and a carrier wave that encodes a data signal. Furthermore, it is common in the art to speak of software, in one form or another (e.g., program, procedure, process, application, module, logic, and so on) as taking an action or causing a result. Such expressions are merely a shorthand way of stating the execution of the software by a processing system to cause the processor to perform an action or produce a result.
- Returning again
FIGS. 2 and 4 , the PIM parameter extraction andrendering module 20 is operable to display on any ofprinter 35, (computer)display 14A, or display ofimage capturing device 12. A general overview of the operation forsystem 10 will now be described. First, a user takes a picture or captures a still image with theimage capturing device 12. In the exemplary embodiment, the user transfers, downloads or otherwise saves the picture or still image (hereinafter referred to as “image data” in memory or a machine readable medium. The image data file will contain the actual picture or image captured plus some header information. - When the
image capturing device 12 or wirelesscamera phone device 12′ does not support PIM or in a regular mode, the captured image may be encoded, and a header is appended thereto. For example, for a still jpeg image, the jpeg format has a header with a PIM parameters slot. When PIM is not supported or in a regular mode, the PIM parameters slot is empty. However, if PIM is supported, the still jpeg image and related header is modified to include the extracted PIM parameters. The automatic PIM extracting sub-module 22A extracts data related to the PIM parameters while the extracted PIM parameter header inserting sub-module 22B inserts the extracted PIM parameters in the header information to create the image data file. - In the exemplary embodiment, the extracted PIM parameters include the PIM parameter data necessary for the histogram equalize image sub-module 24A, the shadow and highlight point setting sub-module 24B and the memory color saturation sub-module 24C to perform their operations by
computer 14. After the a histogram equalize image sub-module 24A, the shadow and highlight point setting sub-module 24B and the memorycolor saturation sub-module 24C perform their operations to create the printers PIM parameters for rendering the image, the PIM header toprinter 35 is modified to include the adjusted parameters of the a histogram equalize image sub-module 24A, the shadow and highlight point setting sub-module 24B and the memorycolor saturation sub-module 24C. - The
printer 35 receives the original image data with a set of PIM parameters in a PIM header. The original image data is not changed or modified in any way. However, the PIM parameters in the printer header are used by the PIM-enabledprinter 35 to render the original image with enhanced color and lighting balance via the PIM header. - If the
printer 35 was not a PIM-enabled, the non-PIM-enabled printer prints only the original image without any modification. Furthermore, PIM parameters can be interpreted not only byprinter 35 which produces a hardcopy display of the image, but by other display devices. Some special monitors may be a rendering device (which can be a printer or a monitor); display rendering is not actually seen until it is printed. - Referring now to
FIGS. 3 and 4 , the wirelesscamera phone device 12′ is operable to take or capture a picture or still image (hereinafter referred to as “image data”). The wirelesscamera phone device 12′ is also operable to send or transmit the image data. Before transmitting the image data, the wirelesscamera phone device 12′ will encode image data and append a jpeg header to form a jpeg image file. The jpeg image file can be transmitted to different devices via wireless communications. In the exemplary embodiment, the wirelesscamera phone device 12′ transmits the jpeg image file tocomputer 14. - The wireless
camera phone device 12′ includes the automatic PIM extracting sub-module 22A and the extracted PIM parameter header inserting sub-module 22B. When the picture or still image is taken, just before the wirelesscamera phone device 12′ compresses the still image, certain processing is applied to the (original) still image to extract some information from the he (original) still image. Then, the header is developed. For a jpeg picture, the header is a jpeg header, but in the jpeg header there is small space reserved for PIM parameters which in the regular mode would have been empty. If the wirelesscamera phone device 12′ includes the automatic PIM parameter extracting sub-module 22A and the extracted PIM parameter header inserting sub-module 22B, then certain PIM parameter data based on the image specifics is extracted and inserted in the jpeg header. - The image file can be sent to different places. If the image file is opened on a
display 14A, it is not going to be different than the original image, the PIM parameters are not seen by regular jpeg decoders. However, if original image is printed, that is, if the original image was sent to theprinter 35, usually theprinter 35 will decode the image and it will print it out. If the printer does not support PIM, then the printer will just decode the mage and then print whatever is in the image (whatever was encoded) exactly as it was taken. If there is PIM information, and if theprinter 35 is PIM-enabled, theprinter 35 prints the image with the automatically calculated PIM parameters and other PIM parameters set by the user or device. - According to the exemplary embodiment, the original image is not modified. The original image is process to automatically note (without user or device intervention) whether the original image is too dark. While keeping the original image and knowing that the image was dark, the automatically calculated PIM parameters advise the printer to do correct the image during rendering without changing the original image.
- In general, there are four elements which are not required to be connected or function as the same device. The first element is a device that generates the original image or picture. The second element extracts the PIM parameter data and inserts them in the image header. The third element automatically calculates a set of PIM parameters automatically based on the extracted PIM parameter data and creates a PIM header which includes at least the set of PIM parameters calculated. The fourth element includes the rendering device.
- In the wireless
camera phone device 12′ application, the wirelesscamera phone device 12′ includes both the first and second elements. In other words, the wirelesscamera phone device 12′ includes a module for capturing a still image and includes the automatic PIM extracting sub-module 22A and the extracted PIM parameter header inserting sub-module 22B. Other image capturing devices may not include both of the first and second elements. - Referring also to
FIG. 6 , aparameter setting procedure 30 forsystem 10 is shown and begins with step S32 where an image file is created by the imagefile creator module 16. Examples of an image file include JPEG and TIFF formats. While only two examples of image files are provided, currently there are many other image files that can be used. The image file is created after the user shoots or captures the picture or scene with theimage capturing device 12. Step S32 is followed by step S34 where the PIM parameters are calculated according to the methods described in detail below via thePIM extraction module 20. In general, thePIM extraction module 20 includes the program code or instructions executable byprocessor 15. - Step S34 is followed by step S36 where the PIM parameters are set. Step S36 is followed by step S38 where the image is printed using the PIM-enabled
printing device 35 with PIM support. - The user can then visually inspect or perform a sensory evaluation to determine if any further adjustments are necessary using the
display 14A on thecomputer 14. Thecomputer 14 is shown as a personal computer having stored therein the computer program code or instructions for carrying out the automatic PIMparameter extraction method 100. Thecomputer 14 may be a Laptop, Notebook, Tablet or other computing device with printing capability and a port for connecting to theprinting device 35. - As best seen in
FIG. 5 , thePIM parameters module 28 includes a gamma value setting, releasing RGB clipping, color space setting, shadow point setting, highlight point setting, contrast setting, brightness setting, RGB color balance setting, saturation setting, sharpness setting, memory color correction setting, HSB (hue, saturation, brightness) correction setting, tone curve setting and channel mix setting. - A plot or curve of the contrast setting is shown in
FIG. 7 . The contrast setting plot is shown increasing at the point of the denoted circle. InFIGS. 8A and 8B , plots or curves of the brightness setting for the output level verses the input level are shown. In the first plot (FIG. 8A ), the brightness setting plot is shown increasing. InFIG. 9 , plots of the highlight point setting and shadow point setting are shown. The arrows at the top indicate the direction of the highlight setting while the bottom arrows indicate the direction of the shadow setting. The memory color correction setting provides RGB offsets for green, sky blue, flesh color and red. - Referring now to
FIG. 10 , in general, the automatic PIMparameter extraction method 100, first, applies or uses the gamma value, brightness, and contrast settings to histogram equalize the image (build histogram equalization function with the three curves) via a histogram equalization process S101. An exemplary histogram equalization process S101 is shown inFIGS. 12A-12B . Second, themethod 100 uses a shadow and highlight point setting process S141 to stretch the image to adjust the image in the range of 0-255. An exemplary shadow and highlight point setting process S141 is shown inFIG. 13 . Third, themethod 100 saturates the memory color for a plurality of colors in a memory color saturation process S 161. For example, all colors except the flesh color can be saturated in the memory color saturation process S161. An exemplary memory color saturation process S161 is shown inFIGS. 14A-14C . - The
method 100 analyzes an image histogram and designs a transformation which will correct some image deficiencies detectable from the histogram, i.e., too bright or too dark image. The designed transformation is then to be implemented using a selected set of PIM tools (parameters). Three PIM parameters (gamma value, contrast, and brightness) are used to construct the transformation. Those are specified in the PIM header as scalars but are expanded by theprinting device 35 to full curves according to the PIM specification. Knowing the scalar-curve mapping allows for “constructing” those curves and using them to construct the desired transformation. Another parameter, sharpness, was also considered. It was concluded that the way sharpness is being defined in the PIM specification (i.e., filtering operation triggered based on exceeding a threshold value) does not represent a reasonable addition to histogram processing nor it has use for edge enhancement. - Referring now to
FIG. 11 , a PIM imageequalization processing process 40 in arendering device 35 such as a PIM-enabledprinting device 35, is shown. Theprocess 40 begins with a transport modified input or a gamma corrected R′G′B′ image at step S42. Step S42 is followed by step S44 where the gamma having a gamma value (γ) is removed according to equation Eq.(1) -
y=x γ Eq.(1) - where x is a source. In general, gamma correction is applied by a rendering device because the rendering device will have a certain response. The rendering device will have a modify the image. Thus, the reverse of such modification is called gamma correction which effectively removes that response or modification. After the gamma correction, the image is back to its original state. The gamma correction transformation may be device specific.
- The first gamma value (γ) and parameters for the subsequent transformations are contained in the PIM header. Accordingly, step S44 is followed by step S46A where the first transformation (H1(x)) is calculated according to equation Eq.(2)
-
y=H1(x) Eq.(2) - Step S46A is followed by the intermediate steps for calculating subsequent transformations up to transform N (HN(x)) at step S46N according to equation Eq.(3)
-
y=HN(x) Eq.(3) - Step S46N is followed by step S48 where the rendering device (
printer 35 or a display monitor) applies the display gamma according to equation Eq.(4) -
x1/γ Eq.(4) - The steps S46A-S46N relate to the PIM parameters. In the exemplary embodiment, the transformations F1, F2, . . . FN relate to the PIM parameters. For example, transformations F1, F2 and F3 can be for the highlight point setting, gamma correction and memory color settings, respectively. All other transformations F4-FN support additional PIM parameters.
- It is well known that histogram equalization will produce an image that will exhibit the greatest dynamic range. However sometimes histogram equalization does not produce the desired contrast in the areas of interest (too dark image with small bright element or too bright image with a small dark element). In those cases histogram equalization is too aggressive (shift dark to grey in the first case and white to grey in the second case). Several methods have been used in an attempt to limit (CAP) the histogram equalization.
- In the
method 100, the function that is used to obtain the histogram equalization is averaged with no transformation (y=x) function with varying weights for the two functions. - The histogram equalization of a source X is performed as Y=T(X) where T( ) is the cumulative density function, defined according to equation Eq.(5) as
-
- where h(k) is the component histogram; and x=0-255. The transform function (T(x)) used in this case is then obtained according to equation Eq.(6)
-
T(x)=H(x)+(1−α)x/255 Eq.(6) - where x=0-255; and a is chosen to be 2/3. The parameter “α” is for equalization aggressiveness within a range between 0 to 1 where if α=0 there is no equalization and if α=1 there is full equalization.
- Referring also to
FIGS. 15A and 15B , a luma histogram plot is shown on an image before and after the transformation for α equal to 2/3. - It is noted that although the goal is to histogram equalize a gamma corrected image, the processing is applied after gamma correction is removed (See Step S44). Accordingly, the following identities are presented.
- In view of the foregoing, the histogram equalization process S101 begins at step S102 where the histogram h(i) of L′ (gamma corrected luma) is computed. Step S102 is followed by step S104 where the cumulative density function H(i) is computed. Step S104 is followed by step S106 where the cumulative density function H(i) is applied according to equation Eq.(7)
-
H(i)=H((i/2551/2.2)*255) Eq.(7) - where i varies from 0-255. Step S106 is followed by step S108 where the transform function (T(i)) is computed according to equation Eq.(6) rewritten (by substituting i or x) into equation (8)
-
T(i)=H(i)+(1−α)i/255 Eq.(8) - where i=0-255; and a is chosen to be 2/3. Step S108 is followed by step S110 where gamma γ is found that minimizes equation Eq.(9)
-
E[(T(i)−(i/255)γ)2] Eq.(9) - where T(i) is the transfer function of equation Eq.(6). Step S110 is followed by step S112 where the difference between the transform function and the component to remove gamma is computed according to equation Eq.(10)
-
R(i)=T(i)−(i/255)γ Eq.(10) - where i=0-255 and where iγ is the means (raising to the power) to remove gamma.
- Continuing to
FIG.12B , step S112 is followed by step S114 where the contrast C setting is computed according to equation Eq.(11) -
- Step S114 is followed by step S116 where the brightness B setting is computed according to equation Eq.(12)
-
- Step S116 is followed by
step S1 18 where the gamma correction value (γ) is computed according to equation Eq.(13) -
γ=γ+1.2 Eq.(13) - because γ=2.2 is equivalent to no additional gamma correction.
- Step S118 is followed by step S120 where the gamma correction value is inserted in the PIM header.
- Using the histogram equalization process S101, a sample of the inserted values in PIM header is shown in TABLE 1.
-
TABLE 1 PIM Header Parameter Inserted Value Gamma Correction Value 16 (1.6) Contrast −3 Brightness −5 - The next batch of PIM parameters for improving print picture quality are: RGB color balance settings, highlight/shadow point setting, and memory colors correction setting. While not wishing to be bound by theory, it is concluded that the RGB color balance setting is an operation that is already performed on the image (white balance) via a matrix multiplication and addition. The exact same degrees of freedom are supported in PIM header which makes this correction unnecessary.
- The shadow and highlight setting process S141 is basically a linear point-to-point mapping and is required when the
image capturing device 12 or the wirelesscamera phone device 12′ uses a RGB format that is other than [0:255]. Commonly, a wireless camera phone device may use a RGB format that is [19:238]. Other RGB formats may be used. Thus, the best use of the shadow and highlight setting process S141 is “stretching” the captured image if it is other than [0:255]. - Referring now to
FIG. 13 , the shadow and highlight setting process S141 begins with step S142 where the gamma corrected R′G′B′ format image is obtained having a range [P:Q]. Step S142 is followed by step S144 where the R′G′B′ format image is converted to a linear RGB formatted image such as in the range [P′:Q′] in accordance with equations Eq.(14A) and Eq. (14B). -
P′=((P/255)γ)*255 Eq. (14A) -
Q′=((Q/255)γ)*255 Eq. (14B) - where gamma (γ) is 2.2. Step S144 is followed by step S146 where the shadow point is set to P′. In step S148, the highlight point is set to 255-Q′. Thus, the image at step S148 has been effectively stretched to the range of [0:255].
- Assume, the captured image or snapshot from wireless
camera phone device 12′ has a 219R′G′B′ format (e.g. the gamma corrected RGB values lie [19:238]). When converted to a linear RGB format (removing gamma) the range is [1:219]. Therefore, the image can be stretched to [0:255] by setting shadow point to 1 and highlight point to 36. As can be appreciated, the shadow and highlight setting process S141 is only required for those devices which produce an image that requires stretching to [0:255]. - The memory color saturation process S161, in general, modifies colors that lie within a certain range to obtain more vivid color representation for colors that are remembered when looked at (such as foliage green, sky blue, skin, water etc.). According to some psychophysical experiments, it has been found that the general public likes green, red, and blue to be saturated and flesh color untouched (flesh color is dependent on cultural preferences and that is the reason it is not modified).
- Furthermore, the memory color saturation process S161 allows the memory colors to be modified independently in the PIM header. The modifiable memory colors include green, sky, blue, flesh color and red. However, the memory color saturation process S161 saturates green, sky, blue and red.
- Referring now to
FIGS. 14A-14C , the memory color saturation process S161 begins with step S162 where the R′G′B′ are normalized according to equations Eq.(15A), (15B) and (15C). -
- Where Rn represents the normalized value of red R′; Gn represents the normalized value of green G′; and Bn represents the normalized value of blue B′. R′G′B′ indicates that the image in a RGB format has been subjected to gamma correction.
- Step S162 is followed by step S164 where the mean color value or average of the R′G′B′ is computed around the color of interest (the actual color coordinate for green, blue, and red are given below and are subject to normalization). Steps S164 is followed by step S166 where a determination is made regarding equation Eq.(16) defined as
-
(R In −R In)2+(G In −G In)2<K2 Eq.(16) - where RIn, and GIn are the normalized color of interest components. The value of K is chosen to be 20 for green and red and 12 for sky blue because there are less color variations in the sky color.
- If the determination at step S166 is “YES,” the process continues to step S168 where a point is added to the average or mean color value (R′G′B′). However, if the determination at step S166 is “NO,” step S166 is followed by step S170. Step S168 is also followed by step S170 where the gamma correction is removed from the average or mean color value denoted as RAVGGAVGBAVG.
- Step S170 is followed by step S172 where the mean color value RAVGGAVGBAVG is transformed into hue (H), saturation (S) and intensity (I) space (hereinafter referred to as “HSI space”) according to equation Eq.(17)
-
- where θ is a memory transformation angle.
- Step S172 is followed by step S174 where if the condition BAVG<=GAVG is true, then the step S174 is followed by step S176 where the hue (H) is defined as H=θ.
- At step S174, if the condition BAVG<=GAVG is not true then BAVG>GAVG. Hence, Step S174 is followed by step S178 where the hue (H) is defined as H=360−θ.
- Steps S176 and S178 are followed by step S180 where the saturation (S) is computed in accordance with equation Eq.(18)
-
- Step S180 is followed by step S182 where I is computed in accordance with equation Eq.(19)
-
I=1/3(R AVG +G AVG +B AVG) Eq.(19) - Step S182 is followed by step S184 where the saturation S is increased by a percentage or fraction (α) without exceeding a maximum saturation of 1. More specifically, at accordance with equation Eq.(20)
-
S=(1+α)S Eq.(20) - subject to S<=1 and 0<α<1.
- Step S184 is followed by the process to transform the HSI space back to RGB space as set forth in
FIG. 15C . - Referring now to
FIG. 15C , steps S186A, 186B and 186C are shown following step S184. At step S186A, a determination is made whether thecondition 0<=H<120 is met. If the determination is “YES”, the transformation process from HSI space to RGB space is defined by equations Eq. (20A), (20B) and (20C) -
- where equation Eq.(20A) is the transformation for blue BO and is computed at step S190A; Eq.(20B) is the transformation for red RO and is computed at step S192A; and Eq.(20C) is the transformation for green GO and is computed at step S194A. The subscript “O” is used to denote an output.
- At step S186B, if the
condition 120<=H<240 is true, the transformation process from HSI space to RGB space is defined by equations Eq. (21A), (21B), (21C) and (21D) -
- where equation Eq.(21A) offsets the hue (H) by 120 at step S188B; equation Eq.(21B) is the transformation for red RO computed at step S190B; equation Eq.(21C) is the transformation for green GO computed at step S192B; and equation Eq.(21D) is the transformation for blue BO computed at step S194B.
- At step S186C, if 240<=H<360 is true, the transformation process from HSI space to RGB space is defined by equations Eq. (22A), (22B), (22C) and (22D)
-
- where equation Eq.(22A) offsets the hue (H) by 240 at step S188C; equation Eq.(22B) is the transformation for green GO computed at step S190C; equation Eq.(22C) is the transformation for blue BO computed at step S192C; and equation Eq.(22D) is the transformation for red RO computed at step S194C.
- The saturated mean color difference is the difference between the input RGB which is represented by R′G′B′ and the output RGB or ROGOBO.
- The values for the center points for green, blue, and red are given in TABLE 2 below:
-
TABLE 2 R G B Green 160 190 60 Blue 96 124 159 Red 179 47 58 - and are normalized to (R+G+B=255) in TABLE 3 below:
-
TABLE 3 R G B Green 100 118 37 Blue 65 83 107 Red 161 42 52 - The application of memory colors adjustment with following parameters is shown below in TABLE 4.
-
TABLE 4 Green Sky Blue Red 2 −15 Green 2 −6 Blue −27 21 - The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (46)
H(i)=H 1/2.2(i);
T(i)=H(i)+(1−α)i/255
E[(T(i)−iγ)2]; and
R(i)=T(i)−i γ.
γ=γ+1.2.
P′=((P/255)γ)*255
and
Q′=((Q/255)γ)*255
H=θ
H=360−θ
I=1/3(R AVG +G AVG +B AVG).
S=(1+α)S
subject to S<=1 and 0<α<1.
H(i)=H 1/2.2(i);
T(i)=H(i)+(1−α)i/255
E[(T(i)−i γ)2]; and
R(i)=T(i)−i γ.
γ=γ+1.2.
P′=((P/255)γ)*255
and
Q′=((Q/255)γ)*255
H=θ
H=360−θ
I=1/3(R AVG +G AVG +B AVG).
S=(1+α)S
H(i)=H 1/2.2(i);
T(i)=H(i)+(1−α)i/255
E[(T(i)−i γ)2]; and
R(i)=T(i)−i γ.
γ=γ+1.2.
P′=((P/255)γ)*255
and
Q′=((Q/255)γ)*255
H=θ
H=360−θ
I=1/3(R AVG +G AVG +B AVG).
S=(1+α)S
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/761,981 US8634103B2 (en) | 2007-06-12 | 2007-06-12 | Print image matching parameter extraction and rendering on display devices |
EP08006435A EP2003872A1 (en) | 2007-06-12 | 2008-03-31 | Print image matching parameter extraction and rendering on display devices |
PCT/US2008/066451 WO2008154550A1 (en) | 2007-06-12 | 2008-06-10 | Print image matching parameter extraction and rendering on display devices |
TW097121974A TW200907853A (en) | 2007-06-12 | 2008-06-12 | Print image matching parameter extraction and rendering on display devices |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/761,981 US8634103B2 (en) | 2007-06-12 | 2007-06-12 | Print image matching parameter extraction and rendering on display devices |
Publications (2)
Publication Number | Publication Date |
---|---|
US20080309963A1 true US20080309963A1 (en) | 2008-12-18 |
US8634103B2 US8634103B2 (en) | 2014-01-21 |
Family
ID=39495653
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/761,981 Expired - Fee Related US8634103B2 (en) | 2007-06-12 | 2007-06-12 | Print image matching parameter extraction and rendering on display devices |
Country Status (4)
Country | Link |
---|---|
US (1) | US8634103B2 (en) |
EP (1) | EP2003872A1 (en) |
TW (1) | TW200907853A (en) |
WO (1) | WO2008154550A1 (en) |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040239955A1 (en) * | 2003-03-12 | 2004-12-02 | Yasuhiko Uchida | Print job creation apparatus and print job creation method |
US20050207644A1 (en) * | 2004-03-22 | 2005-09-22 | Fuji Xerox Co., Ltd. | Image processing apparatus, image processing method and program product therefor |
US20050243347A1 (en) * | 2004-03-08 | 2005-11-03 | Ikuo Hayaishi | Conversion of color image to monochrome image |
US7081918B2 (en) * | 2000-04-28 | 2006-07-25 | Fuji Photo Film Co., Ltd. | Image processing method, image processing apparatus and recording medium storing program therefor |
US7181091B2 (en) * | 2001-03-16 | 2007-02-20 | Fuji Photo Film Co., Ltd. | Method, apparatus, and recording medium for correcting appreciation data |
US7327875B2 (en) * | 2002-11-27 | 2008-02-05 | Kabushiki Kaisha Tosiba | Method and apparatus for color conversion |
US7428082B2 (en) * | 2002-05-07 | 2008-09-23 | Seiko Epson Corporation | Update control of image processing control data |
US7636473B2 (en) * | 2004-03-12 | 2009-12-22 | Seiko Epson Corporation | Image color adjustment |
US7683973B2 (en) * | 2003-07-31 | 2010-03-23 | Canon Kabushiki Kaisha | Image processing method and apparatus performing color adjustment |
US7724977B2 (en) * | 2001-07-12 | 2010-05-25 | Do Labs | Method and system for providing formatted data to image processing means in accordance with a standard format |
-
2007
- 2007-06-12 US US11/761,981 patent/US8634103B2/en not_active Expired - Fee Related
-
2008
- 2008-03-31 EP EP08006435A patent/EP2003872A1/en not_active Withdrawn
- 2008-06-10 WO PCT/US2008/066451 patent/WO2008154550A1/en active Application Filing
- 2008-06-12 TW TW097121974A patent/TW200907853A/en unknown
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7081918B2 (en) * | 2000-04-28 | 2006-07-25 | Fuji Photo Film Co., Ltd. | Image processing method, image processing apparatus and recording medium storing program therefor |
US7181091B2 (en) * | 2001-03-16 | 2007-02-20 | Fuji Photo Film Co., Ltd. | Method, apparatus, and recording medium for correcting appreciation data |
US7724977B2 (en) * | 2001-07-12 | 2010-05-25 | Do Labs | Method and system for providing formatted data to image processing means in accordance with a standard format |
US7428082B2 (en) * | 2002-05-07 | 2008-09-23 | Seiko Epson Corporation | Update control of image processing control data |
US7327875B2 (en) * | 2002-11-27 | 2008-02-05 | Kabushiki Kaisha Tosiba | Method and apparatus for color conversion |
US20040239955A1 (en) * | 2003-03-12 | 2004-12-02 | Yasuhiko Uchida | Print job creation apparatus and print job creation method |
US7683973B2 (en) * | 2003-07-31 | 2010-03-23 | Canon Kabushiki Kaisha | Image processing method and apparatus performing color adjustment |
US20050243347A1 (en) * | 2004-03-08 | 2005-11-03 | Ikuo Hayaishi | Conversion of color image to monochrome image |
US7636473B2 (en) * | 2004-03-12 | 2009-12-22 | Seiko Epson Corporation | Image color adjustment |
US20050207644A1 (en) * | 2004-03-22 | 2005-09-22 | Fuji Xerox Co., Ltd. | Image processing apparatus, image processing method and program product therefor |
Also Published As
Publication number | Publication date |
---|---|
WO2008154550A1 (en) | 2008-12-18 |
TW200907853A (en) | 2009-02-16 |
US8634103B2 (en) | 2014-01-21 |
EP2003872A1 (en) | 2008-12-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7375848B2 (en) | Output image adjustment method, apparatus and computer program product for graphics files | |
US6535301B1 (en) | Image processing apparatus, image processing method, image processing program recording medium, color adjustment method, color adjustment device, and color adjustment control program recording medium | |
US9055263B2 (en) | Apparatus, method and computer program for correcting an image using lookup tables generated by inputted image analysis | |
US7652717B2 (en) | White balance correction in digital camera images | |
US8391646B2 (en) | Image processing device and method that performs image quality adjustment and resize processing on image data | |
US20050243347A1 (en) | Conversion of color image to monochrome image | |
US7924326B2 (en) | Image quality adjustment processing device | |
EP1130899B1 (en) | Method, apparatus and recording medium for image processing | |
JP2007094742A (en) | Image signal processor and image signal processing program | |
US7613338B2 (en) | Image processing method and apparatus, program, and storage medium | |
US20080062484A1 (en) | Image processing device and image processing method | |
US20100182620A1 (en) | Image processing device and image processing method | |
JP2004096506A (en) | Image forming method, image processor and image recording device | |
US20050068587A1 (en) | Monotone conversion process for color images | |
US8634103B2 (en) | Print image matching parameter extraction and rendering on display devices | |
JP4006590B2 (en) | Image processing apparatus, scene determination apparatus, image processing method, scene determination method, and program | |
JP2000092337A (en) | Image processing method, its device and recording medium | |
US20040042025A1 (en) | Image processing method, image processing apparatus, image recording apparatus, program and recording medium | |
JP2005167929A (en) | Image processing using information associated with image data | |
JP2004222078A (en) | Image processor, image processing method, and program thereof | |
JP2004328534A (en) | Image forming method, image processing apparatus and image recording apparatus | |
US20230388465A1 (en) | Chroma adjustment method, electronic device and readable storage medium | |
JP2004104248A (en) | Image processing apparatus, image reader, image forming apparatus, and image processing method | |
JP2003032618A (en) | Device for producing process control data contained image filter | |
JP2004295410A (en) | Image processing apparatus, image processing method and image processing program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: QUALCOMM INCORPORATED, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:QUAN, SHUXUE;ATANASSOV, KALIN;JIANG, XIAOYUN;REEL/FRAME:019585/0287;SIGNING DATES FROM 20070628 TO 20070718 Owner name: QUALCOMM INCORPORATED, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:QUAN, SHUXUE;ATANASSOV, KALIN;JIANG, XIAOYUN;SIGNING DATES FROM 20070628 TO 20070718;REEL/FRAME:019585/0287 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.) |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.) |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20180121 |