Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberCN101494794 A
Publication typeApplication
Application numberCN 200810186925
Publication date29 Jul 2009
Filing date10 Dec 2008
Priority date22 Jan 2008
Also published asCN101494794B, EP2083563A1, EP2083563B1, US8169491, US20090185041
Publication number200810186925.4, CN 101494794 A, CN 101494794A, CN 200810186925, CN-A-101494794, CN101494794 A, CN101494794A, CN200810186925, CN200810186925.4
Inventors吴炫和, 姜东求, 李性德, 魏浩千
Applicant三星电子株式会社;三星电机株式会社
Export CitationBiBTeX, EndNote, RefMan
External Links: SIPO, Espacenet
Apparatus and method of obtaining image and apparatus and method of processing image
CN 101494794 A
Abstract
Provided are an apparatus and method of obtaining a high-quality image by efficiently processing an image obtained at low-light levels. The image obtaining apparatus comprises a sensor which detects a pixel array value by using a color filter including color pixels for obtaining a color image and a reference pixel for obtaining a reference image used for hand-trembling function estimation; an exposure controller which controls exposure times of the color pixels and the reference pixel; and an image generator which generates a long-exposure color image signal and a short-exposure reference image signal from the detected pixel array value, the long-exposure color image signal and the short-exposure reference image signal being aligned with each other. Accordingly, accurate estimation of thehand-trembling function is possible, thereby enabling the high-quality color image restoration.
Claims(16)  translated from Chinese
1、一种图像获得设备,包括: 传感器,通过使用彩色滤光片检测像素阵列值,所述彩色滤光片包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素; 曝光控制器,控制彩色像素和参考像素的曝光时间;和图像产生器,从检测的像素阵列值产生长曝光彩色图像信号和短曝光参考图像信号,所述长曝光彩色图像信号和短曝光参考图像信号互相对齐。 An image obtaining apparatus, comprising: a sensor detection pixel by using a color filter array value, the color filter includes means for obtaining a color image color pixels for obtaining a reference for hand-trembling function estimation The reference image pixel; exposure controller, control the color pixels and the reference pixel exposure time; and an image generator, generating a long-exposure color image signal and the short-exposure reference image signal, the long-exposure color image signal from the pixel array value detected and a short-exposure reference image signal aligned with each other.
2、 如权利要求1所述的图像获得设备,其中,曝光控制器同步彩色像素和参考像素的曝光开始时间,并控制彩色像素的曝光时间长于参考像素的曝光时间。 2. The image obtaining apparatus according to claim, wherein the exposure controller synchronization of the color pixels and the reference pixel exposure start time, and controls the exposure time of the color pixels the exposure time is longer than the reference pixel.
3、 如权利要求2所述的图像获得设备,其中,为了获得手抖函数的初始信息,曝光控制器控制传感器在曝光彩色像素的同时,重复发生参考像素的短曝光。 3, as in claim 2, wherein the image obtaining apparatus, wherein, in order to obtain initial information of the hand-trembling function, the exposure controller controls the sensor while the color pixels in the exposure, the repeated occurrence of short-exposure reference pixels.
4、 如权利要求1所述的图像获得设备,其中,图像产生器对通过彩色像素和参考像素检测的像素阵列值执行插值,从而产生彩色像素的各个颜色的长曝光图像信号和短曝光图像信号。 4, as in claim 1, wherein the image obtaining apparatus, wherein the image generator through the color pixels and the reference pixel detector pixel array value interpolation is performed to generate long-exposure image signals of the respective colors of the color pixels and short-exposure image signal .
5、 一种获得图像的方法,该方法包括:通过控制用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的曝光时间来获得像素阵列值,所述彩色像素和参考像素包括在彩色滤光片中;以及通过使用获得的像素阵列值产生长曝光彩色图像信号和短曝光参考图像信号,所述长曝光彩色图像信号和短曝光参考图像信号互相对齐。 5, a method of obtaining an image, the method comprising: means for obtaining a color image by controlling the color pixels for obtaining a reference for the hand-trembling function estimation reference pixel image exposure time to obtain a pixel array value, the color pixels and the reference pixel included in the color filter; and the pixel array value obtained by using the long-exposure color image signal is generated and the short-exposure reference image signal, the long-exposure color image signal and the short-exposure reference image signal aligned with each other.
6、 如权利要求5所述的方法,其中,在曝光时间的控制步骤中,彩色像素和参考像素的曝光开始时间彼此同步,且彩色像素的曝光时间被设置为长于参考像素的曝光时间。 6. The method of claim 5, wherein, in the step of controlling the exposure time, the color pixels and the reference pixel exposure start time synchronized with each other, and the exposure time of the color pixels is set to be longer than the exposure time of the reference pixel.
7、 如权利要求6所述的方法,其中,为了获得手抖函数的初始信息,在曝光时间的控制步骤中,在曝光彩色像素的同时,控制参考像素的曝光时间,从而重复发生参考像素的短曝光。 7. The method of claim 6, wherein, in order to obtain initial information of the hand-trembling function, exposure time in the control step, the exposure of the color pixels while controlling the exposure time reference pixel, the reference pixel whereby recurring Short exposure.
8、 如权利要求5所述的方法,其中,在图像信号的产生步骤中,对通过彩色像素和参考像素检测的像素阵列值执行插值,从而产生彩色像素的各个颜色的长曝光图像信号和短曝光图像信号。 8. The method of claim 5, wherein, in the step of generating an image signal of the color pixels and the reference pixel by pixel array detection value interpolation is performed to generate long-exposure image signals of the respective colors of the color pixels and short exposure image signal.
9、 一种图像处理设备,包括:图像获得单元,通过控制包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的彩色滤光片的曝光时间来获得互相对齐的长曝光彩色图像信号和短曝光参考图像信号;函数估计单元,通过使用长曝光彩色图像信号和短曝光参考图像信号来估计手抖函数;和图像恢复单元,通过使用估计的手抖函数对长曝光彩色图像信号执行去巻积来产生恢复的图像信号。 9. An image processing apparatus comprising: image obtaining means for obtaining by controlling a color image comprising color pixels and a reference pixel for obtaining a reference for the hand-trembling function estimation image of a color filter exposure time aligned with each other to obtain long-exposure color image signal and the short-exposure reference image signal; function estimating unit, by using the long-exposure color image signal and the short-exposure reference image signal to estimate trembling function; and an image restoration unit, by using the estimated tremor Functions of the long-exposure color image signal to perform convolving the image signal to generate recovered.
10、 如权利要求9所述的图像处理设备,其中,图像获得单元通过同步彩色像素和参考像素的曝光开始时间并控制彩色像素的曝光时间长于参考像素的曝光时间来获得长曝光彩色图像信号和短曝光参考图像信号。 10. The image processing apparatus according to claim 9, wherein the image acquiring unit by synchronizing the color pixels and the reference pixel and controls the exposure time of the color pixels start exposure time longer than an exposure time of the reference pixel to obtain long-exposure color image signal and short-exposure reference image signal.
11、 如权利要求IO所述的图像处理设备,其中,图像获得单元通过在曝光彩色像素的同时控制重复发生参考像素的短曝光来收集多个短曝光参考图像信号,函数估计单元通过^f吏用收集的参考图像信号来追踪手抖的轨迹来获得信息,并使用获得的信息作为手抖函数的初始信息。 11, as claimed in claim IO image processing apparatus, wherein the image acquiring unit through the color pixels in the exposure while controlling short-exposure reference pixel repetition to collect a plurality of short-exposure reference image signal, the function estimating unit by ^ f Officials with the reference image signal to track the trajectory of the hand-trembling collected to obtain information, and using the obtained information as the initial information of the hand-trembling function.
12、 如权利要求9所述的图像处理设备,其中,函数估计单元减小长曝光彩色图像信号和短曝光参考图像信号的噪声,将减噪的长曝光彩色图像信号转换为长曝光单通道图像信号,并通过使用转换的长曝光单通道图像信号和减噪的短曝光参考图像信号来估计手抖函数。 12. The image processing apparatus according to claim 9, wherein the noise reduction function estimating unit long-exposure color image signal and the short-exposure reference image signal, the noise-reduced long-exposure color image signal into a long-exposure single-channel image signal, and by using long-exposure single-channel image signal and the noise-reduced short-exposure reference image signal converter to estimate the hand-trembling function.
13、 一种处理图^f象的方法,该方法包括:通过控制包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的彩色滤光片的曝光时间来获得互相对齐的长曝光彩色图像信号和短曝光参考图像信号;通过使用长曝光彩色图像信号和短曝光参考图像信号来估计手抖函数;以及通过使用估计的手抖函数对长曝光彩色图像信号执行去巻积来产生恢复的图像信号。 13. A method of treating FIG like f ^, the method comprising: controlling includes means for obtaining a color image of the color pixels and the reference pixel used to obtain estimates for the hand-trembling function reference image of a color filter exposure time to get aligned with each other in the long-exposure color image signal and the short-exposure reference image signal; by using the long-exposure color image signal and the short-exposure reference image signal to estimate trembling function; and a hand-trembling function by using the estimated long-exposure color image convolving the signal to be generated to perform image signal recovery.
14、 如权利要求13所述的方法,其中,在获得图像信号的步骤中,彩色像素和参考像素的曝光开始时间彼此同步,且彩色像素的曝光时间被设置为长于参考像素的曝光时间。 14. The method of claim 13, wherein, in the step of obtaining the image signal, color pixels and the reference pixel exposure start time synchronized with each other, and the exposure time of the color pixels is set to be longer than the exposure time of the reference pixel.
15、 如权利要求14所述的方法,其中,在获得图像信号的步骤中,通过在曝光彩色像素的同时控制重复发生参考像素的短曝光来收集多个短曝光参考图像信号,通过使用收集的参考图像信号来追踪手抖的轨迹来获得信息, 并使用获得的信息作为手抖函数的初始信息。 15. The method of claim 14, wherein, in the step of obtaining an image signal by the color pixels in the exposure while controlling recurring short-exposure reference pixels to collect a plurality of short-exposure reference image signal by using the collected reference image signal to track the trajectory of camera shake to obtain information, and use the information obtained as a function of initial information tremor.
16、 如权利要求13所述的方法,其中,手抖函数的估计的步骤包括: 减小长曝光彩色图l象信号和短曝光参考图傳_信号的噪声; 将减噪的长曝光彩色图像信号转换为长曝光单通道图像信号;以及通过使用转换的长曝光单通道图像信号和减噪的短曝光参考图像信号估计手抖函数。 16. The method of claim 13, wherein the estimated hand-trembling function comprises: reducing the noise exposure color image signal and Figure l short-exposure reference image transmission _ signal; the noise of the long-exposure color image long-exposure single-channel signal into an image signal; and by using the converted long-exposure single-channel image signal and the noise-reduced short-exposure reference image signal estimated hand-trembling function.
Description  translated from Chinese

获得图像的设备和方法以及处理图像的设备和方法 Apparatus and method for obtaining an image and a device and method for processing images

本申请要求于2008年1月22日在韩国知识产权局提交的10-2008-0006478号韩国专利申请的优先权,该申请全部公开于此以资参考。 This application claims priority on January 22, 2008 in the Korean Intellectual Property Office filed Korean Patent Application No. 10-2008-0006478, the entire disclosure of this application is incorporated herein by reference.

技术领域 Technical Field

本发明涉及一种获得和处理图像的设备和方法,更具体地,涉及一种通过有效地处理在低光照等级获得的图像来获得高质量彩色图像的设备和方法。 The present invention relates to an apparatus and method for obtaining and processing images, and more particularly, relates to an apparatus and method for obtaining high-quality color image by efficiently processing an image obtained by low light level.

背景技术 Background

随着数字相机的使用的增多,有更多机会在低光照情况(诸如,弱光的室内或夜晚)下进行拍摄。 With the increasing use of digital cameras, there are more opportunities to shoot in low light situations (such as indoor low light or at night) under. 此外,相机用户对于能够在甚至是低光照条件下获得清晰的图像的功能的需求增加。 In addition, the user can get a clear camera in low light conditions and even the function of the demand for increased image. 因此,相机制造商竟相研究和开发在低光照下具有优越性能的数字相机。 Therefore, camera manufacturers race to the research and development of digital cameras have superior performance in low light. 然而,由于稳定由拍摄者的手抖或由去除拍摄的图像的噪声造成的不稳定图像会限制得到清晰的图像。 However, since the stable to get a clear picture from the photographer's hand-shake or unstable image is limited by the removal of the captured image noise.

发明内容 DISCLOSURE

提出本发明以克服上述问题,其目的在于提供一种获得互相对齐的长曝光彩色图像和短曝光参考图像的设备和方法,以不改变相机的现有结构而在低光照下恢复高质量的更清晰的彩色图像。 The present invention proposes to overcome the above problems, and aims to provide an apparatus and method for obtaining long-exposure color image and the short-exposure reference image aligned with each other, so as not to change the camera's existing structures and restore quality in low light more clear color images.

本发明还提供一种处理图像的设备和方法,以通过使用互相对齐的长曝光彩色图像和短曝光参考图像获得清晰的彩色图像。 The present invention also provides an apparatus and method of processing an image, aligned with each other by using long-exposure color image and the short-exposure reference image to obtain a clear color images.

本发明公开了一种图像获得设备,包括:传感器,通过使用彩色滤光片检测像素阵列值,所述彩色滤光片包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素;曝光控制器,控制彩色像素和参考像素的曝光时间;图像产生器,从检测的像素阵列值中产生长曝光彩 The present invention discloses an image obtaining apparatus, comprising: a sensor detection pixel by using a color filter array value, the color filter includes a color image is obtained and used to obtain color pixels for hand-trembling function estimation The reference image reference pixels; exposure controller, control the color pixels and the reference pixel exposure time; an image generator for generating a long exposure color value from the pixel array to detect the

色图像信号和短曝光参考图像信号,所述长曝光彩色图像信号和短曝光参考图像信号互相对齐。 Color image signal and the short-exposure reference image signal, the long-exposure color image signal and the short-exposure reference image signal aligned with each other. 本发明还公开了一种获得图像的方法,该方法包括:通过控制用于获得 The present invention also discloses a method for obtaining an image, the method comprising: control means for obtaining

彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的 Color pixels for obtaining a color image and a reference image for the hand-trembling function estimation reference pixel

曝光时间来获得像素阵列值,所述彩色像素和参考像素包括在彩色滤光片中;以及通过使用获得的像素阵列值产生长曝光彩色图像信号和短曝光参考图像信号,所述长曝光彩色图像信号和短曝光参考图像信号互相对齐。 Exposure time to obtain a pixel array value, the color pixels and the reference pixel included in the color filter; and the pixel array value obtained by using the long-exposure color image signal is generated and the short-exposure reference image signal, the long-exposure color image signal and the short-exposure reference image signal aligned with each other.

本发明还公开了一种图像处理设备,包括:图像获得单元,通过控制包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的彩色滤光片的曝光时间来获得互相对齐的长曝光彩色图像信号和短曝光参考图像信号;函数估计单元,通过使用长曝光彩色图像信号和短曝光参考图像信号来估计手抖函数;图像恢复单元,通过使用估计的手抖函数对长曝光彩色图像信号执行去巻积来产生恢复的图像信号。 The present invention also discloses an image processing apparatus comprising: an image acquisition unit, by controlling includes means for obtaining a color image color pixels for obtaining a color filter for hand-trembling function estimation reference image reference pixels exposure time to get aligned with each other in the long-exposure color image signal and the short-exposure reference image signal; function estimating unit, by using the long-exposure color image signal and the short-exposure reference image signal to estimate trembling function; image restoration unit, by using the estimated hand-trembling function of the long-exposure color image signal to perform convolving generating a restored image signal.

本发明还公开了一种处理图像的方法,该方法包括:通过控制包括用于获得彩色图像的彩色像素和用于获得用于手抖函数估计的参考图像的参考像素的彩色滤光片的曝光时间来获得互相对齐的长曝光彩色图像信号和短曝光参考图像信号;通过使用长曝光彩色图像信号和短曝光参考图像信号来估计手抖函数;以及通过使用估计的手抖函数对长曝光彩色图像信号执行去巻积来产生恢复的图像信号。 The present invention also discloses a method for processing an image, the method comprising: controlling includes means for obtaining a color image of the color pixels and the reference pixel used to obtain estimates for the hand-trembling function reference image of a color filter exposure time to get aligned with each other in the long-exposure color image signal and the short-exposure reference image signal; by using the long-exposure color image signal and the short-exposure reference image signal to estimate trembling function; and a hand-trembling function by using the estimated long-exposure color image convolving the signal to be generated to perform image signal recovery.

附图说明 Brief Description

被包括以提供对本发明进一步理解并合并于此的附图是本说明书的一部分,示出本发明的示例性实施例,并与本发明的描述一起用于解释本发明的各个方面。 Are included to provide a further understanding of the present invention and incorporated herein by reference is part of the specification, illustrate exemplary embodiments of the present invention, and together with the description, serve to explain the invention various aspects of the invention.

图1是示出根据本发明实施例的图像获得设备的框图;图2A和图2B示出根据本发明实施例的用于获得图像的方法的彩色滤光片阵列的配置; 1 is a block diagram showing an image obtaining apparatus according to an embodiment of the present invention; FIGS. 2A and 2B shows a configuration method for obtaining an image according to an embodiment of the present invention, a color filter array;

图3A和图3B是根据本发明实施解释例如何控制每个彩色滤光片的曝光时间的示图; 3A and 3B are to explain how to control the embodiment of the exposure time of each color filter according to an embodiment of the present invention shown in FIG;

图4示出如何从使用黑和白像素作为参考像素的彩色滤光片的像素阵列值产生具有相同分辨率的长曝光彩色图像和短曝光单色图像;图5是根据本发明实施例的图像处理设备的框图;图6是根据本发明实施例的使用像素阵列值获得恢复的图像的方法的流程图; Figure 4 illustrates a pixel array value of how to use the black and white pixels from the reference pixel to generate long-exposure color filter and a short-exposure color image having the same resolution monochrome image; FIG. 5 is an image in accordance with an embodiment of the present invention, block diagram of a processing apparatus; FIG. 6 is a flowchart of obtaining a restored image using a pixel array value according to an embodiment of the method of the present invention;

图7是根据本发明实施例的获得图像的方法的流程图;图8是根据本发明实施例的处理图像的方法的流程图;图9示出依照本发明的根据获得和处理图像的方法获得的图像。 FIG. 7 is a flowchart of a method of obtaining an image of the present embodiment of the invention; FIG. 8 is a flowchart of the image processing method according to the present embodiment of the invention; FIG. 9 shows a obtained in accordance with obtaining and processing images in accordance with the method of the present invention. images.

具体实施方式 DETAILED DESCRIPTION

以下参照示出本发明的示例性实施例的附图来更全面地描述本发明。 The following reference shows an exemplary embodiment of the present invention, reference to more fully describe the present invention. 在以下对本发明的描述中,当认为描述模糊本发明的观点时,将省略对该领域的技术人员共知的相关功能或结构的详细描述。 In the following description of the invention, when the detailed description of each function or structure of the present invention to describe the blur of view, will be omitted in this field of art known to think. 考虑本发明中的组件的功能来定义在此使用的术语,可根据使用者或操作者的意图或习惯改变所述术语。 The present invention contemplates the functional components to define the terms used herein, the terms can be changed according to the user or operator's intention or custom.

图1是示出根据本发明实施例的图像获得设备100的框图。 Figure 1 is a block diagram illustrating obtaining an image apparatus 100 according to an embodiment of the present invention. 参照图1,图像获得设备100包括传感器110、曝光控制器120和图像产生器130。 Referring to FIG. 1, the image obtaining apparatus 100 includes a sensor 110, the exposure controller 120 and the image generator 130.

可用以下等式1表示将图像输入到图像获得设备(诸如,数字相机)。 Using the following equation 1 shows an image input to the image obtained device (such as a digital camera).

等式l Equation l

B=IŪK B = IŪK

这里,B表示输入图像,I表示高质量图像,K指示模糊内核(blur kernel ),即是,手抖的点扩展函数(PSF), 0表示巻积操作。 Here, B denotes an input image, I represents a high quality image, K indicates blur kernel (blur kernel), that is, tremor of the point spread function (PSF), 0 represents convolving operation. 因此,需要准确估计模糊内核以获得高质量图像。 Therefore, in order to accurately estimate the blur kernel to obtain a high quality image.

图像获得设备100期望获得作为完全对齐的图像的长曝光图像和短曝光图像。 Image obtaining device 100 is desirable to obtain a perfectly aligned image of the long-exposure image and the short exposure images. 通过拍摄需要准确估计手抖的PSF的相同场景来获得长曝光图像和短曝光图像。 By shooting needs accurate estimates tremor same scene PSF to obtain long-exposure image and the short exposure images. 众所周知,如果用于PSF估计的获得的图像信号没有被正确对齐,则PSF估计的准确度会大幅地降低。 As we all know, if the estimated PSF obtained image signals are not correctly aligned, the estimated accuracy of the PSF will be substantially reduced.

传感器110通过获得经由形成彩色滤光片的像素发送的作为彩色信息的像素阵列值来感测图像。 Sensor 110 as the pixel array value obtained by the color information to be transmitted via the image sensing pixels forming a color filter. 传感器110包括感测图像的图像传感器,诸如电荷耦合器(CCD)或互补金属氧化物半导体(CMOS)。 An image sensor 110 includes a sensing an image sensor such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS). 彩色滤光片可如典型的数字相机一样被放置在图像传感器的光学元件的前方。 Color filter can be the same as a typical digital camera is placed in front of the image sensor of the optical element.

根据本发明的当前实施例,彩色滤光片可以是常规使用的彩色滤光片。 According to the current embodiment of the present invention, the color filter may be used in conventional color filters. 例如,包括在彩色滤光片中的彩色像素包括红(R)像素、绿像素(G)和蓝像素(B)。 For example, in a color filter including color pixels comprising red (R) pixels, green pixels (G) and blue pixel (B). 通常,由彩色滤光片获得的像素阵列值仅用于获得彩色图像。 Typically, the pixel array value obtained by the color filter is used only for obtaining a color image.

然而,在本发明中,控制用于包括在彩色滤光片中的像素的曝光时间以 However, in the present invention, the control for the pixels included in the color filter of the exposure time

7获得互相正确对齐的彩色图像和参考图像。 7 properly aligned with each other to obtain a color image and the reference image. 参考图像是被参考用于估计手抖 Reference picture is a reference used to estimate the tremor

(以下称为"手抖函数")的PSF的图像。 (Hereinafter referred to as "hand-trembling function") of the PSF images.

通过短曝光创建参考图像,从而将被用于获得参考图像的参考像素可以是能够发送具有高感光度的光的像素。 Create a reference image by a short exposure, which will be used to obtain the reference image reference pixels may be able to send a pixel having a high light sensitivity. 例如,参考像素可以是黑和白(W)像素或绿(G)像素。 For example, the reference pixel can be black and white (W) pixels or green (G) pixels. 然而,只要手抖函数可被有效地估计,彩色滤光片的类型或系统,或形成彩色滤光片的像素的颜色都可以以不同方式被改变。 However, as long as the hand-trembling function can be efficiently estimated, the color filter of the type or system, or the color of a pixel forming a color filter can be changed in various ways.

因此,通过使用由R、 G、 B和W像素形成的彩色滤光片的阵列,传感器110可获得三通道长曝光彩色像素阵列值和单通道短曝光单色阵列值。 Thus, by using a color filter array by the R, G, B and W pixels are formed, the sensor 110 can be obtained three-channel long-exposure color pixel array value and single short exposure monochrome array value.

曝光控制器120控制传感器110的彩色像素和参考像素的曝光时间。 The exposure controller 120 controls the color pixel sensor 110 and the reference pixel exposure time. 曝光控制器120同步彩色像素和参考像素的曝光开始时间,并控制彩色像素的曝光时间长于参考像素的曝光时间。 Exposure controller 120 to synchronize the color pixels and the reference pixel exposure starting time, and controls the exposure time of the color pixels the exposure time is longer than the reference pixel. 在这种情况下,由于在实质上短时间内获得参考图像,故曝光控制器120可控制传感器IIO感测具有高感光度的参考图像。 In this case, since the reference image obtained in a short time, in essence, it is the exposure controller 120 may control the sensor IIO sensing with high sensitivity of the reference image.

此外,曝光控制器120可控制传感器110,故在保持曝光彩色像素的同时,可重复发生参考像素的短曝光。 In addition, the exposure controller 120 may control the sensor 110, so that while maintaining the exposure of the color pixel, the pixel may be repeated occurrence of short-exposure reference. 可通过研究通过重复参考像素的短曝光获得的参考图像的改变的信息来估计手抖的轨迹,关于估计的手抖的轨迹的信息可被用作手抖函数的初始信息。 Change repeatable reference pixel obtained by short-exposure reference image information study to estimate the trajectory of camera shake, information about the estimated locus of hand trembling can be used as the initial information of the hand-trembling function.

图像产生器130从感测的像素阵列值产生互相正确对齐的长曝光彩色图像和短曝光参考图像。 Image generator 130 generates each other correctly aligned long-exposure color image and the short-exposure reference image from the pixel array value sensed. 才艮据本发明的当前实施例,图像产生器130对通过彩色像素和参考像素检测的像素阵列值执行插值,并产生彩色像素的各个颜色 It was Gen current embodiment of the present invention, an image generator 130 through the pixel array of color pixels and the reference pixel value interpolation is performed to detect and produce various colors of the color pixels

的长曝光参考图像信号和短曝光图像信号。 Long-exposure reference image signal and the short exposure image signal.

如上所述,根据本发明的当前实施例,由于可获得互相完全对齐且具有相同开始时间的长曝光彩色图像和短曝光参考图像,故从开始就避免了对齐错误,从而以后不需执行对齐图像的附加过程。 As described above, according to the current embodiment of the present invention, since the availability of perfectly aligned with each other and having a long-exposure color image and the short-exposure reference image the same start time, and so from the outset to avoid alignment errors, which do not need to perform the alignment after image Additional process.

图2A和图2B示出根据本发明实施例的用于获得图像的方法的彩色滤光片阵列的配置。 2A and 2B shows a configuration of a color filter array method for obtaining an image of an embodiment of the present invention.

图2A示出包括R、 G、 B和W像素的彩色滤光片阵列。 2A shows a color filter array including R, G, B and W pixels. 在图2A中,R、G和B像素被用作彩色像素,W像素被用作参考像素。 In FIG. 2A, R, G, and B pixels are used as color pixels, W pixels are used as reference pixels.

图2B示出包括R、 G和B像素的彩色滤光片阵列。 2B shows a color filter array including R, G and B pixels. 在图2B中,R、 Gl和B像素被用作彩色像素,G2像素被用作参考像素。 In Figure 2B, R, Gl, and B pixels are used as color pixels, G2 pixel is used as a reference pixel. 此外,在图2B中,由于Gl像素被用作彩色像素且G2像素被用作参考像素,故分别示出都是G像素的Gl和G2像素。 Further, in FIG. 2B, since the pixels are used as color pixels Gl and G2 pixel is used as a reference pixel, respectively, so that the G pixels are shown Gl and G2 pixels.

图3A和图3B是根据本发明实施例解释如何控制每个彩色滤光片的曝光时间的示图。 3A and 3B are embodiments of the present invention to explain how to control the exposure time of each color filter shown in FIG.

在图3A和图3B中,水平轴表示时间。 3A and 3B, the horizontal axis represents time. 如图3A所示,曝光控制器120将彩色像素(R、 G和B像素)和参考像素(W或G2像素)的曝光开始时间彼此同步,并控制彩色像素(R、 G和B像素)的曝光时间长于参考像素(W或G2像素)的曝光时间。 3A, the exposure controller 120 color pixels (R, G, and B pixels) and the reference pixel (W or G2 pixel) the exposure start time synchronization with each other, and controls the color pixels (R, G, and B pixels) the exposure time is longer than the reference pixel (W or G2 pixel) exposure time.

此外,为了获得如图3B中所示的手抖函数的初始信息,曝光控制器120可控制传感器110使得在曝光彩色像素的同时重复发生参考像素的短曝光。 Furthermore, in order to obtain initial information hand-trembling function as shown in FIG. 3B, the exposure controller 120 may control the sensor 110 such that short-exposure reference pixel repetition occurs while exposure color pixels.

图4示出如何从使用黑和白像素作为参考像素的彩色滤光片的像素阵列值产生具有相同清晰度的长曝光彩色图像和短曝光单色图像。 Figure 4 illustrates a pixel array value of how to use the black and white pixels from the reference pixel to generate long-exposure color filter of a color image and a short-exposure monochrome image with the same resolution.

如图4所示,拜尔型(Bayer-type)彩色滤光片可通过将从彩色像素阵列410获得的像素阵列值插值来产生三通道长曝光RGB彩色图像420和单通道黑白图像430。 As shown, the Bayer type (Bayer-type) color filter can be produced by interpolation from the color pixel array value obtained by four pixel array 410 of three-channel long-exposure RGB color image 420 and a single-channel 430 black and white images.

图5是^f艮据本发明实施例的图像处理设备500的框图。 Figure 5 is a block diagram of an image processing apparatus according to the present invention ^ f Burgundy embodiment 500.

图像处理设备500包括图像获得单元510、函数估计单元520和图像恢复单元530。 The image processing device 500 includes an image obtaining unit 510, the function estimating unit 520 and the image restoration unit 530.

图像获得单元510具有与图1中的图像获得设备100的结构相应的结构。 Image obtaining unit 510 has a structure corresponding to the structure of the device 100 to obtain the image in Figure 1. 图像获得单元510具有包括用于获得彩色图像的彩色像素和用于获得用于估计手抖函数的参考图像的参考像素的彩色滤光片,控制彩色像素的曝光时间和参考像素的曝光时间以获得互相正确对齐的长曝光彩色图像信号和短曝光参考图像信号。 Image obtaining unit 510 for obtaining a color image having including color pixels for obtaining a color filter for estimating a hand-trembling function of reference pixels of the reference image, the control color pixels and the reference pixel exposure time exposure time to obtain correctly aligned with each other long-exposure color image signal and the short-exposure reference image signal.

图像获得单元510同步彩色像素和参考像素的曝光开始时间,并控制彩色像素的曝光时间长于参考像素的曝光时间,以获得长曝光彩色图像信号和短曝光参考图像信号。 Image obtaining unit 510 to synchronize the color pixels and the reference pixel exposure starting time, and controls the exposure time of the color pixels of the reference pixel is longer than the exposure time, to obtain a long-exposure color image signal and the short-exposure reference image signal.

函数估计单元520使用长曝光彩色图像信号和短曝光参考图像信号估计手抖函数。 Function estimating unit 520 using the long-exposure color image signal and the short-exposure reference image signal estimation trembling function. ,

根据本发明的当前实施例,可通过最小化对象函数的等式(诸如,以下满足1^>0且5>,=1的等式2)计算手抖函数。 According to the current embodiment of the present invention, by minimizing the equation object functions (such as to meet the following 1 ^> 0 and 5>, = 1, equation 2) calculated hand-trembling function.

等式2 '' Equation 2 ''

A:* = argmin|5 _ / Ū《||2 +剩|2这里,2J&f是用于稳定解的Tikhonov正则化项,且参数入可被固定为3。 A: * = argmin | 5 _ / Ū "|| 2 + left | 2 here, 2J & f is a stable solution of Tikhonov regularization term, and the parameters can be fixed into 3. 然而,Tikhonov正则化项仅是示例,能够表示手抖函数的特点的任何其它项也能够被用作为正则化项。 However, Tikhonov regularization term is only an example, it can represent the characteristics of hand-trembling function of any other items can also be used as a regularization term. 此外,参数可以是固定值,但是也可以是在优化期间在每次重复时改变的值。 In addition, the parameter may be a fixed value, but may be changed in each repetition period value is optimized.

根据本发明的当前实施例,表示长曝光彩色图像信号的值取代B,表示短曝光参考图像信号的值取代I,从而可估计手抖函数。 According to the current embodiment of the present invention, showing the long-exposure color image signal values substituents B, represents the value of short-exposure reference image signal is substituted I, so that the hand-trembling function can be estimated.

同时,根据本发明的当前实施例,在曝光彩色像素的同时,通过重复发生参考像素的短曝光来获得参考图像,关于参考图像的信息被用作手抖函数的初始信息,从而可更准确和快速地估计手抖函数。 Meanwhile, according to the current embodiment of the present invention, while the color pixels in the exposure, by short exposure recurring reference pixel to obtain the reference picture, information about the reference images is used as initial information of the hand-trembling function, and thus can be more accurately and quickly estimated hand-trembling function.

为此,图像获得单元510通过在曝光彩色像素的同时重复参考像素的短曝光来收集多个短曝光参考图像信号。 For this reason, the image obtained by the exposure unit 510 repeatedly while the color pixels of the reference pixel short exposure to collect a plurality of short-exposure reference image signal. 随后,由于获得已经以预定时间间隔拍摄的参考图像的信号,故函数估计单元520通过使用收集的参考图像信号根据运动估计方法来追踪手抖的轨迹。 Then, since the signal has been obtained at a predetermined time interval with reference images taken, therefore a reference image signal function estimation unit 520 by using the collected according to the motion estimation method to track the trajectory of hand-shake. 追踪的手抖的轨迹被用作手抖函数的初始信息,从而可更准确和快速地估计手抖函凄t。 Tremor trajectory tracking is used as initial information trembling function, which can more accurately and quickly assess tremor sad letter t.

图像恢复单元530使用估计的手抖函数对通过图像获得单元510获得的长曝光彩色图像信号执行去巻积,并产生恢复图像信号。 Image restoration unit 530 using the estimated hand-trembling function by the image obtaining unit 510 obtains long-exposure color image signal to convolving execution and generating a restored image signal.

此外,根据本发明的当前实施例,可对用于估计手抖函数的长曝光彩色图像信号和短曝光参考图像信号执行以下处理。 In addition, according to the current embodiment of the present invention, it can be used for a long hand-trembling function estimation exposure color image signal and the short-exposure reference image signal to perform the following processing.

首先,函数估计单元520减小长曝光彩色图傳_信号和短曝光参考图像信号的噪声。 First, the function estimating unit 520 to reduce the long-exposure color image signal and pass _ short-exposure reference image signal noise. 随后,函数估计单元520将减噪的长曝光彩色图像信号转换为长曝光单通道图像信号,以统一用于估计手抖函数的图像信号的通道数。 Subsequently, the noise reduction function estimating unit 520 of the long-exposure color image signal into a long-exposure single-channel image signal for the number of channels in a uniform image signal estimated hand-trembling function.

可使用各种方法统一图像信号的通道数。 The number of channels can be used in various ways unified image signal. 例如,在长曝光彩色图像中, 可使用三个通道中的一个(例如,红、绿和蓝)的图像,或RGB图像的平均图像可被产生为单通道图像。 For example, the long-exposure color image, you can use one of three channels (for example, red, green, and blue), the average picture images, or RGB image can be generated as a single-channel image. 或者,RGB图像可被转换为灰度图以产生单通道图像。 Alternatively, RGB images can be converted to grayscale to generate single-channel image.

接下来,函凄t估计单元520通过使用转换的长曝光单通道图像信号和减噪的短曝光参考图像信号估计手抖函数。 Next, the letter sad t estimation unit 520 by using a long exposure single-channel image signal conversion and noise reduction of short-exposure reference image signal estimation trembling function.

图6是根据本发明实施例的使用像素阵列值获得恢复的图像的方法的流程图。 FIG 6 is a flowchart of obtaining a restored image using a pixel array value according to an embodiment of the method of the present invention. 在图6中,对通过使用如图4所示的RGBW像素阵列产生的图像执行处理。 In FIG. 6, by using the RGBW pixel of the array shown in FIG. 4 performs processing to generate an image. 根据本发明的当前实施例,通过使用RGBW像素阵列获得的单色(W) 像素阵列和RGB像素阵列互相完全对齐。 According to the current embodiment of the present invention, the color obtained by using the RGBW pixel array (W) pixel array and an RGB pixel array perfectly aligned with each other. 将从单色像素阵列获得的像素阵列值插值以产生短曝光黑白图像 Interpolating the pixel array value obtained from the monochrome pixel array to produce a short-exposure monochrome image

img—white—SET,即是,参考图像(操作S610 )。 img-white-SET, that is, the reference image (operation S610). 随后,将从RGB像素阵列获得的像素阵列值插值以产生长曝光彩色图像imgjgb—LET (操作S620)。 Subsequently, the pixel array value interpolation obtained from the RGB pixel array to generate a long-exposure color image imgjgb-LET (operation S620).

减小单通道黑白图像的噪声以获得减噪的短曝光黑白图像img—white_SET_NR (操作S630 )。 Black and white image of a single channel to reduce noise to obtain noise reduction of short-exposure black and white image img-white_SET_NR (operation S630). 此外,同样通过减小长曝光彩色图像img—rgb—LET的噪声获得减噪的长曝光彩色图像img—rgb—LET—NR (操作S640)。 In addition, also by reducing the long-exposure color image img-rgb-LET noise gain long exposure noise reduction color image img-rgb-LET-NR (operation S640). 在这种情况中,考虑到短曝光黑白图像具有短曝光时间,从而具有比长曝光彩色图像更多的噪声的事实,对短曝光黑白图像使用噪声减小滤波器。 In this case, taking into account the short-exposure black and white image has a short exposure time, thus having a long-exposure color image more than the noise of the fact that the use of short-exposure monochrome image noise reduction filter.

由于长曝光彩色图像img—rgb—LET—NR具有三通道且短曝光黑白图像img—white—SET—NR具有单通道,故长曝光彩色图像img_rgb—LET一NR和短曝光黑白图像img—white一SET一NR需要具有相同的通道数以估计手抖的PSF, 即,手抖函数。 Due to long exposure color image img-rgb-LET-NR has three channels and the short-exposure black and white image img-white-SET-NR has a single channel, so the long-exposure color image img_rgb-LET NR and a short-exposure black and white image img-white one SET a NR have the same number of channels required to estimate the tremor PSF, namely, trembling function. 为此,在长曝光彩色图像img—rgb一LET—NR上执行灰度转换以产生灰度图象img—gray—LET (操作S650 )。 To do this, it performs gradation conversion on the long-exposure color image img-rgb LET-NR to produce a gray-scale image img-gray-LET (operation S650).

随后,使用减噪的短曝光黑白图像img—white—SET—NR和灰度图象img—gray—LET估计手抖的PSF (操作S660 )。 Subsequently, the noise-reduced short-exposure black and white image img-white-SET-NR and grayscale image img-gray-LET estimated tremor PSF (operation S660). 使用估计的PSF和已经获得的长曝光彩色图像img—rgb—LET—NR产生恢复的彩色图像(或恢复的RGB图像) (操作S670 )。 PSF and using the estimated long-exposure color image img-rgb-LET-NR to produce a color image recovery (or restoration of RGB images) (operation S670) has been obtained.

图7是根据本发明实施例的获得图像的方法的流程图。 FIG. 7 is a flowchart of a method embodiment of the present invention, the image obtained in the embodiment. 通过控制包括在彩色滤光片中的彩色像素和参考像素的曝光时间获得像素阵列值(操作S710)。 Obtaining a pixel array value (operation S710) by controlling the color filter includes color pixels and the reference pixel exposure times. 在这种情况下,彩色像素用于获得彩色图像且参考像素用于获得参考图像以进行手抖函数估计。 In this case, the color pixels for obtaining a color image and a reference pixel for obtaining a reference image for hand-trembling function estimation. 为了在操作S710中控制曝光时间,使彩色像素和参考像素的曝光开始时间彼此同步,且彩色像素的曝光时间被设置为长于参考像素的曝光时间。 In order to control the exposure time in operation S710, so that the color pixels and the reference pixel exposure start time synchronization with each other, and the color pixels is set to be longer than the exposure time of the reference pixel exposure time.

通过使用获得的像素阵列值获得互相正确对齐的长曝光彩色图像信号和短曝光参考图像信号(操作S720)。 By using the obtained pixel array value obtained correctly aligned with each other long-exposure color image signal and the short-exposure reference image signal (operation S720). 根据本发明的当前实施例,在操作S720, 可对从彩色像素和参考像素获得的像素阵列值执行插值,以产生彩色像素的各个彩色的长曝光图像信号和短曝光图像信号。 Long-exposure image signal according to the current embodiment of the present invention, at operation S720, to perform interpolation on the pixel array value obtained from the color pixels and the reference pixel to generate respective color color pixels and a short-exposure image signal.

图8是根据本发明实施例的处理图像的方法的流程图。 Figure 8 is a flowchart of a method for processing an image according to the present invention embodiment. 控制包括在彩色滤光片中的彩色像素和参考像素的曝光时间以获得互相正确对齐的长曝光彩色图像信号和短曝光参考图像信号(操作S810)。 Control is included in the color filter of the color pixels and the reference pixel exposure times for each correctly aligned long-exposure color image signal and the short-exposure reference image signal (operation S810). 在这种情况中,彩色像素用于获得彩色图像且参考像素用于获得参考图像以进行 In this case, the color pixels for obtaining a color image and a reference pixel for obtaining a reference image for

ii手抖函数估计。 ii trembling function estimation. 此外,使彩色像素和参考像素的曝光开始时间彼此同步,且彩色像素的曝光时间被设置为长于参考像素的曝光时间。 In addition, so that the color pixels and the reference pixel exposure start time synchronization with each other, and the exposure time of the color pixels is set to be longer than the exposure time of the reference pixel.

通过使用长曝光彩色图像信号和短曝光参考图像信号来估计手抖函数 By using the long-exposure color image signal and the short-exposure reference image signal estimated hand-trembling function

(操作S820 )。 (Operation S820).

通过使用估计的手抖函数对长曝光彩色图像信号执行去巻积以产生恢复的图像信号(操作S830)。 By using the estimated hand-trembling function to execute the long-exposure color image signal to generate a recovery to convolving the image signal (operation S830).

在操作S810,在曝光彩色像素的同时,通过重复参考像素的短曝光来收集多个短曝光参考图像信号。 In operation S810, while the color pixels in the exposure, by repeating the short-exposure reference pixels to collect a plurality of short-exposure reference image signal. 随后,在操作S820,使用获得的参考图像信号来追踪手抖的轨迹,所述轨迹信息可被用作手抖函数的初始信息。 Subsequently, the reference image signal at operation S820, using the obtained to track the trajectory of the hand-trembling, the trajectory information may be used as the initial information of the hand-trembling function. 因此,可更准确和快速地估计手抖函数。 Therefore, more accurate and rapid estimation of hand-trembling function.

在操作S820,长曝光彩色图像信号和短曝光参考图像信号的噪声被减小。 In operation S820, the long-exposure color image signal and the short-exposure reference image signal noise is reduced. 为了统一长曝光彩色图像信号和短曝光参考图像信号的通道数,将减噪的长曝光彩色图像信号转换为长曝光单通道图像信号。 In order to unify the long-exposure color image signal and the short-exposure reference image signal, the number of channels, the noise of the long-exposure color image signal into a long-exposure single-channel image signal. 随后,转换的长曝光单通道图像信号和减噪的短曝光参考图像信号用于估计手抖函数。 Subsequently, the converted long-exposure single-channel image signal and the noise-reduced short-exposure reference image signal used for hand-trembling function estimation.

图9示出依照本发明的根据获得和处理图像的方法获得的图像。 Figure 9 shows an image obtained in accordance with the acquisition and processing images according to the method of the present invention.

图像910是高感光度短曝光黑白图像,图像920是低感光度长曝光彩色图像(图9中用黑白示出)。 910 is a high-sensitivity images in black and white short-exposure image, the image 920 is a low-speed long-exposure color image (shown in FIG. 9 by black and white). 如果具有对齐错误,则不能够进行准确的PSF 估计,这会引起严重的伪影(artifact),致使如图像940的低质量图像恢复。 If you have an alignment error, it is not possible to accurately estimate the PSF, which can cause severe artifacts (artifact), resulting in low-quality images such as the image 940 recovery.

然而,根据本发明的当前实施例,当使用高感光度短曝光黑白图像和低感光度彩色图像时,可解决由于对齐错误引起的手抖函数的不准确估计的问题。 However, the problem according to the current embodiment of the present invention, when a high-speed short-exposure black and white image and a low-sensitivity color image, solve trembling function due to alignment error caused by inaccurate estimates.

当比较通过根据本发明的图像处理方法的恢复的彩色图像930与通过使用不准确的手抖函数恢复的彩色图像940时,显而易见地,根据本发明恢复的图像930比图像940具有更高的质量。 When comparing the color image 930 restored by the image processing method of the present invention, the color image 940 by using the inaccurate hand-trembling function recovery apparent that, in accordance with the present invention, a higher quality of the restored image 930 than the image 940 .

根据本发明,控制包括在彩色滤光片中的彩色像素和参考像素的曝光时间,从而具有相同图像获得开始时间的长曝光彩色图像信号和短曝光参考图像信号互相完全对齐。 According to the present invention, the control is included in the color filter of the color pixels and the reference pixel exposure time, so as to have the same image obtaining starting time of the long-exposure color image signal and the short-exposure reference image signal is perfectly aligned with each other. 使用所述长曝光彩色图像信号和短曝光参考图像信号, 从根本上避免了对齐错误,从而能够进行手抖函数的准确估计。 Using the long-exposure color image signal and the short-exposure reference image signal, avoiding alignment errors and fundamentally, thus enabling accurate estimation of hand-trembling function.

此外,通过使用根据本发明估计的手抖函数,可恢复清晰的彩色图像。 In addition, by using the present invention estimated hand-trembling function, restore clear color images. 根据本发明,处理图像的设备和方法可用于在夜晚拍摄景色、实施夜晚监控、 拍摄室内场景等。 According to the present invention, apparatus and method for processing an image it can be used to shoot scenery at night, monitoring the implementation of the night, shoot indoor scenes and so on. 本领域的技术人员应该理解:在不脱离本发明的精神和范围的情况下, 可以对本发明进行各种修改和改变。 Those skilled in the art would understand: without departing from the spirit and scope of the present invention may be made to the present invention, various modifications and changes. 因此,本发明覆盖提供的这些本发明的修改和改变,该修改和改变落入附加权利要求和其同等物的范围内。 Accordingly, the present invention cover the modifications and variations of this invention provided fall within the appended claims and their equivalents in the scope of this modification and change.

Classifications
International ClassificationH04N9/04
Cooperative ClassificationG06T2207/10024, G06T5/002, G06T2207/20201, G06T5/003, H04N5/23248
European ClassificationG06T5/00D, H04N5/232S
Legal Events
DateCodeEventDescription
29 Jul 2009C06Publication
12 Jan 2011C10Request of examination as to substance
24 Apr 2013C14Granted