US20110019040A1 - Image sensor and image capture system with extended dynamic range - Google Patents
Image sensor and image capture system with extended dynamic range Download PDFInfo
- Publication number
- US20110019040A1 US20110019040A1 US12/898,809 US89880910A US2011019040A1 US 20110019040 A1 US20110019040 A1 US 20110019040A1 US 89880910 A US89880910 A US 89880910A US 2011019040 A1 US2011019040 A1 US 2011019040A1
- Authority
- US
- United States
- Prior art keywords
- color filter
- pixels
- integration time
- pattern
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
- H04N25/58—Control of the dynamic range involving two or more exposures
- H04N25/581—Control of the dynamic range involving two or more exposures acquired simultaneously
- H04N25/583—Control of the dynamic range involving two or more exposures acquired simultaneously with different integration times
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
Definitions
- the present invention pertains to semiconductor-based image sensors with increased dynamic range.
- Solid state image sensors are now used extensively in many types of image capture applications.
- the two primary image sensor technologies utilized are Charge Coupled Devices CCD and CMOS x-y addressable devices.
- CCD Charge Coupled Device
- CMOS x-y addressable devices There exists many different specific embodiments of both technologies, including Active Pixel Sensors (APS) and Passive Pixel Sensors (PPS) for CMOS x-y addressable devices. All are basically comprised of a set or array of photodetectors that convert incident light into an electrical signal that can be readout and used to construct an image correlated to the incident light pattern.
- the exposure or integration time for the array of photodetectors can be controlled by well known mechanisms.
- the signal represents the amount of light incident upon a pixel photosite.
- the dynamic range (DR) of an imaging sensing device is defined as the ratio of the effective maximum detectable signal level, typically referred to as the saturation signal, (V sat ), with respect to the rms. noise level of the sensor, ( ⁇ noise ). This is shown in Equation 1.
- Image sensor devices such as charge coupled devices (CCD) that integrate charge created by incident photons have dynamic range limited by the amount of charge that can be collected and held in a given photosite, (V sat ).
- V sat the amount of charge that can be collected and detected in a pixel is proportional to the pixel area.
- DSC digital still camera
- the number of electrons representing Vsat is on the order of 13,000 to 20,000 electrons. If the incident light is very bright and creates more electrons that can be held in the pixel or photodetector, these excess electrons are extracted by the anti-blooming mechanism in the pixel and do not contribute to an increased saturation signal.
- the maximum detectable signal level is limited to the amount of charge that can be held in the photodetector or pixel.
- the DR is also limited by the sensor noise level, ⁇ noise . Due to the limitations on Vsat, much work has been done in CCD's to decrease ⁇ noise to very low levels. Typically, commercial megapixel DSC devices have a DR of 1000:1 or less.
- V sat is limited by the amount of charge that can be held and isolated in the photodetector. Excess charge is lost. This can become even more problematic with APS and PPS compared to CCD due to the active and passive components within the pixel, limiting the area available for the photodetector, and due to the low voltage supply and clocks used in CMOS devices.
- APS devices have been used to provide image sensor systems on a chip, the digital and analog circuits used on APS devices such as timing and control and analog to digital conversion, that are not present on CCD's, provide a much higher noise floor on APS devices compared to CCD. This is due to higher temporal noise as well as possibly quantization noise from the on-chip analog to digital converter.
- Prior art APS devices also suffer from poor sensitivity to light due to the limited fill factor induced by integration of active components in the pixel, and by loss of transmission of incident light through the color filter layer placed above the pixel.
- the present invention provides a means to control the integration separately for any given spatial pattern on the image sensor, and more specifically for a pattern that is compatible with one or two dimensions of the kernel in the CFA pattern. This is done by providing separate TG or RG busses for pixels in a given row or set of rows, or by providing any means to control integration time separately for a given pattern of pixels in the image sensor array. By doing so, valid data is always available for the dark and bright regions of an image simultaneously.
- FIG. 1 a is a prior art pixel array
- FIG. 1 b is another prior art pixel array
- FIG. 2 a is a pixel array of the present invention
- FIG. 2 b is an alternative embodiment of the present invention.
- FIG. 3 is graph graphically illustrating the implementation of FIGS. 2 a and 2 b ;
- FIG. 4 a is an illustration of two integration control lines per row
- FIG. 4 b is an illustration of one integration time signal line per row.
- FIG. 5 is a camera for implementing the pixel array of FIGS. 2 a and 2 b into a preferred commercial embodiment.
- FIGS. 1 a and 1 b Typical prior art image sensor pixel arrays are shown in FIGS. 1 a and 1 b .
- the image sensor in FIG. 1 a can be of any technology type such as CCD or CMOS APS.
- the pixel array 10 in FIG. 1 a comprises a set of photodetectors.
- the integration time is constant for each pixel. The drawback of this approach is that if the integration time is long, pixels in the bright areas of an image will become saturated and the image details in the bright region will be lost. If the integration time is chosen to be short, the image quality in dark regions of the image will be poor due to low signal and high noise.
- the image sensor in FIG. 1 b was disclosed in U.S. patent application Ser. No. 08/960,418, filed Jul.
- the image sensor pixel array 30 of the present invention includes an array that facilitates different programmable integration times, but in a different spatial pattern than that shown in FIG. 1 b .
- an x-y addressable CMOS image sensor this can be accomplished with separate transfer gates or reset gates.
- For a CCD image sensor this can be accomplished by having separate transfer gates.
- the image sensor pixel array 30 in FIG. 2 a is constructed to have pixels with two different integration times for mated pairs of rows 40 a and 40 b that are correlated with the color filter array pattern pitch or kernel. Pixels with long integration times are referred to as fast pixels. Pixels with short integration times are referred to as slow pixels.
- the effective dynamic range of the image sensor is extended as shown in FIG. 3 .
- region 1 low light level region
- both the slow and fast pixels of the sensor have not saturated.
- the fast pixels will have signal levels that are well above the noise floor.
- the slow pixels will have signal levels that are within a predetermined ratio compared to the sensor noise floor.
- region 2 both the slow and fast pixels have not saturated, and both have adequate signal-to-noise ratio.
- region 3 high light level regions, the fast pixels have saturated or clipped and do not contain valid signal level information.
- the slow pixels have not saturated and do contain valid signal level information with adequate signal to noise ratio.
- the missing information from the fast pixels can be determined by interpolation of the slow pixels.
- a single frame capture is taken, and spatially adaptive image processing performed.
- region 2 standard prior art color image processing methods are employed to render an image.
- interpolation of the slow pixels is used to determined the missing signal information in the fast pixels.
- Isat saturation illumination level
- the sensor architecture of FIG. 2 a is designed to provide an integration time pattern with two rows of a first integration time, and the two adjacent rows with a second integration time.
- This can be accomplished with any type of image sensor by having multiple or separate controls for integration time in this pattern.
- CMOS and other x-y addressable image sensors this can be accomplished simply by having the image sensor timing arranged with two separate sets of integration pointers that are applied to the pairs of alternating rows signal lines that control integration time.
- this requires that the transfer gate interconnects are constructed so that there are separate and isolated connections to the transfer gate lines for at least alternating pairs of rows.
- a second embodiment of the present invention is shown in the array in FIG. 2 b .
- the sensor array 50 is constructed to have two separate and programmable integration times in a 2 by 2 pixel pattern 60 a and 60 b .
- this is achieved by having multiple signal lines per row that are used to control integration time, such as transfer gate or reset gate. These multiple signal lines per row are connected to alternating pairs of pixels to produce the integration time pattern shown in FIG. 2 b.
- FIG. 4 a the routing of the multiple signal lines 70 that control integration time is shown.
- One disadvantage with routing multiple signal lines 70 to control integration time for each row is reduction of fill factor or a larger pixel size in order to fit the extra signal lines into the pixel pitch. This is overcome by the signal line routing architecture shown in FIG. 4 b .
- a single integration time control line 80 is used per row, but it is actually routed to pixels in two adjacent rows.
- the signal line 80 in the adjacent row is routed in a similar manner to create the integration time pattern shown in FIG. 2 b .
- this approach although a single row of data is readout from the sensor at one time, the pixels contained within the data stream are from physically adjacent rows in the array.
- the interlaced data must be corrected in the camera image memory. This is also a feature of the present invention. Since either on-chip or in-camera memory can be set up to write data into two or more row locations, there is no need to have the sensor read out all pixels from a physical row at the same time.
- this provides an image sensor and image capture system with wide intra-scene dynamic range and wide exposure latitude.
- a single image capture can render a full range of image information with optimization of the integration time for low light levels without clipping signal information in the high light regions of an image. This can greatly simplify the exposure control system and algorithms in an imaging system since choice of exposure or integration time does not need to be as precise.
- an image capture system using such a sensor can be used to measure or determine the dynamic range of a scene to set the two integration times appropriately.
- two widely separated integration times can be used to determine the maximum and minimum light levels in the scene.
- the two integration times can then be adjusted to cover the range of illumination in the scene. For example, if the dynamic range of the scene to be captured is within the inherent dynamic range of the image sensor, then the two integration times can be set to the same value. If the scene contains a dynamic range that is wider than the true dynamic range of the sensor, then the two integration times can be set to match or optimally cover the dynamic range of the scene.
- a camera 90 for implementing the image sensor of the present invention is one of many consumer-oriented commercial embodiments.
Abstract
An image sensor includes a plurality of pixels; a color filter pattern spanning at least a portion of the pixels, wherein the color filter pattern forms a color filter kernel having colors in a predetermined arrangement; and a mechanism for controlling integration time of the pixels, wherein the integration time of the plurality of pixels is spatially variant in a pattern that is correlated with the color filter array kernel.
Description
- This is a continuation application of U.S. application Ser. No. 10/654,313 filed Sep. 3, 2003.
- The present invention pertains to semiconductor-based image sensors with increased dynamic range.
- Solid state image sensors are now used extensively in many types of image capture applications. The two primary image sensor technologies utilized are Charge Coupled Devices CCD and CMOS x-y addressable devices. Currently, there exists many different specific embodiments of both technologies, including Active Pixel Sensors (APS) and Passive Pixel Sensors (PPS) for CMOS x-y addressable devices. All are basically comprised of a set or array of photodetectors that convert incident light into an electrical signal that can be readout and used to construct an image correlated to the incident light pattern. The exposure or integration time for the array of photodetectors can be controlled by well known mechanisms. The signal represents the amount of light incident upon a pixel photosite. The dynamic range (DR) of an imaging sensing device is defined as the ratio of the effective maximum detectable signal level, typically referred to as the saturation signal, (Vsat), with respect to the rms. noise level of the sensor, (σnoise). This is shown in
Equation 1. -
Dynamic Range=Vsat /θnoise Equation 1 - Image sensor devices such as charge coupled devices (CCD) that integrate charge created by incident photons have dynamic range limited by the amount of charge that can be collected and held in a given photosite, (Vsat). For example, for any given CCD, the amount of charge that can be collected and detected in a pixel is proportional to the pixel area. Thus for a commercial device used in a megapixel digital still camera (DSC), the number of electrons representing Vsat is on the order of 13,000 to 20,000 electrons. If the incident light is very bright and creates more electrons that can be held in the pixel or photodetector, these excess electrons are extracted by the anti-blooming mechanism in the pixel and do not contribute to an increased saturation signal. Hence, the maximum detectable signal level is limited to the amount of charge that can be held in the photodetector or pixel. The DR is also limited by the sensor noise level, θnoise. Due to the limitations on Vsat, much work has been done in CCD's to decrease θnoise to very low levels. Typically, commercial megapixel DSC devices have a DR of 1000:1 or less.
- The same limitations on DR also exist for APS and PPS devices. The Vsat is limited by the amount of charge that can be held and isolated in the photodetector. Excess charge is lost. This can become even more problematic with APS and PPS compared to CCD due to the active and passive components within the pixel, limiting the area available for the photodetector, and due to the low voltage supply and clocks used in CMOS devices. In addition, since APS devices have been used to provide image sensor systems on a chip, the digital and analog circuits used on APS devices such as timing and control and analog to digital conversion, that are not present on CCD's, provide a much higher noise floor on APS devices compared to CCD. This is due to higher temporal noise as well as possibly quantization noise from the on-chip analog to digital converter.
- In commonly assigned U.S. Pat. No. 6,069,377,issued May 30, 2000, entitled IMAGE SENSOR INCORPORATING SATURATION TIME MEASUREMENT TO INCREASE DYNAMIC RANGE, by Prentice et al., Prentice discloses the prior art approaches to extending dynamic range of APS devices, and discloses a new invention to extend dynamic range. This method has the disadvantage of requiring more than four transistors per pixel and limits the size of the pixel that can be made. In U.S. Pat. No. 6,307,195, issued Oct. 23, 2001, entitled VARIABLE COLLECTION OF BLOOMING CHARGE TO EXTEND DYNAMIC RANGE, and U.S. Pat. No. 6,486,504, issued Nov. 26, 2002, entitled CMOS IMAGE SENSOR WITH EXTENDED DYNAMIC RANGE, both by Guidash, Guidash discloses extending dynamic range by collection of the charge that blooms from the photodetector, and by co-integration of the photodetector and floating diffusion within a single pixel. These approaches have the potential disadvantage of spatial variation of the photodetector saturation level contributing to fixed pattern noise in the sensor, and does not increase the sensitivity of the sensor.
- Prior art APS devices also suffer from poor sensitivity to light due to the limited fill factor induced by integration of active components in the pixel, and by loss of transmission of incident light through the color filter layer placed above the pixel.
- From the foregoing discussion it should be apparent that there remains a need within the prior art for a device that retains extended dynamic range while retaining low fixed pattern noise, small pixel, and high sensitivity.
- The present invention provides a means to control the integration separately for any given spatial pattern on the image sensor, and more specifically for a pattern that is compatible with one or two dimensions of the kernel in the CFA pattern. This is done by providing separate TG or RG busses for pixels in a given row or set of rows, or by providing any means to control integration time separately for a given pattern of pixels in the image sensor array. By doing so, valid data is always available for the dark and bright regions of an image simultaneously.
- These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.
-
FIG. 1 a is a prior art pixel array; -
FIG. 1 b is another prior art pixel array; -
FIG. 2 a is a pixel array of the present invention; -
FIG. 2 b is an alternative embodiment of the present invention; -
FIG. 3 is graph graphically illustrating the implementation ofFIGS. 2 a and 2 b; -
FIG. 4 a is an illustration of two integration control lines per row; -
FIG. 4 b is an illustration of one integration time signal line per row; and -
FIG. 5 is a camera for implementing the pixel array ofFIGS. 2 a and 2 b into a preferred commercial embodiment. - Typical prior art image sensor pixel arrays are shown in
FIGS. 1 a and 1 b. The image sensor inFIG. 1 a can be of any technology type such as CCD or CMOS APS. Thepixel array 10 inFIG. 1 a comprises a set of photodetectors. The integration time is constant for each pixel. The drawback of this approach is that if the integration time is long, pixels in the bright areas of an image will become saturated and the image details in the bright region will be lost. If the integration time is chosen to be short, the image quality in dark regions of the image will be poor due to low signal and high noise. The image sensor inFIG. 1 b was disclosed in U.S. patent application Ser. No. 08/960,418, filed Jul. 17, 2002, entitled ACTIVE PIXEL SENSOR WITH PROGRAMMABLE COLOR BALANCE, by Guidash, in which each color of thepixel array 20 associated with the CFA pattern has a separate integration time to achieve charge domain white balance. This has the same drawbacks as those cited for the image sensor pixel array inFIG. 1 a. - Referring to
FIG. 2 a, the imagesensor pixel array 30 of the present invention includes an array that facilitates different programmable integration times, but in a different spatial pattern than that shown inFIG. 1 b. For an x-y addressable CMOS image sensor this can be accomplished with separate transfer gates or reset gates. For a CCD image sensor this can be accomplished by having separate transfer gates. The imagesensor pixel array 30 inFIG. 2 a is constructed to have pixels with two different integration times for mated pairs ofrows FIG. 3 . Inregion 1, low light level region, both the slow and fast pixels of the sensor have not saturated. The fast pixels will have signal levels that are well above the noise floor. The slow pixels will have signal levels that are within a predetermined ratio compared to the sensor noise floor. Inregion 2, both the slow and fast pixels have not saturated, and both have adequate signal-to-noise ratio. Inregion 3, high light level regions, the fast pixels have saturated or clipped and do not contain valid signal level information. The slow pixels have not saturated and do contain valid signal level information with adequate signal to noise ratio. Since the valid information is correlated with the CFA pattern, the missing information from the fast pixels can be determined by interpolation of the slow pixels. With the separate integration time architecture shown inFIG. 3 , a single frame capture is taken, and spatially adaptive image processing performed. Inregion 2, standard prior art color image processing methods are employed to render an image. For an area of pixels in the image capture that fall intoregion 3, interpolation of the slow pixels is used to determined the missing signal information in the fast pixels. This results in a loss of true MTF in the extremely bright areas of the image, but leads to an effectively higher saturation illumination level, Isat. This effectively extends the intra-scene dynamic range of the image sensor. Although true spatial resolution is degraded in the extreme bright regions, the image content that would otherwise be lost in the image capture is preserved. - The sensor architecture of
FIG. 2 a is designed to provide an integration time pattern with two rows of a first integration time, and the two adjacent rows with a second integration time. This can be accomplished with any type of image sensor by having multiple or separate controls for integration time in this pattern. For CMOS and other x-y addressable image sensors this can be accomplished simply by having the image sensor timing arranged with two separate sets of integration pointers that are applied to the pairs of alternating rows signal lines that control integration time. This could be transfer gate lines in each row, or reset gates lines in each row, or any other per row signal that is used to control integration time for that row. In the case of CCD image sensors, this requires that the transfer gate interconnects are constructed so that there are separate and isolated connections to the transfer gate lines for at least alternating pairs of rows. - A second embodiment of the present invention is shown in the array in
FIG. 2 b. In this embodiment, thesensor array 50 is constructed to have two separate and programmable integration times in a 2 by 2pixel pattern FIG. 2 b. - Referring to
FIG. 4 a, the routing of themultiple signal lines 70 that control integration time is shown. One disadvantage with routingmultiple signal lines 70 to control integration time for each row is reduction of fill factor or a larger pixel size in order to fit the extra signal lines into the pixel pitch. This is overcome by the signal line routing architecture shown inFIG. 4 b. In this case a single integrationtime control line 80 is used per row, but it is actually routed to pixels in two adjacent rows. Thesignal line 80 in the adjacent row is routed in a similar manner to create the integration time pattern shown inFIG. 2 b. With this approach, although a single row of data is readout from the sensor at one time, the pixels contained within the data stream are from physically adjacent rows in the array. In order to properly reconstruct the image, the interlaced data must be corrected in the camera image memory. This is also a feature of the present invention. Since either on-chip or in-camera memory can be set up to write data into two or more row locations, there is no need to have the sensor read out all pixels from a physical row at the same time. - As previously discussed, this provides an image sensor and image capture system with wide intra-scene dynamic range and wide exposure latitude. A single image capture can render a full range of image information with optimization of the integration time for low light levels without clipping signal information in the high light regions of an image. This can greatly simplify the exposure control system and algorithms in an imaging system since choice of exposure or integration time does not need to be as precise.
- It should also be noted that an image capture system using such a sensor can be used to measure or determine the dynamic range of a scene to set the two integration times appropriately. During the metering phase of a camera system, two widely separated integration times can be used to determine the maximum and minimum light levels in the scene. The two integration times can then be adjusted to cover the range of illumination in the scene. For example, if the dynamic range of the scene to be captured is within the inherent dynamic range of the image sensor, then the two integration times can be set to the same value. If the scene contains a dynamic range that is wider than the true dynamic range of the sensor, then the two integration times can be set to match or optimally cover the dynamic range of the scene.
- Referring to
FIG. 5 , there is shown acamera 90 for implementing the image sensor of the present invention is one of many consumer-oriented commercial embodiments. - The invention has been described with reference to a preferred embodiment. However, it will be appreciated that variations and modifications can be effected by a person of ordinary skill in the art without departing from the scope of the invention.
-
- 10 pixel array
- 20 pixel array
- 30 pixel array
- 40 a mated pair of rows
- 40 b mated pair of rows
- 50 sensor array
- 60 a 2 by 2 pixel pattern
- 60
b 2 by 2 pixel pattern - 70 multiple signal line
- 80 single integration time control line
- 90 camera
Claims (24)
1. An image sensor comprising:
(a) a plurality of pixels arranged in an array of rows and columns;
(b) a color filter pattern spanning at least a portion of the pixels, wherein the color filter pattern forms a plurality of color filter kernels having at least one color of every color in the color filter pattern in a predetermined arrangement with an identical pattern of colors in each color filter kernel, and wherein the color filter kernels are arranged in at least two different uniformly distributed sets that are correlated with the color filter pattern; and
(c) a mechanism for independent control of an integration time of each uniformly distributed set , wherein a first uniformly distributed set has a first integration time and a second uniformly distributed set has a second integration time that is different from the first integration time.
2. The image sensor as in claim 1 , wherein the color filter pattern is a Bayer color filter pattern.
3. The image sensor as in claim 1 , wherein the color filter pattern is a 2×2 kernel.
4. The image sensor as in claim 3 , wherein the at least two different uniformly distributed sets comprise an alternating pattern of two lines of 2×2 kernels.
5. The image sensor as in claim 3 , wherein the at least two different uniformly distributed sets comprise 2×2 kernels.
6. The image sensor as in claim 5 , wherein the integration time pattern of adjacent two lines groups is offset by two pixels.
7. The image sensor of claim 1 wherein the integration time pattern is a multiple of the color filter kernel.
8. An image sensor comprising:
(a) a plurality of pixels arranged in an array of rows and columns; and
(b) an integration time control line for each row of pixels, wherein each integration time control line is routed to a portion of the pixels in one row and to a portion of the pixels in an adjacent row to provide output signal values having signals that are generated from pixels within at least two physically separate rows within the array.
9. A camera comprising:
(a) an image sensor comprising:
(a1) a plurality of pixels arranged in an array of rows and columns;
(b) a color filter pattern spanning at least a portion of the pixels, wherein the color filter pattern forms a plurality of color filter kernels having at least one color of every color in the color filter pattern in a predetermined arrangement with an identical pattern of colors in each color filter kernel, and wherein the color filter kernels are arranged in at least two different uniformly distributed sets that are correlated with the color filter pattern; and
(c) a mechanism for independent control of an integration time of each uniformly distributed set, wherein a first uniformly distributed set has a first integration time and a second uniformly distributed set has a second integration time that is different from the first integration time.
10. The camera as in claim 9 , wherein the color filter pattern is a Bayer color filter pattern.
11. The camera as in claim 9 , wherein the color filter pattern is a 2×2 kernel.
12. The camera as in claim 11 , wherein the at least two different uniformly distributed sets comprise an alternating pattern of two lines of 2×2 kernels.
13. The camera as in claim 11 , wherein the at least two different uniformly distributed sets comprise 2×2 kernels.
14. The camera as in claim 13 , wherein the integration time pattern of adjacent two lines groups is offset by two pixels.
15. The camera as in claim 1 , wherein the integration time pattern is a multiple of the color filter kernel.
16. The camera as in claim 9 further comprising a mechanism that reads out at least a subset of the plurality of pixels and uses the signal values obtained from the readout to determine the integration times of the plurality of pixels.
17. A camera comprising:
(a) an image sensor comprising:
(a1) a plurality of pixels arranged in an array of rows and columns; and
(a2) an integration time control line for each row of pixels, wherein each integration time control line is routed to a portion of the pixels in one row and to a portion of the pixels in an adjacent row to produce output signal values having signals that are generated from pixels within at least two physically separate rows within the array.
18. The camera as in claim 17 , further comprising:
(b) memory; and
(c) means for writing the output signal values into two row locations in the memory for each row of pixels, wherein the output signal values are reconstructed in the memory.
19. The image sensor of claim 1 , further comprising an integration time control line for each row of pixels, wherein each integration time control line is routed to a portion of the pixels in two adjacent rows to provide output signal values having signals that are generated from pixels within the adjacent rows within the array.
20. The camera of claim 9 , wherein the image sensor further comprises an integration time control line for each row of pixels, wherein each integration time control line is routed to a portion of the pixels in two adjacent rows to provide output signal values having signals that are generated from pixels within the adjacent rows within the array.
21. The image sensor of claim 8 , further comprising a color filter pattern spanning at least a portion of the pixels, wherein the color filter pattern forms a plurality of color filter kernels having at least one color of every color in the color filter pattern in a predetermined arrangement with an identical pattern of colors in each color filter kernel, and wherein the color filter kernels are arranged in at least two different uniformly distributed sets that are correlated with the color filter pattern.
22. The image sensor of claim 21 , wherein the color filter pattern is a 2×2 kernel.
23. The image sensor of claim 8 , further comprising:
a memory; and
means for writing the output signal values into two row locations in the memory for each row of pixels to reconstruct an image.
24. An image sensor comprising:
(a) a plurality of pixels arranged in an array of rows and columns;
(b) a color filter pattern spanning at least a portion of the pixels, wherein the color filter pattern forms a plurality of color filter kernels having at least one color of every color in the color filter pattern in a predetermined arrangement with an identical pattern of colors in each color filter kernel, and wherein the color filter kernels are arranged in at least two different uniformly distributed sets that are correlated with the color filter pattern; and
(c) a mechanism for independent control of an integration time of each uniformly distributed set, wherein a first uniformly distributed set has a first integration time and a second uniformly distributed set has a second integration time that is different from the first integration time, and wherein at least a portion of the pixels in at least one of the uniformly distributed sets does not contain valid signal level information.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/898,809 US20110019040A1 (en) | 2003-09-03 | 2010-10-06 | Image sensor and image capture system with extended dynamic range |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/654,313 US7830435B2 (en) | 2003-09-03 | 2003-09-03 | Image sensor and image capture system with extended dynamic range |
US12/898,809 US20110019040A1 (en) | 2003-09-03 | 2010-10-06 | Image sensor and image capture system with extended dynamic range |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/654,313 Continuation US7830435B2 (en) | 2003-09-03 | 2003-09-03 | Image sensor and image capture system with extended dynamic range |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110019040A1 true US20110019040A1 (en) | 2011-01-27 |
Family
ID=34218061
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/654,313 Active 2027-08-03 US7830435B2 (en) | 2003-09-03 | 2003-09-03 | Image sensor and image capture system with extended dynamic range |
US12/898,809 Abandoned US20110019040A1 (en) | 2003-09-03 | 2010-10-06 | Image sensor and image capture system with extended dynamic range |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/654,313 Active 2027-08-03 US7830435B2 (en) | 2003-09-03 | 2003-09-03 | Image sensor and image capture system with extended dynamic range |
Country Status (7)
Country | Link |
---|---|
US (2) | US7830435B2 (en) |
EP (1) | EP1661390A2 (en) |
JP (1) | JP2007504770A (en) |
KR (1) | KR101038386B1 (en) |
CN (2) | CN102307282A (en) |
TW (1) | TWI387334B (en) |
WO (1) | WO2005024948A2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130088624A1 (en) * | 2011-10-11 | 2013-04-11 | Omnivision Technologies, Inc. | High dynamic range sub-sampling architecture |
WO2020009550A1 (en) * | 2018-07-06 | 2020-01-09 | Samsung Electronics Co., Ltd. | Method and apparatus for capturing dynamic images |
Families Citing this family (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7489352B2 (en) * | 2002-11-15 | 2009-02-10 | Micron Technology, Inc. | Wide dynamic range pinned photodiode active pixel sensor (APS) |
FR2861242B1 (en) * | 2003-10-17 | 2006-01-20 | Trixell Sas | METHOD FOR CONTROLLING A PHOTOSENSITIVE DEVICE |
US7446812B2 (en) * | 2004-01-13 | 2008-11-04 | Micron Technology, Inc. | Wide dynamic range operations for imaging |
US8081234B2 (en) * | 2004-12-29 | 2011-12-20 | Intel Corporation | Technique for increased exposure range in image sensors |
US7214998B2 (en) * | 2005-07-26 | 2007-05-08 | United Microelectronics Corp. | Complementary metal oxide semiconductor image sensor layout structure |
KR100769548B1 (en) | 2005-12-30 | 2007-10-23 | 엠텍비젼 주식회사 | Color filter array and image interpolation method |
JP4487944B2 (en) * | 2006-02-09 | 2010-06-23 | ソニー株式会社 | Solid-state imaging device |
US20080198251A1 (en) * | 2007-02-15 | 2008-08-21 | Micron Technology, Inc. | Method, apparatus, and system providing multiple pixel integration periods |
US20100044676A1 (en) | 2008-04-18 | 2010-02-25 | Invisage Technologies, Inc. | Photodetectors and Photovoltaics Based on Semiconductor Nanocrystals |
US7923801B2 (en) * | 2007-04-18 | 2011-04-12 | Invisage Technologies, Inc. | Materials, systems and methods for optoelectronic devices |
KR101380615B1 (en) * | 2007-06-28 | 2014-04-14 | 삼성전자주식회사 | Method and apparatus for improving dynamic range of images |
JP4480740B2 (en) * | 2007-07-03 | 2010-06-16 | シャープ株式会社 | Solid-state imaging device, manufacturing method thereof, and electronic information device |
KR101424717B1 (en) * | 2007-09-13 | 2014-08-01 | 삼성전자주식회사 | Apparatus and method for exposure time setting |
US8102435B2 (en) | 2007-09-18 | 2012-01-24 | Stmicroelectronics S.R.L. | Method for acquiring a digital image with a large dynamic range with a sensor of lesser dynamic range |
US7940311B2 (en) * | 2007-10-03 | 2011-05-10 | Nokia Corporation | Multi-exposure pattern for enhancing dynamic range of images |
US8203195B2 (en) | 2008-04-18 | 2012-06-19 | Invisage Technologies, Inc. | Materials, fabrication equipment, and methods for stable, sensitive photodetectors and image sensors made therefrom |
US8405750B2 (en) * | 2009-06-08 | 2013-03-26 | Aptina Imaging Corporation | Image sensors and image reconstruction methods for capturing high dynamic range images |
US8605177B2 (en) * | 2009-09-16 | 2013-12-10 | Altasens, Inc. | Image sensor with wide dynamic range |
US8314873B2 (en) | 2009-09-30 | 2012-11-20 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US20110074997A1 (en) * | 2009-09-30 | 2011-03-31 | Border John N | Methods for capturing and reading out images from an image sensor |
US8134628B2 (en) * | 2009-09-30 | 2012-03-13 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8194164B2 (en) * | 2009-09-30 | 2012-06-05 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8149303B2 (en) * | 2009-09-30 | 2012-04-03 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8144220B2 (en) | 2009-09-30 | 2012-03-27 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8279317B2 (en) * | 2009-09-30 | 2012-10-02 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8194165B2 (en) | 2009-09-30 | 2012-06-05 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8279316B2 (en) * | 2009-09-30 | 2012-10-02 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8194166B2 (en) | 2009-09-30 | 2012-06-05 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8294803B2 (en) * | 2009-09-30 | 2012-10-23 | Truesense Imaging, Inc. | Methods for capturing and reading out images from an image sensor |
US8916947B2 (en) | 2010-06-08 | 2014-12-23 | Invisage Technologies, Inc. | Photodetector comprising a pinned photodiode that is formed by an optically sensitive layer and a silicon diode |
US8890986B2 (en) * | 2011-04-27 | 2014-11-18 | Aptina Imaging Corporation | Method and apparatus for capturing high dynamic range images using multi-frame interlaced exposure images |
JP2012257193A (en) * | 2011-05-13 | 2012-12-27 | Sony Corp | Image processing apparatus, image pickup apparatus, image processing method, and program |
US8913153B2 (en) * | 2011-10-06 | 2014-12-16 | Aptina Imaging Corporation | Imaging systems and methods for generating motion-compensated high-dynamic-range images |
US20130128083A1 (en) * | 2011-11-23 | 2013-05-23 | Himax Imaging Limited | High dynamic range image sensing device and image sensing method and manufacturing method thereof |
US9172889B2 (en) | 2012-02-09 | 2015-10-27 | Semiconductor Components Industries, Llc | Imaging systems and methods for generating auto-exposed high-dynamic-range images |
US9007488B2 (en) | 2012-03-08 | 2015-04-14 | Semiconductor Components Industries, Llc | Systems and methods for generating interpolated high-dynamic-range images |
US9338372B2 (en) | 2012-09-19 | 2016-05-10 | Semiconductor Components Industries, Llc | Column-based high dynamic range imaging systems |
JP2014086889A (en) * | 2012-10-24 | 2014-05-12 | Toshiba Corp | Solid-state imaging apparatus |
CN103139491A (en) * | 2013-03-25 | 2013-06-05 | 天利半导体(深圳)有限公司 | High-dynamic-range low-noise complementary metal oxide semiconductor (CMOS) sensor chip |
WO2015191734A1 (en) | 2014-06-10 | 2015-12-17 | Edward Hartley Sargent | Multi-terminal optoelectronic devices for light detection |
KR20160071203A (en) | 2014-12-11 | 2016-06-21 | 삼성전자주식회사 | Image signal processor, method for operating thereof, and mobile computing device including same |
FR3050596B1 (en) * | 2016-04-26 | 2018-04-20 | New Imaging Technologies | TWO-SENSOR IMAGER SYSTEM |
CN110531401B (en) * | 2019-09-11 | 2021-04-09 | 南华大学 | Nuclear radiation detector and nuclear radiation detection method |
WO2021102366A1 (en) | 2019-11-20 | 2021-05-27 | Gigajot Technology, Inc. | Scalable-pixel-size image sensor |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6069377A (en) * | 1999-05-13 | 2000-05-30 | Eastman Kodak Company | Image sensor incorporating saturation time measurement to increase dynamic range |
US6307195B1 (en) * | 1999-10-26 | 2001-10-23 | Eastman Kodak Company | Variable collection of blooming charge to extend dynamic range |
US6486504B1 (en) * | 1999-10-26 | 2002-11-26 | Eastman Kodak Company | CMOS image sensor with extended dynamic range |
US20030098917A1 (en) * | 2001-11-26 | 2003-05-29 | Kazuya Oda | Solid-state image sensor efficiently utilizing its dynamic range and image pickup apparatus using the same |
US6999119B1 (en) * | 1998-04-10 | 2006-02-14 | Nikon Corporation | Image-capturing element, image-capturing circuit for processing signal from image-capturing element, image-capturing device, driving method of image-capturing element |
US7411626B2 (en) * | 2002-03-22 | 2008-08-12 | Olympus Corporation | Image acquiring apparatus and image acquiring method |
US7479998B2 (en) * | 2001-01-09 | 2009-01-20 | Sony Corporation | Image pickup and conversion apparatus |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3971065A (en) * | 1975-03-05 | 1976-07-20 | Eastman Kodak Company | Color imaging array |
DE69315366T2 (en) * | 1992-04-10 | 1998-03-19 | Mitsubishi Electric Corp | Rotatable color filter for a color image display |
JP3156503B2 (en) * | 1994-05-27 | 2001-04-16 | 松下電器産業株式会社 | Driving method of solid-state imaging device and signal processing circuit of solid-state imaging device |
CA2238534C (en) * | 1996-09-27 | 2001-10-09 | Markus Bohm | Local autoadaptive optical sensor |
US6714239B2 (en) * | 1997-10-29 | 2004-03-30 | Eastman Kodak Company | Active pixel sensor with programmable color balance |
US6831691B1 (en) * | 1998-04-15 | 2004-12-14 | Minolta Co., Ltd. | Solid-state image pickup device |
US6665010B1 (en) * | 1998-07-21 | 2003-12-16 | Intel Corporation | Controlling integration times of pixel sensors |
US7030917B2 (en) * | 1998-10-23 | 2006-04-18 | Hewlett-Packard Development Company, L.P. | Image demosaicing and enhancement system |
JP3707725B2 (en) * | 2000-05-30 | 2005-10-19 | 松下電器産業株式会社 | Still image solid-state imaging device |
US6943837B1 (en) * | 1999-12-31 | 2005-09-13 | Intel Corporation | Method and apparatus for colormetric channel balancing for solid state image sensor using time division multiplexed sampling waveforms |
JP2001352486A (en) * | 2000-06-07 | 2001-12-21 | Matsushita Electric Ind Co Ltd | Image pickup device |
JP2002185864A (en) * | 2000-12-12 | 2002-06-28 | Sony Corp | Solid-state image sensors and its drive method |
US6943831B2 (en) * | 2001-01-24 | 2005-09-13 | Eastman Kodak Company | Method and apparatus to extend the effective dynamic range of an image sensing device and use residual images |
JP3905708B2 (en) * | 2001-01-26 | 2007-04-18 | ペンタックス株式会社 | Image interpolation device |
JP3627711B2 (en) * | 2002-02-14 | 2005-03-09 | ミノルタ株式会社 | Color imaging device |
-
2003
- 2003-09-03 US US10/654,313 patent/US7830435B2/en active Active
-
2004
- 2004-09-01 WO PCT/US2004/028325 patent/WO2005024948A2/en active Application Filing
- 2004-09-01 CN CN201110283424XA patent/CN102307282A/en active Pending
- 2004-09-01 KR KR1020067004315A patent/KR101038386B1/en active IP Right Grant
- 2004-09-01 CN CN200480024674XA patent/CN1843027B/en active Active
- 2004-09-01 EP EP04782749A patent/EP1661390A2/en not_active Withdrawn
- 2004-09-01 JP JP2006526154A patent/JP2007504770A/en active Pending
- 2004-09-02 TW TW093126565A patent/TWI387334B/en active
-
2010
- 2010-10-06 US US12/898,809 patent/US20110019040A1/en not_active Abandoned
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6999119B1 (en) * | 1998-04-10 | 2006-02-14 | Nikon Corporation | Image-capturing element, image-capturing circuit for processing signal from image-capturing element, image-capturing device, driving method of image-capturing element |
US6069377A (en) * | 1999-05-13 | 2000-05-30 | Eastman Kodak Company | Image sensor incorporating saturation time measurement to increase dynamic range |
US6307195B1 (en) * | 1999-10-26 | 2001-10-23 | Eastman Kodak Company | Variable collection of blooming charge to extend dynamic range |
US6486504B1 (en) * | 1999-10-26 | 2002-11-26 | Eastman Kodak Company | CMOS image sensor with extended dynamic range |
US7479998B2 (en) * | 2001-01-09 | 2009-01-20 | Sony Corporation | Image pickup and conversion apparatus |
US20030098917A1 (en) * | 2001-11-26 | 2003-05-29 | Kazuya Oda | Solid-state image sensor efficiently utilizing its dynamic range and image pickup apparatus using the same |
US7411626B2 (en) * | 2002-03-22 | 2008-08-12 | Olympus Corporation | Image acquiring apparatus and image acquiring method |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130088624A1 (en) * | 2011-10-11 | 2013-04-11 | Omnivision Technologies, Inc. | High dynamic range sub-sampling architecture |
US8599284B2 (en) * | 2011-10-11 | 2013-12-03 | Omnivision Technologies, Inc. | High dynamic range sub-sampling architecture |
WO2020009550A1 (en) * | 2018-07-06 | 2020-01-09 | Samsung Electronics Co., Ltd. | Method and apparatus for capturing dynamic images |
US11050945B2 (en) | 2018-07-06 | 2021-06-29 | Samsung Electronics Co., Ltd. | Method and apparatus for capturing dynamic images |
Also Published As
Publication number | Publication date |
---|---|
TWI387334B (en) | 2013-02-21 |
CN1843027B (en) | 2012-06-20 |
CN1843027A (en) | 2006-10-04 |
US7830435B2 (en) | 2010-11-09 |
CN102307282A (en) | 2012-01-04 |
TW200520550A (en) | 2005-06-16 |
KR20060118425A (en) | 2006-11-23 |
WO2005024948A2 (en) | 2005-03-17 |
US20050045980A1 (en) | 2005-03-03 |
WO2005024948A3 (en) | 2005-09-01 |
JP2007504770A (en) | 2007-03-01 |
KR101038386B1 (en) | 2011-06-01 |
EP1661390A2 (en) | 2006-05-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7830435B2 (en) | Image sensor and image capture system with extended dynamic range | |
US6710804B1 (en) | CMOS active pixel image sensor with extended dynamic range and sensitivity | |
US9531976B2 (en) | Systems and methods for operating image sensor pixels having different sensitivities and shared charge storage regions | |
US8350940B2 (en) | Image sensors and color filter arrays for charge summing and interlaced readout modes | |
US5881184A (en) | Active pixel sensor with single pixel reset | |
US7026596B2 (en) | High-low sensitivity pixel | |
US6970195B1 (en) | Digital image sensor with improved color reproduction | |
Yadid-Pecht et al. | Wide intrascene dynamic range CMOS APS using dual sampling | |
US6714239B2 (en) | Active pixel sensor with programmable color balance | |
US6175383B1 (en) | Method and apparatus of high dynamic range image sensor with individual pixel reset | |
US7411169B2 (en) | Wide dynamic range image sensor and method of use | |
US20050128327A1 (en) | Device and method for image sensing | |
US20080106625A1 (en) | Multi image storage on sensor | |
US7746398B2 (en) | Wide dynamic range active pixel with knee response | |
US10033947B2 (en) | Multi-port image pixels | |
El Gamal | High dynamic range image sensors | |
US6515702B1 (en) | Active pixel image sensor with a winner-take-all mode of operation | |
US10218944B2 (en) | Methods and devices for increasing the spectral response of an image sensor | |
US10880500B2 (en) | Pixel apparatus and CMOS image sensor using the same | |
Roca et al. | Programmable resolution imager for imaging applications | |
Roca Moreno et al. | Programmable resolution imager for imaging applications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OMNIVISION TECHNOLOGIES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:EASTMAN KODAK COMPANY;REEL/FRAME:026227/0213 Effective date: 20110415 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |