WO1999009737A1

WO1999009737A1 - Color channel independent electronic shutter for solid state image sensor

Info

Publication number: WO1999009737A1
Application number: PCT/US1997/021557
Authority: WO
Inventors: Warren Severin; Werner Metz
Original assignee: Intel Corporation
Priority date: 1997-08-15
Filing date: 1997-11-25
Publication date: 1999-02-25
Also published as: TW361042B; AU5457098A

Abstract

An improved color image sensor (100) semiconductor integrated circuit (IC) where photosites (111-124) of a particular color receive an independent electronic shutter control (300) signal which allows the exposure time of pixels corresponding to each color to be set independently. The embodiments aim to improve image quality under lower illumination conditions by improving signal-to-noise ratio in the color channels having lower illumination, and permit some manufacturing variation in the sensor IC, in the Color Filter Array (CFA), and in the optical component manufacturing processes. The invention may be particularly useful in portable digital image capture systems, such as the digital camera, but may also find use in color scanners and certain color copiers.

Description

COLOR CHANNEL INDEPENDENT ELECTRONIC SHUTTER FOR SOLID STATE IMAGE SENSOR

BACKGROUND INFORMATION

This invention relates to semiconductor integrated circuit techniques for sensing images in color.

Producing high quality images in color has become increasingly desirable for electronic still image capture and video systems. Conventional methods of sensing and producing an image in color include the use of electronic charged coupled device (CCD) or complimentary metal oxide semiconductor (CMOS) sensor arrays exposed to incident light. The sensor array is an array of photosites or pixels, each photosite loosely defined as a region containing photodetecting circuitry that includes, for example, photodiodes or photogates and associated processing circuitry. In some cases, a color filter array (CFA) is pasted or otherwise processed over the array of photodetecting circuits, such that each photodetecting circuit is covered by a bandpass optical filter that lets light of a particular color pass into and thus be detected by the corresponding circuit. Sensor arrays typically have two or more colors distributed evenly, or according to other schemes, in the array of photosites, where a group of photosites may be assigned a particular color.

To obtain sharp color images from such sensor arrays over a broad range of illuminants, it may be desirable to control the amount of light energy that is incident on the sensor array. One technique for doing so is to modulate the incident light using a physical shutter having variable timing, as in a conventional film camera. The idea of a shutter has been applied to digital solid state cameras in the form of an electronic shutter. For example, in CMOS sensor arrays, the electronic shutter is typically a transistor that couples a photo detecting element, such as a photodiode, to a charge storage element, such as a capacitor, in each photosite. The electronic shutter transistor operates as a switch in response to a shutter control signal that specifies the "exposure time" by defining the time interval during which the shutter transistor is turned on allowing charge to transfer from the exposed photodiode and accumulate in the capacitor. Alternatively, the shutter transistor may be used to drain pre-stored charge from the capacitor.

Typically, prior art systems that use such electronic shutters provide a single shutter control signal for the entire sensor array. Thus, in most prior art systems, photosites of different colors receive the same exposure time. Although some systems permit different exposure times for each color, they do so by requiring separate and sequential exposures for each color, typically by using a mechanical apparatus to change the color filter over a panchromatic sensor array. Taking sequential exposures for each color effectively precludes taking color pictures of scenes that include motion. Moreover, such a complicated mechanical apparatus, normally used in devices such as flatbed scanners, would present reliability problems and would not be easily adaptable to portable image capture systems such as a digital handheld camera.

Although the technique of using a single shutter control line for the entire sensor array is relatively simple to implement, such an application may also yield poor quality images for illuminants other than broad daylight. Under low or uneven light conditions, colors for which there is a dearth of illumination are less apparent in the resulting image due to increased noise levels in the analog signals received from the photosites tuned for those "weaker" colors. Image processing algorithms may be used to somewhat improve image quality by canceling out the increased noise in the signals for the weaker colors. But such a software solution may introduce undesirable delays before the final image can be viewed. Moreover, the results of such software corrections are not always predictable or consistent.

Therefore, a hardware approach may be desirable to yield more consistent and predictable noise levels.

Also, it is always desirable to realize a color image system that can somewhat compensate for manufacturing variations in the myriad of photosites in a sensor array, including variations in the photodetecting circuitry and the bandpass filters of the CFA.

SUMMARY

This invention is directed at a circuit having photosites of different colors, and a separate control line for each color, a control line being coupled to the read out circuitry of photosites having the same color. A control signal received on the control line specifies a duration for the read out circuitry of the associated photosites to form light generated signals corresponding to the particular color.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the invention may be better understood by referring to the Figures, description, and claims below, where:

Figure 1A illustrates an image sensor circuit according to a first embodiment of the invention.

Figure IB shows exemplary shutter control signals for each color, according to another embodiment of the invention. Figure 2 shows an image sensor circuit according to a second embodiment of the invention.

Figure 3 shows an image capture system according to a third embodiment of the invention.

DETAILED DESCRIPTION

As described in detail below, the embodiments of the invention are image sensor circuits and imaging systems which may be used to support still and video imaging. For purposes of explanation, specific embodiments are set forth below to provide a thorough understanding of the invention. However, as understood by one skilled in art, from reading this disclosure, the invention may be practiced without such details. Furthermore, well-known elements, devices, process steps, and the like, are not set forth in detail in order to avoid obscuring the invention.

Figure 1A illustrates a block diagram of an image sensor circuit 100 according to one embodiment of the invention. The circuit 100 has a sensor array of pixels or photosites 111, 112, ..., arranged in rows and columns for that particular embodiment. A CFA (not shown) may be pasted or otherwise processed over the photodetecting circuitry in the sensor array such that each photosite is in effect configured to detect incident light of a particular color, as indicated by the color label adjacent each photosite.

An exemplary photosite 111 may have a green filter covering, and may include a row select input, an electronic shutter input, and an analog output. Not shown in each photosite is conventional read out circuitry which couples to a photodetecting element such as a photodiode and allows the formation of a light-generated signal which is provided at the analog output. A single shutter control line is coupled to the shutter input of all photosites having the same color in a given row, where each control line receives an independent pulse- type shutter control signal, as shown in Figure IB.

The embodiment of the invention as circuit 100 also features a column Analog-to-Digital (A/D) converter where the light-generated signals from a given column are multiplexed and converted by the column A/D converter into digital format one row at a time. Another scheme may use A/D converters for each row, so that the digital signals are read per column. In either case, the digital signals may then be subsequently stored and further processed.

The array of circuit 100 employs one particular distribution of colors. However, the CFA can take on different embodiments with the colors being distributed across the sensor circuit according to other patterns. For example, Figure 2 illustrates another embodiment of the invention as circuit 200 wherein the CFA has alternating columns of a particular color. In this case, a single shutter control line for each color couples with the shutter inputs of all photosites for a given color in multiple columns.

Although both Figures 1 and 2 show the sensor circuit having three colors red, green and blue, the sensor circuit can also be configured with other combinations of two or more colors in its CFA, there being different colors available other than red, green, and blue, such as cyan, yellow, and magenta.

The embodiments of the invention in Figures 1A and 2 receive a number of shutter control signals, an example of which is illustrated in Figure IB as pulse type signals having different pulse widths. The shutter control signals in Figure IB control an electronic shutter (not shown) inside each photosite. The electronic shutter may be a transistor switch which when turned on will allow the formation of light-generated signals inside a capacitor in the photosite. To read out the light-generated signal, row or column select signals are supplied to the readout circuitry in the photosite which causes the light-generated signal to appear (for example, as an analog voltage) at an output port of the photosite. In the embodiment of Figure IB, the shutter control signals are pulses which go active simultaneously for all colors in the sensor circuit, but which end at different times depending on the spectral content of the illuminant. For example, the pulse widths for the red, blue, and green shutter control signals shown in Figure IB may have a ratio relationship related to the color temperature or spectrum of the illuminant. There are many techniques available for determining the color temperature or color spectrum of the illuminant.

One technique for obtaining the color spectrum of the illuminant and the relative pulse timing of the shutter control signals is briefly described below. First, the pulse width for a desired color's shutter control signal is let to be proportional to the ratio of the measured light energy for a base color to the measured light energy of the desired color during a given exposure time. The sensor array is exposed to an object having uniform reflectivity across all colors (e.g., a 20% gray card) for an initial exposure time which is the same for all photosites. Next, the measured light energy for each color is obtained by, for example, measuring the output of a number of photosites having the same color filter.

One color is then selected as the base color, such as the color with highest intensity. Thus, if the pulse width for the red shutter control signal is set to tred - 1 time unit, then the pulse with t_x for a color X is given by tx = tred * Redmeas/Xmeas ^nere Redmeas and Xmeas are proportional to measured light energy signals for the base color Red and desired color X, respectively. The measured energy levels for each color are obtained using the same exposure time, but because the illuminant may have non-uniform intensity over the different colors, the measured energy levels for different colors may be different. For this example, since the intensity of the illuminant is highest at red, the shutter control signal pulse widths for blue and green would be larger than for red, in proportion to the ratios described above.

The spectral information may be normalized prior to further processing of the

digitized light-generated signals, in that the normalized energy value x =

for each color is integrated over the pulse width or integration time tx to obtain the total number of photons detected in the interval for the particular color.

Another technique for obtaining the spectral ratios needed for the relative timing of the different shutter control signals uses a histogram of digital image data received from the sensor circuit. The needed information about the spectral content of the illuminant may be obtained after exposure of the sensor circuit, uniform in time across the different colors. This should yield for a given detected value (digital) the total number of photosites of the same color which detected that value. The histogram will thus present a distribution of photosites versus color energy, for a given scene and illuminant. By selecting a statistic, such as 95% of the cumulative distribution of pixels, the digital values for the different colors can be used to define a ratio that will be multiplied by a predetermined pulse width to give the desired pulse width of the particular color.

Some of the advantages of the embodiments of the invention may be illustrated by the following example. An optical subject such as the 20% reflective gray card typically reflects 20% of all incident energy substantially uniformly across all colors. A sensor circuit in accordance with an embodiment of the invention is placed in front of the 20% gray card so as to capture an image of the gray card. If the illuminant were, for example, daylight, such that the incident optical energy at each color was substantially the same, then a single exposure with uniform shutter timing (corresponding to equal pulse widths for the different color shutter control signals) would result in the same amount of energy collected by each photosite. This presents the ideal illuminant situation.

If, however, the illuminant were a tungsten bulb such that the energy content at blue was considerably less then the energy content at red, then uniform shutter timing would result in an undesirably low detected signal level for the blue frequency. The blue signals (or blue color channel) would thus exhibit a lowered dynamic range as compared to the red signals (red color channel).

By using independent electronic shutter timing for the blue and red channels to increase the exposure time for the blue channel relative to the exposure time for the red channel, the analog signal level for the weaker channel may be increased prior to A/D conversion thus allowing greater dynamic range in the weaker channel.

A system embodiment of the invention as an imaging or image capture system is illustrated in Figure 3. The system includes an optical interface that directs light reflected from an optical subject to a sensor circuit 300. The sensor circuit 300 includes a color sensor array as in any one of the circuits 100 and 200 described earlier. The sensor circuit 300 receives a number of independent shutter control or integration timing signals (one per color) from the control unit 308. The control unit 308 may be implemented as a hardwired logic circuit, or as a programmed processor with a suitable I/O peripheral, and may or may not be located in the same IC containing the sensor circuit 300. The control unit 308 receives exposure time values and range measurements from block 304 that are used to define the shutter control timings. In block 304, the color temperature of the illuminant may be automatically determined as described above, or alternatively may be set manually by the user. The timing for each color may be automatically computed and provided, either by hardwired logic circuits or perhaps from an I/O peripheral of a programmed processor, in one or both of the block 304 and control unit 308.

The embodiment shown in Figure 3 also provides that sensor signals from the sensor circuit 300 be transferred to signal processing unit 312. In one embodiment, the sensor signals may be in analog form to be converted into digital form by an A/D converter in the signal processing unit 312. Alternatively, the A/D conversion units may be part of sensor circuit 300 so that the sensor signals passed to unit 312 are digital. In either case, the A D conversion units may be located on the same IC as the sensor array.

In addition, the unit 312 may be configured to perform digital image processing such as noise suppression and color space conversion. Such digital processing by unit 312 may be performed by a programmed processor or by dedicated hardwired logic circuits.

After image data has been prepared by unit 312, the data may be stored in a data storage 316 which may be any conceivable type of storage device suitable for storing digital data. Modern examples include a non-volatile random access memory and a rotating media device such as magnetic and/or optical disk storage. A data link interface 320 permits the image data to be transferred outside the image capture system to, for example, a desktop computer via a serial communications link. The embodiments of the invention described above may be used to assist in correcting for manufacturing variations which induce a type of imbalance in the different color channels. For example, the optical filters for a given color channel may have unequal bandpass properties due to differences in the CFA between production batches. Also, the optical components of the imaging system may present non-uniformity across the various colors. Such variations add to the imbalance between the color channels that is created by illuminants having non-uniform intensity. The independent shutter control of the various embodiments of the invention helps to equalize the light-generated analog signals between the different color channels, and therefore also help reduce the effects of the manufacturing variations.

Another advantageous feature of the circuits 100 and 200 appears when the sensor array is implemented as a single chip. This allows simultaneous rather than sequential duration exposures for all colors, so that the imaging system which incorporates the single chip sensor array may yield higher quality color images of a moving scene.

To summarize, the embodiments of the invention described above present the design of an improved and novel color image sensor circuit that features independent electronic shutter control for each color channel. Of course, the embodiments of the invention described above are subject to other variations in structure and implementation. For example, semiconductor IC fabrication techniques other than standard CMOS may be used to implement the different embodiments. Thus, the details above should be interpreted as illustrative and not in a limiting sense.

Claims

CLAIMSWhat is claimed is:

1. A circuit comprising: a first plurality of photosites each having an optical filter of a first color, photodetecting circuitry, and read out circuitry providing a first light- generated signal; a second plurality of photosites each having an optical filter of a second color, photodetecting circuitry, and read out circuitry providing a second light- generated signal; first control line coupled to the read out circuitry of said first plurality of photosites; and second control line coupled to the read out circuitry of said second plurality of photosites, wherein said first and second control lines receive first and second control signals, respectively, each of said respective control signals specifying a duration for the read out circuitry of said respective plurality of photosites to form light- generated signals.

2. A circuit as in claim 1 wherein the first and second plurality of photosites are part of a first row of photosites.

3. A circuit as in claim 1 further comprising: a third plurality of photosites each having an optical filter of a third color and read out circuitry providing a third light-generated signal.

4. A circuit as in claim 3 wherein the first color is red, the second color is green, and the third color is blue.

5. A circuit as in claim 1 wherein said optical filters of first, and second colors are part of a color filter array that is disposed over the photodetecting circuitry of said first and second plurality of photosites.

6. An imaging system comprising: an optical interface; sensor circuit having a plurality of shutter control lines, each shutter control line for a different color channel; signal processing means coupled to the sensor circuit to receive sensor signals and provide digital image data; data storage means for storing the digital image data; means for determining an exposure time and generating exposure time values; and control unit for generating a plurality of shutter control signals in response to receiving exposure time values, each shutter control signal for a different color channel.

7. An imaging system as in claim 6 further comprising a data link interface for transferring the digital image data outside the imaging system.

8. An imaging system as in claim 6 wherein the means for determining an exposure time includes a programmed processor.

9. An imaging system as in claim 6 wherein the signal processing means includes a programmed processor.

10. An imaging system as in claim 6 wherein the data storage means includes a non-volatile random access memory.

11. An imaging system comprising: an optical interface; sensor circuit having a plurality of shutter control lines, each shutter control line for a different color channel; signal processing unit coupled to the sensor circuit to receive sensor signals and provide digital image data; data storage device for storing the digital image data; exposure time measuring unit for generating exposure time values; and, control unit for generating a plurality of shutter control signals in response to receiving exposure time values, each shutter control signal for a different color channel.