Background technology
Human vision system has three kinds of cones: the long cone, the middle cone and short-sighted awl.These cones are can respond to respectively to correspond to redness, green and blue light wavelength.Visual cortex in the human brain can obtain signal from this cone of three kinds, and then will carry out interpolation value (interpolation) to form the image of a shaping.The importance of human visual system's biological structure is: a large amount of, the whole visible spectrums that can handle convert to dramatically has only red, blue and green three primary colors.
Therefore, the digitized video inductor in digital camera also is designed to be similar to human visual system's the cone.A kind of the most frequently used image sensor array be by doctor Bel (Bryce E.Bayer) in the United States Patent (USP) notification number: 3,971,065 disclose, be referred to as Bel (Bayer) filter inductor array 100, this Bel's filter inductor array is arranged in Bel's sample attitude image sensor 10, as shown in Figure 1.One Bel's filter array 100 has red, the green and blue three kinds of multi-color filtrate lenslets that are positioned on this image sensor, long to obtain respectively, in and short wavelength's spectral information.In addition, this Bel's filter array is green information is sampled as the plum blossom shape, and red and blue data then is sampled as rectangular-shaped.This green data is to take a sample under a upper frequency, and this is because human vision system has higher acuity for medium wavelength.In a single image sensor, the colour information that other two look (red and blue) is lacked will use the value of color of neighborhood pixels to carry out the action of interpolation value.
Fig. 2 is the example that has shown the colored interpolation value that is used for this 3 * 3 Bel filter array.In the example the 1st kind and the 4th kind of array are to be the center with the green, so will be carried out the color-element of interpolation value redness and blueness are arranged.The 2nd kind in the example is to be the center with the redness, so will be carried out the color-element of interpolation value green and blueness is arranged.The 3rd kind of array in the example is to be the center with the blueness, so will be carried out the color-element of interpolation value green and redness arranged.
For convenience of description, each pixel that is arranged in Bel's 3 * 3 array examples is designated as the numeral of Fig. 3.
In known techniques, a kind of method of Bel's array interpolation value is used so-called linear benefit interpolation method, and the pixel that is lacked in each Bel's array is to carry out the interpolation value in the following manner:
In the example 1:
R=(a3+a5)/2
B=(a1+a7)/2
In the example 2:
G=(a1+a3+a5+a7)/4
B=(a0+a2+a6+a8)/4
In the example 3:
G=(a1+a3+a5+a7)/4
R=(a0+a2+a6+a8)/4
In the example 4:
R=(a1+a7)/2
B=(a3+a5)/2
Another kind of known Bel's array is mended interpolation method and is disclosed in United States Patent (USP) notification number 5,382,976 and 5,373,322.Above-mentioned disclosed content is to use marginal information to decide to wait and is used for the neighbor of interpolation value.Described method as shown in Figure 4.The process of described method is at first imported one 3 * 3 Bel's sample attitude blocks (step S10).In step S12, deciding Bel's block of being imported according to 4 kinds of shown arrays of Fig. 2 is that to belong to that of that Fig. 2 a kind of.In step S14, the example of Bel's array is divided into two kinds: with the green pixel be the center and non-be the center with the green pixel.If center pixel is not a green pixel, then belong to Bel's array of the 2nd kind and the 3rd kind, therefore, in step 16, calculate the difference value of green pixel, that is:
H_diff=|a3-a5| and
V_diff=|a1-a7|.
In step S20, this green pixel that lacks is carried out the interpolation value, that is:
If H_diff<V_diff
G=(a3+a5)/2
Otherwise, if V_diff<H_diff
G=(a1+a7)/2
Otherwise
G=(a1+a3+a5+a7)/4.
In step S22, this redness that lacks or blue pixel are carried out the interpolation value, that is:
If example II
B=(a0+a2+a6+a8)/4
Otherwise, if example III
R=(a0+a2+a6+a8)/4.
On the other hand, if center pixel is green, then in step S18, the redness and the green that lack are carried out the interpolation value, that is:
If example I
R=(a3+a5)/2
B=(a1+a7)/2
Otherwise, if example IV
R=(a1+a7)/2
B=(a3+a5)/2
Yet,, will become very big in the heterochromia of the pixel of all ternarys, and can cause uncomfortable aberration at the edge of height change if by using above-mentioned method to carry out the colour element interpolation value that Bel's colored filter array is lacked.In addition, for the preferable pixel that lacks of construction again, United States Patent (USP) notification number 5,373,322 and 6,549,233 all teaching a kind of Bel's filter array of broad, it has more adjacent pixels, but uses the method for above-mentioned patent will increase the cost that is performed on the hardware significantly.D.Alleysson, people such as S.Susstrunk and J.Herault are in April, 2005 IEEE image processing proceedings the 14 volume No. four 439-449 page or leaf teaching " linearity that the human visual system caused is translated mosaic (Linear Demosaicing) ".The disclosed content of people such as D.Alleysson is that more complicated Bel's filter array is mended interpolation method, and its interpolation value that needs to implement in frequency domain is handled.People such as Endo at No. 6,853,748, United States Patent (USP) bulletin case US and Acharya at United States Patent (USP) bulletin case US6, in 917,381, teaching be used for multiplication and the division that this Bel's array is mended interpolation method, but it also can increase the cost that is performed on the hardware.
Summary of the invention
A purpose of the present invention is to solve the above problems, and provide a kind of colour of the Bel's of being used for filter array image to mend interpolation method, it not only only uses one 3 * 3 block of pixels, and provides by implementing the edge simultaneously and colored different information carries out correct reconstruction.
The present invention mends interpolation method by the colour that a kind of Bel's filter is provided and reaches above-mentioned purpose.This method is at first imported 3 * 3 Bel's sample attitude blocks with a center pixel.Then, decide the example (case) of a colored filter interpolation value according to the kind of center pixel.Then, the horizontal direction of acquisition colored filter interpolation value example and Bel's pixel of vertical direction are average, and the difference value (difference) of Bel's pixel of the horizontal direction of acquisition colored filter interpolation value example and vertical direction.Then, carry out the interpolation value for the color-element that lacks at the center pixel of level and vertical direction.Then, based on this Bel's pixel difference value and average, obtain colored difference value (color differences) in level and vertical direction.Then, determine a plain picture movable (pixel activity) of this 3 * 3 Bel pixel region.At last, when this element picture activity when being high, be chosen in this color-element of interpolation value in the direction of low colored difference value; When this pixel activity when being low, decision is in Bel's pixel difference value of level and vertical direction; Wherein:
If should be during, in lower Bel's pixel difference value direction of this Bel's pixel difference value, select the result of colored interpolation value in difference value of a direction much larger than another; And
If should not have significantly not simultaneously in the pixel difference value of level and vertical direction, it is that interpolation value colour uses that this Bel's pixel is on average treated as.
The present invention has following three advantages:
1. because this colour that lacks is to carry out the interpolation value at a certain specific direction, this direction gave if it were not for lower colored difference value, be exactly lower pixel difference value, thus this image after the interpolation value will from this colour difference value, have lower vision and disturb.
2. because in the process that the interpolation value is handled, only use 3 * 3 pixel region, so the complexity that hardware is carried out can be reduced for the benefit interpolation method of 5 * 5 pixel region significantly.
Therefore 3. this interpolation value process only relates to the simple mathematical computing, for example: add, subtract and the logical bit computing, be performed in cost on the hardware with respect to reducing significantly for the benefit interpolation method of taking advantage of and removing of people institute teachings such as people such as Endo and Acharya.
In following detailed description, cooperate by an embodiment and graphicly can understand the present invention maturely, but this embodiment is not in order to restriction the present invention.
Embodiment
The present invention discloses a kind of colour of image of the Bel's of being used for filter array to mend interpolation method, and this method not only has only one 3 * 3 pixel regions of use, but also by carrying out the correctness that edge and colored different information improve reconstruction simultaneously.
Fig. 5 A and Fig. 5 B show that the present invention is used for the flow chart of steps that Bel's array is mended interpolation method.This step is at first imported one and is had center pixel in that (x, y) one 3 * 3 Bel's sample attitude blocks of position are shown in step S110.
In step S112, decide the example of a colored filter interpolation value according to the kind of this center pixel.The example of this Bel's interpolation value is just as shown at Fig. 2.
In step S114, calculate the Bel pixel difference value (difference) of this colored filter interpolation value example in level and vertical direction.
In step S116, the Bel's pixel that obtains this colored filter interpolation value example is average.Relevantly be used for calculating Bel's pixel difference value and average method will further be discussed in following preferred embodiment.
In step S118, carry out the interpolation value of level and vertical direction for the color-element that lacks of this center pixel.
In step S120,, calculate colored difference value in level and vertical direction according to this result of interpolation value (comprised this Bel's pixel difference value and average).
In step S122, determine a pixel activity of this 3 * 3 pixel region.
In step S124, if a higher pixel activity is arranged, then this method is carried out step S126; Otherwise this method is carried out step S128.
In step S126, be chosen in the value of the interpolation color-element of the direction of low colored difference value.
Otherwise,, then determine Bel's pixel difference value, shown in step S128 in level and vertical direction if a lower pixel activity is arranged.
In step S130, if be much larger than another in the difference value of a direction, then this method is carried out step S132; Otherwise this method is carried out step S134.
In step S132, be chosen in the result of this colour interpolation value of Bel's pixel difference value direction lower in this Bel's pixel difference value.
Yet,, use pixel average as this interpolation value colour, i.e. step S134 if do not have very large words in the pixel difference value of level and vertical direction.
Embodiment 1
Fig. 6 A and Fig. 6 B are the flow charts that shows a preferred embodiment of Bel's array interpolation value method of the present invention.Shown in step S200, import Bel's sample attitude image, and an initial setting coordinate of the center pixel of this 3 * 3 pixel region becomes as the y=1 of step S202 and the x=1 of step S204.
Fig. 7 is the schematic diagram that is used to illustrate Bel's image and order that should colour interpolation value.As shown in Figure 7, the size of this Bel's image 700 is the wide height of taking advantage of.Fig. 7 shows that further one 3 * 3 pixel regions 710 and its center pixel 711 are GTG.The order of this colour interpolation value is a laser direction as shown by arrows.
In step S206,, determine the example of this Bel's interpolation value based on content as shown in Figure 2.
In step S208, a green pixel difference value, horizontal pixel difference value and vertical pixel difference value are by calculating it with reference to 3 * 3 pixel regions among the figure 3.This horizontal pixel difference value (h-diff) and vertical pixel difference value (v-diff) are as follows respectively:
H_diff=|a0+2 * a3+a6-a2-2 * a5-a8|, and
v_diff=|a0+2×a1+a2-a6-2×a7-a8|。
Yet,, obtain this green pixel difference value (g_diff) according to the difference of this Bel's example.If this Bel's example is example 1 or example 4, then this green pixel difference value (g_diff) is as follows:
g_diff=|a4-(a0+a2+a6+a8)/4|
Otherwise if this Bel's example is example 2 or example 3, then g_diff is as follows:
g_diff=|a1+a7-a3-a5|/2
In step S210, calculate the average h_ave of a horizontal pixel, the average v_ave of vertical pixel and the average corner_ave of center pixel respectively, as follows:
h_ave=(a3+a5)/2,
V_ave=(a1+a7)/2, and
corner_ave=(a0+a2+a6+a8)/4
In step S212, according to this Bel's example, the color-element that this center pixel lacks is carried out interpolation in the horizontal direction, and is as follows:
Example I: in red column, green is the center
R_h=h_ave
B_h=clip(a4+v_ave-corner_ave)
Example II: redness is the center
G_h=h_ave
B_h=clip(G_h+corner_ave-v_ave)
Example III: blueness is the center
G_h=h_ave
R_h=clip(G_h+corner_ave-v_ave)
Example IV: in the blue column, green is the center
R_h=clip(a4+v_ave-corner_ave)
B_h=h_ave
In aforesaid equation, this bracket (x) (clip (x)) is to be used for determining that the value in bracket is limited between the 0-255, if each value is to represent with 8bit.(x) is as follows for this bracket:
If x<0, then X=0;
If x>255, then X=255;
Otherwise x=x
In step S214, be from the horizontal interpolation value color-element of step S212, calculate this horizontal color difference value, as follows:
Example I:
h_color_diff=|a4-R_h|+|a4-B_h|
Example II:
h_color_diff=|G_h-a4|+|G_h-B_h|
Example III:
h_color_diff=|G_h-R_h|+|G_h-a4|
Example IV:
h_color_diff=|a4-R_h|+|a4-B_h|
Similarly, in step S216, be vertically this center pixel of interpolation lack color-element, for:
Example I: in red column, green is the center
R_v=clip(a4+h_ave-corner_ave)
B_v=v_ave
Example II: redness is the center
G_v=v_ave
B_v=clip(G_v+corner_ave-h_ave)
Example III: blueness is the center
G_v=v_ave
R_v=clip(G_v+corner_ave-h_ave)
Example IV: in blue column, green is the center
R_v=v_ave
B_v=clip(a4+h_ave-corner_ave)
In step S218, be from the vertical interpolation value color-element of step S216, calculate this vertical color difference value, as follows:
Example I:
v_color_diff=|a4-R_v|+|a4-B_v|
Example II:
v_color_diff=|G_v-a4|+|G_v-B_v|
Example III:
v_color_diff=|G_v-R_v|+|G_v-a4|
Example IV:
v_color_diff=|a4-R_v|+|a4-B_v|
After calculating this pixel difference value and the colour that lacks of level and vertical direction carried out the interpolation value, the interpolation result of the value of color that this is selected as lacking is to be determined by following.
In step S222, this green pixel difference value g_diff compares with this horizontal pixel difference value h_diff and vertical differentiation value v_diff, to determine this pixel activity.
If g_diff is simultaneously greater than h_diff and v_diff, then this horizontal color difference value h_color_diff and vertical color difference v_color_diff are by mutual comparison, shown in step S224.
If h_color_diff is less than this v_color_diff, then in step S226, select the horizontal interpolation value value of color among the step S212.Otherwise, if h_color_diff not greater than v_diff, then selects the vertical interpolation value value of color of step S216 in step S228.Then, the program of this interpolation value is leapt to step S236.
Yet,, in step S230 and step S232, further compare h_diff and v_diff if the condition that does not satisfy in step S222 compares.If h_diff is less than v_diff (step S230), then in step S226, select the horizontal interpolation value value of color of step S212.Otherwise,, then in step S228, select the vertical interpolation value value of color of step 216 if v_diff is less than this h_diff (step S232).But, if when this condition does not relatively satisfy S230 and S232 simultaneously, this value of color that lacks will carry out following linear interpolation value:
Example I: in red column, green is the center
R_v=h_ave
B_v=v_ave
Example II: redness is the center
G_v=(h_ave+v_ave)/2
B_v=corner_ave
Example III: blueness is the center
G_v=(h_ave+v_ave)/2
R_v=corner_ave
Example IV: in the blue column, green is the center
R_v=v_ave
B_v=h_ave
If v_diff equals h_diff, then use the linear colour of interpolation value.For center pixel coordinate (x, y) carry out this color-element interpolation value that lacks after, this horizontal coordinate x increases (step 236) in the x=x+1 mode.If x remains less than wide-1 (step S238), then this interpolation value program is got back to step S206.Yet if do not satisfy in the condition of step S238, vertical coordinate y increases in the y=y+1 mode, and in step S242, y and high-1 (hight-1) compare.If satisfy the condition of step S242, then interpolation value program is got back to step S204; Otherwise,, then finish the colored interpolation value that lacks of Bel's image if do not satisfy the condition of step S242.