US20030202694A1 - Diagnostic apparatus for abnormalities in image recognition system - Google Patents
Diagnostic apparatus for abnormalities in image recognition system Download PDFInfo
- Publication number
- US20030202694A1 US20030202694A1 US10/421,083 US42108303A US2003202694A1 US 20030202694 A1 US20030202694 A1 US 20030202694A1 US 42108303 A US42108303 A US 42108303A US 2003202694 A1 US2003202694 A1 US 2003202694A1
- Authority
- US
- United States
- Prior art keywords
- image
- unit
- abnormality
- pixel
- diagnostic apparatus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V10/507—Summing image-intensity values; Histogram projection analysis
Definitions
- the present invention relates to a diagnostic apparatus for abnormalities in an image recognition system.
- Image recognition systems are used in various fields. For example, they are mounted in vehicles to assist the safe cruising of the vehicles or are mounted in industrial robots to play important roles in automatic control of the robots. Below, the invention will be explained with reference to the example of an image recognition system mounted in a vehicle.
- Image recognition systems mounted in vehicles are used to recognize the cruising environment in front of the vehicles or to measure the range to or azimuth of obstacles in front of the vehicles.
- an image input unit camera
- image recognition unit image recognition sensor
- an image recognition sensor is also utilized as a hybrid sensor combined with milliwave or other radar used to assist vehicle cruising. Further, an image recognition sensor is also utilized as part of other control devices, for example, the automatic cruise control systems of vehicles similarly used to assist vehicle cruising.
- the above problem may arise when for example one or more cords in the above 8-bit cable breaks, when there is pixel burn in the image sensor of the image input unit (camera), etc.
- An object of the present invention is to provide a diagnostic apparatus for abnormalities in an image recognition system able to obtain a quick and accurate grasp of abnormalities.
- a diagnostic apparatus provided with a histogram computing means ( 11 ) for performing predetermined computations on an image signal fetched into an image recognition unit ( 3 ) to generate a density histogram and an abnormality judging means ( 12 ) for analyzing the density histogram generated by the histogram computing means ( 11 ) to judge the presence of an abnormality and, as a second aspect of the invention, a diagnostic apparatus provided with a memory means ( 21 ) for receiving as input and storing information of an image signal fetched into an image recognition unit ( 3 ) and a comparing/judging means ( 22 ) for reading out and comparing information of image signals of a plurality of images different in time relating to a target pixel from the memory means ( 21 ) and judging whether they are the same or different and, in that case, judging that there is an abnormality when they are the same.
- FIG. 1 is a view of the basic configuration of a diagnostic apparatus for abnormalities in an image recognition system according to the present invention
- FIG. 2 is a first view of a density histogram for explaining Embodiment 1 of the present invention
- FIG. 3 is a second view of a density histogram for explaining Embodiment 1 of the present invention.
- FIG. 4 is a third view of a density histogram for explaining Embodiment 1 of the present invention.
- FIG. 5 is a fourth view of a density histogram for explaining Embodiment 1 of the present invention.
- FIG. 6 is a view of an input image used as an example for obtaining the histograms shown in FIG. 2 to FIG. 5;
- FIG. 7 is a view of a group of pixels on an image sensor outputting an input image
- FIG. 8 is a flow chart of an example of the operation of Embodiment 2;
- FIG. 9 is a flow chart of an example of the operation of Embodiment 3.
- FIG. 10 is a flow chart of an example of the operation of Embodiment 4.
- FIG. 11 is a view illustrating a case effective for working Embodiment 6;
- FIG. 12 is a view of a diagnostic apparatus for abnormalities according to Embodiment 7;
- FIG. 13( a ) and FIG. 13( b ) are views of examples of input images when pixel burn occurs;
- FIG. 14 is a view of an example of division of an image
- FIG. 15 is a flow chart of an example of operation of Embodiment B.
- FIG. 16 is a view of a diagnostic apparatus for abnormalities according to Embodiment C;
- FIG. 17( a ) and FIG. 17( b ) are flow charts of an example of operation of Embodiment C;
- FIG. 18 is a view of a diagnostic apparatus for abnormalities according to Embodiment D;
- FIG. 19( a ) to FIG. 19( d ) are views showing simply the operation of Embodiment D;
- FIG. 20 is a view of a diagnostic apparatus for abnormalities according to Embodiment H.
- FIG. 21( a ) to FIG. 21( c ) are views for explaining interpolation in Embodiment H.
- FIG. 1 is a view of the basic configuration of a diagnostic apparatus for abnormalities in an image recognition system according to the present invention. Note that in the present invention, a first aspect for dealing with breakage of a cable and a second aspect for dealing with pixel burn of the image sensor are separately proposed, but FIG. 1 shows a configuration combining these two aspects. The present invention however can be worked as only the first aspect or worked as only the second aspect. Of course, if combining the two aspects, better results of diagnosis of abnormalities can be obtained.
- reference numeral 1 shows an image recognition system. This includes an image input unit (camera) 2 and an image recognition unit 3 for receiving as input an image signal IM (image) from the image input unit 2 and recognizing the image. It further includes a diagnostic apparatus 4 for diagnosing abnormalities in the image recognition system 1 .
- the diagnostic apparatus 4 as shown at the left side in the figure, is provided with a histogram computing means 11 for performing predetermined computation on the image signal IM fetched into the image recognition unit 3 to generate a density histogram and an abnormality judging means 12 for analyzing the density histogram generated at the histogram computing means 11 to judge the presence of an abnormality.
- the above configuration forms the already mentioned first aspect of the invention.
- the abnormality is for example breakage of any of the cords C 1 , C 2 , C 3 . . . C 8 of the 8-bit cable.
- the diagnostic apparatus 4 is provided with a memory means 21 for receiving as input and storing information of an image signal IM fetched into an image recognition unit 3 and a comparing/judging means 22 for taking note of one pixel among a plurality of pixels forming the above image, reading out and comparing information of image signals IM of a plurality of images (frame) different in time relating to that noted pixel from the memory means 21 , and judging whether they are the same or different.
- the apparatus judges that there is an abnormality when they are the same.
- the above configuration forms the already explained second aspect of the invention.
- the abnormality is pixel burn occurring in the image sensor of the image input unit (camera) 2 .
- both the first aspect and second aspect of the diagnostic apparatus 4 are shown led out to the center of FIG. 1 as independent units for easier understanding, but in actuality they may be formed integrally as part of the image recognition unit 3 .
- the controlled object 5 is the already explained control device. If this control device is for example a tracking control device for a vehicle in the front, the apparatus immediately actuates a braking operation or other safety operation when receiving the above alarm signal.
- FIG. 2 to FIG. 5 are views of density histograms for explaining Embodiment 1
- FIG. 6 is a view of an input image used as an example for obtaining the histograms of FIG. 2 to FIG. 5
- FIG. 7 is a view of a group of pixels on an image sensor for outputting an input image.
- FIG. 6 An example of the rear end of a vehicle cruising in front captured by the image sensor (FIG. 7) of the image input unit (camera) 2 mounted in a vehicle is shown as an input image in FIG. 6.
- the input image shown in FIG. 6 is generated by the outputs of pixels (photo diodes) of the image sensor forming the light receiving portion of the image input unit 2 shown in FIG. 1. These pixels, as shown by P in FIG. 7, are arranged for example in a matrix of 640 ⁇ 480 pixels. Further, the outputs of these pixels P are expressed as 8 bits of a signal. These eight bits of the signal are transferred in parallel by the eight-bit cords C 1 , C 2 , C 3 . . . C 8 (FIG. 1).
- FIG. 3 shows a density histogram when the cord C 1 among the eight cords breaks.
- the cord C 1 is for the least significant bit (LSB). In this case, every other value in the profile is missing.
- FIG. 4 shows a density histogram when the cord C 2 among the eight cords breaks.
- the cord C 2 is for the second bit. In this case, every other two values in the profile are missing.
- FIG. 5 shows a density histogram when the cord C 8 among the eight cords breaks.
- the cord C 8 is for the most significant bit (MSB).
- MSB most significant bit
- the abnormality detecting means 12 of FIG. 1 analyzes the density histogram defined by the number of appearances of the pixels comprising the image for each gradation (0 to 255) and judges that there is an abnormality when not detecting that “a corresponding pixel P appears for all gradations (corresponding to FIG. 2)” (i.e., corresponding to FIG. 3 to FIG. 5).
- the means judges that there is an abnormality when confirming that “a corresponding pixel P appears for all gradations (0 to 255)” is not detected a plurality of consecutive times.
- FIG. 8 is a flow chart of an example of the operation of Embodiment 2.
- the flow chart is comprised of steps S 11 to S 17 .
- S 11 Start the computation by the histogram computing means 11 (FIG. 1).
- S 12 Judge if there is any gradation not output.
- S 13 If there isn't any gradation not output (NO at S 12 ), initialize the counter.
- the counter is not shown, but can be provided as software or hardware in the abnormality judging means 12 of FIG. 1.
- S 15 When it is judged at S 12 that there is a gradation not output (YES at S 12 ), increment the counter and judge whether or not the judgment was performed for exactly the prescribed consecutive number of times T.
- S 16 If continuing for T or more times, judge that there is a breakage abnormality.
- FIG. 9 is a flow chart of an example of the operation of Embodiment 3.
- the flow chart is comprised of steps S 21 to S 26 .
- S 21 Fetch the input image from the image input unit (camera) 2 of FIG. 1 to the image recognition unit 3 .
- S 22 Judge if the current time t has reached T 1 .
- T 1 is the time interval of discrete sampling.
- S 26 Here, start the inherent image recognition processing.
- Embodiment 4 whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected suitably after input of power and also detected together with an existing initial check on the image input unit 2 and image recognition unit 3 at the time of turning on the power before that.
- FIG. 10 is a flow chart of an example of the operation of Embodiment 4.
- steps S 31 to S 33 are characterizing steps of this example.
- steps S 34 to S 38 are substantially the same as in the above FIG. 9.
- S 31 Turn on the power of the vehicle.
- S 32 Execute the breakage judgment processing (for example, FIG. 8).
- S 33 When there is a breakage abnormality (YES), the routine ends. When there isn't one (NO), the routine proceeds to S 34 .
- the routine after S 34 is the same as in FIG. 9.
- Embodiment 4 the breakage judgment processing is performed two times (S 32 and S 35 ). This would appear at first glance to invite an increase in the processing load of the CPU, but usually when starting a vehicle, there is a routine for automatically executing an initial check of the image recognition system, in particular the image input unit 2 and the image recognition unit 3 of FIG. 1, so if executing the breakage judgment processing S 32 making use of this routine, the processing load of the CPU will not become that large.
- a look at the prior breakage judgment processing S 32 and the ordinary breakage judgment processing S 35 shows that the two differ completely in regards to the environment in which the vehicle is placed. That is, in the latter (S 35 ), the vehicle is cruising, while in the former (S 32 ), the vehicle is normally in the garage.
- Embodiment 5 whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected selectively only when it is judged that the fluctuations in the density histogram are large in accordance with the surroundings.
- Embodiment 6 when it is judged from the surroundings that the fluctuations in the density histogram are small, whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected after determining that any of the shutter speed, automatic gain control (AGC), or iris has been operated.
- AGC automatic gain control
- Embodiment 6 it is determined when the density of the input image will become larger and the breakage judgment processing is executed at that time.
- FIG. 11 is a view illustrating a case effective for working Embodiment 6.
- FIG. 31 shows the image sensor portion of the image input unit (camera) 2 of FIG. 1, in particular the structures of the front end and rear end.
- reference numeral 31 is a lens.
- iris 32 for adjusting the amount of light striking the image sensor 33
- shutter for adjusting the total amount of the incident light
- AGC circuit 34 for applying AGC to the image signal input from the image sensor 33 .
- the shutter may be a typical mechanical shutter, but in the figure an electronic type shutter 35 for controlling the exposure time is illustrated.
- Embodiment 6 like Embodiment 3, also has the advantage of being able to reduce the processing load of the CPU.
- Embodiment 7 further provides an abnormality locating means for judging if a density profile pattern shown by the density histogram at the time of detection has a trend close to any of a plurality of abnormality patterns envisioned in advance when detecting whether “a corresponding pixel P appears for all of the gradations (0 to 255)” and, when having that trend, locating the abnormality envisioned unambiguously from the abnormality patterns.
- FIG. 12 is a view of a diagnostic apparatus for abnormalities according to an embodiment 7.
- Embodiment 7 is configured as configured in FIG. 1 plus the above abnormality locating means 41 .
- the abnormality locating means 41 concludes that no abnormality has occurred and the situation is normal when judging that the density profile pattern shown by the density histogram at the time of detection does not have a trend close to any of the above-mentioned plurality of abnormality patterns even if the abnormality judging means 12 detects that a corresponding pixel P does not appear for any of the gradations (0 to 255).
- the abnormality judging means 41 takes note of only for example the initial gradation 0 and the final gradation 255 and judges the appearance of a corresponding pixel P for them even when gradations (0 to 255) are set corresponding to the plurality of stages from the initial gradation 0 to the final gradation 255. At that time, the abnormality locating means 41 locates a unique abnormality when judging that the pixel P does not appear.
- the output has 256 gradations (0 to 255).
- the gradations on the input image become 0 to 128.
- the output gradations skip values and become the 128 gradations of 0, 2, 4 . . . 254. In this way, in the case of breakage, some sort of trend appears, so it is possible to judge breakage based on this trend.
- the abnormality covered by the second aspect is pixel burn occurring in the image sensor 33 of the image input unit (camera) 2 . Further, to detect such pixel burn, as shown at the right side of the above-mentioned FIG.
- a memory means 21 for receiving as input and storing the information of the image signal IM fetched into the image recognition unit 3 and a comparing/judging means 22 for taking note of one pixel in the plurality of pixels forming an image, reading out and comparing information of image signals IM of a plurality of images (frames) different in time relating to a target pixel from the memory means 21 , and judging whether the information are the same or different.
- the apparatus judges that there is an abnormality when they are the same.
- FIG. 13( a ) and FIG. 13( b ) are views showing examples of input images when pixel burn occurs.
- FIG. 13( a ) and FIG. 13( b ) are the portions surrounded by the circles R. Three black dots appear. These black dots appear at the portions of the same pixels both at the time t and at a time t+1 after the elapse of time from this. This means pixel burn.
- the comparing/judging means 22 detects any change between information of a current image signal and information of a prior image signal within a certain time (for example, 1 second) set in advance for the same pixel and judges that there is an abnormality when there is no change.
- the comparing/judging means 22 divides an image in advance into a plurality of sub-image areas, shifts each sub-image area in time, and then executes the comparison and judgment for each of these.
- FIG. 14 is a view of an example of division of an image. Note that the input image covered is the same as that shown in FIG. 6 and FIG. 13.
- FIG. 15 is a flow chart of an example of the operation of Embodiment B.
- the flow chart is comprised of steps S 41 to S 47 .
- S 41 Set one of the four search areas ( 51 to 54 ) shown in FIG. 14. First, set the area 51 .
- S 42 Perform the processing for judging pixel burn for the area 51 .
- S 43 Increment the time count for the above-mentioned “time shift” (counter formed as software or hardware in comparing/judging means 22 ).
- S 44 Judge if that time is over the above certain time (for example, 1 second) Tc.
- S 47 Reset the time t to 0 again, switch the search area at S 42 from the area 51 to the area 52 , and repeat the above in the same way for S 43 ⁇ S 44 ⁇ S 45 ⁇ S 46 ⁇ S 47 .
- S 47 next switch from the area 52 to the area 53 .
- Embodiment C further provision is made of an abnormal pixel candidate memory means for storing only a plurality of abnormal pixel candidates judged once to be abnormal by the comparing/judging means 22 .
- the comparing/judging means 22 continues to repeatedly execute comparison and judgment only for the abnormal pixel candidates in the abnormal pixel candidate memory means. Each time it is judged that the information are different, the abnormal pixel candidate is erased from the abnormal pixel candidate memory means.
- FIG. 16 is a view of a diagnostic apparatus for abnormalities according to Embodiment C.
- Embodiment C is configured as configured in FIG. 1 (second aspect of the invention) plus the above abnormality pixel candidate memory means 61 .
- the pixel burn judgment processing by the comparing/judging means 22 (for example, S 45 and S 46 of FIG. 15) is repeated at a certain set cycle. It records candidates of pixels with no change in gradation first in the memory means 61 . Pixels with changes in gradations among these candidates are successively erased with each repetition of this cycle. The candidate pixels remaining at the end are the true burn pixels sought. This operation is shown in FIG. 17.
- FIG. 17( a ) and FIG. 17( b ) are flow charts of an example of the operation of Embodiment C. This is comprised of steps S 51 to S 53 .
- S 51 Extract the position of the coordinates with changes in gradation by the comparing/judging means 22 .
- S 52 Refer to the memory means 21 and compare these with the previous data.
- Embodiment D further provision is made of a no-change flag recording means for recording corresponding to each pixel a no-change flag showing that it has been judged once by the comparing/judging means 22 that there is no change between information.
- the comparing/judging means 22 continues to repeatedly execute the comparison and judgment only for pixels with no-change flags in the no-change flag recording means. Each time it is judged that the information are different, the no-change flag corresponding to that pixel is erased from the no-change flag recording means.
- FIG. 18 is a view of a diagnostic apparatus for abnormalities according to an Embodiment D.
- Embodiment D is configured as configured in FIG. 1 (second aspect of the invention) plus the above no-change flag recording means 71 .
- Embodiment D While Embodiment C recorded the data shown by FIG. 17( b ), Embodiment D records flags showing if there has been a change in the corresponding pixels in the recording means 71 . The flags show only whether there has been a change, so 1 bit is sufficient. Therefore, Embodiment D enables a great reduction in the required memory capacity.
- FIGS. 19 ( a ), 19 ( b ), 19 ( c ), and 19 ( d ) are views simply showing the operation of Embodiment D.
- FIG. 19( b ) is a record of the no-change flags (shown by hatching) obtained by the current pixel burn judgment processing.
- FIG. 19( d ) shows another example of the results of the “prior pixel burn judgment processing”.
- efficient processing is possible by comparing and processing only the three pixels with no change of the previous time.
- Embodiment E the above comparison and judgment are executed suitably after input of power and also executed together with an existing initial check on the image input unit 2 and image recognition unit 3 at the time of turning on the power before that.
- Embodiment E is exactly the same in content and effect as in Embodiment 4.
- Embodiment F the above-mentioned comparison and judgment are selectively performed only when it is judged that the information of the image signal is obtained effectively in accordance with the surroundings.
- Embodiment F is exactly the same in content and effect as in Embodiment 5.
- Embodiment G the above comparison and judgment are executed after obtaining a grasp of operation of any of the shutter speed, AGC, or iris at the image input unit 2 .
- Embodiment G is exactly the same in content and effect as in Embodiment 6.
- Embodiment H will be explained.
- a pixel information reproducing means for reproducing information of a pixel signal which should have been obtained from a pixel P judged to be abnormal by interpolation from information on pixel signals obtained from adjacent pixels surrounding the pixel P.
- FIG. 20 is a view of a diagnostic apparatus for abnormalities according to Embodiment H.
- Embodiment H is configured as configured in FIG. 1 (second aspect of invention) plus the above image information reproducing means 81 . Note that the illustration of the alarm (AL) signal line is omitted.
- FIG. 21( a ), FIG. 21( b ), and FIG. 21( c ) are views for explaining the interpolation in Embodiment H.
- FIG. 21( a ) shows that the pixel at the coordinates (x+1, y) is a burned pixel P′.
- the gradation of the burned pixel P′ is interpolated from the gradations of the surrounding pixels, that is, P(x,y) and P(x+2,y). This state is shown in FIG. 21( b ).
- Embodiment H is preferably applied to the portion of an image not having that much of an effect on image recognition, for example, a portion away from the center of the image such as shown by R in FIG. 13( a ) and FIG. 13( b ).
Abstract
Description
- 1. Field of the Invention
- The present invention relates to a diagnostic apparatus for abnormalities in an image recognition system.
- 2. Description of the Related Art
- Image recognition systems are used in various fields. For example, they are mounted in vehicles to assist the safe cruising of the vehicles or are mounted in industrial robots to play important roles in automatic control of the robots. Below, the invention will be explained with reference to the example of an image recognition system mounted in a vehicle.
- Image recognition systems mounted in vehicles are used to recognize the cruising environment in front of the vehicles or to measure the range to or azimuth of obstacles in front of the vehicles. In this case, an image input unit (camera) and image recognition unit (image recognition sensor) are essential for such image recognition systems.
- Note that an image recognition sensor is also utilized as a hybrid sensor combined with milliwave or other radar used to assist vehicle cruising. Further, an image recognition sensor is also utilized as part of other control devices, for example, the automatic cruise control systems of vehicles similarly used to assist vehicle cruising.
- Summarizing the problems to be solved by the invention, when certain components of the image input unit (camera) and image recognition unit, in particular the image input unit and the cable (normally an 8-bit cable) for connecting this with the image recognition unit, are mounted in a vehicle, they are subject to extremely harsh usage environments. Therefore, the probability of faults occurring the image input unit and cable becomes higher. This being the case, the problem arises that when the image recognition system is used for assisting vehicle cruising, the safety of the cruising vehicle is liable to be impaired by such faults or other abnormalities.
- For example, the above problem may arise when for example one or more cords in the above 8-bit cable breaks, when there is pixel burn in the image sensor of the image input unit (camera), etc.
- An object of the present invention is to provide a diagnostic apparatus for abnormalities in an image recognition system able to obtain a quick and accurate grasp of abnormalities.
- To attain the above object, there are provided, as a first aspect of the invention, a diagnostic apparatus provided with a histogram computing means (11) for performing predetermined computations on an image signal fetched into an image recognition unit (3) to generate a density histogram and an abnormality judging means (12) for analyzing the density histogram generated by the histogram computing means (11) to judge the presence of an abnormality and, as a second aspect of the invention, a diagnostic apparatus provided with a memory means (21) for receiving as input and storing information of an image signal fetched into an image recognition unit (3) and a comparing/judging means (22) for reading out and comparing information of image signals of a plurality of images different in time relating to a target pixel from the memory means (21) and judging whether they are the same or different and, in that case, judging that there is an abnormality when they are the same.
- These and other objects and features of the present invention will become clearer from the following description of the preferred embodiments given with reference to the attached drawings, wherein:
- FIG. 1 is a view of the basic configuration of a diagnostic apparatus for abnormalities in an image recognition system according to the present invention;
- FIG. 2 is a first view of a density histogram for explaining
Embodiment 1 of the present invention; - FIG. 3 is a second view of a density histogram for explaining
Embodiment 1 of the present invention; - FIG. 4 is a third view of a density histogram for explaining
Embodiment 1 of the present invention; - FIG. 5 is a fourth view of a density histogram for explaining
Embodiment 1 of the present invention; - FIG. 6 is a view of an input image used as an example for obtaining the histograms shown in FIG. 2 to FIG. 5;
- FIG. 7 is a view of a group of pixels on an image sensor outputting an input image;
- FIG. 8 is a flow chart of an example of the operation of
Embodiment 2; - FIG. 9 is a flow chart of an example of the operation of
Embodiment 3; - FIG. 10 is a flow chart of an example of the operation of
Embodiment 4; - FIG. 11 is a view illustrating a case effective for working Embodiment 6;
- FIG. 12 is a view of a diagnostic apparatus for abnormalities according to Embodiment 7;
- FIG. 13(a) and FIG. 13(b) are views of examples of input images when pixel burn occurs;
- FIG. 14 is a view of an example of division of an image;
- FIG. 15 is a flow chart of an example of operation of Embodiment B;
- FIG. 16 is a view of a diagnostic apparatus for abnormalities according to Embodiment C;
- FIG. 17(a) and FIG. 17(b) are flow charts of an example of operation of Embodiment C;
- FIG. 18 is a view of a diagnostic apparatus for abnormalities according to Embodiment D;
- FIG. 19(a) to FIG. 19(d) are views showing simply the operation of Embodiment D;
- FIG. 20 is a view of a diagnostic apparatus for abnormalities according to Embodiment H; and
- FIG. 21(a) to FIG. 21(c) are views for explaining interpolation in Embodiment H.
- Preferred embodiments of the present invention will be described in detail below while referring to the attached figures.
- FIG. 1 is a view of the basic configuration of a diagnostic apparatus for abnormalities in an image recognition system according to the present invention. Note that in the present invention, a first aspect for dealing with breakage of a cable and a second aspect for dealing with pixel burn of the image sensor are separately proposed, but FIG. 1 shows a configuration combining these two aspects. The present invention however can be worked as only the first aspect or worked as only the second aspect. Of course, if combining the two aspects, better results of diagnosis of abnormalities can be obtained.
- In FIG. 1,
reference numeral 1 shows an image recognition system. This includes an image input unit (camera) 2 and animage recognition unit 3 for receiving as input an image signal IM (image) from theimage input unit 2 and recognizing the image. It further includes adiagnostic apparatus 4 for diagnosing abnormalities in theimage recognition system 1. - The
diagnostic apparatus 4, as shown at the left side in the figure, is provided with a histogram computing means 11 for performing predetermined computation on the image signal IM fetched into theimage recognition unit 3 to generate a density histogram and an abnormality judging means 12 for analyzing the density histogram generated at the histogram computing means 11 to judge the presence of an abnormality. - The above configuration forms the already mentioned first aspect of the invention. The abnormality is for example breakage of any of the cords C1, C2, C3 . . . C8 of the 8-bit cable.
- Further, the
diagnostic apparatus 4, as shown at the right side in the figure, is provided with a memory means 21 for receiving as input and storing information of an image signal IM fetched into animage recognition unit 3 and a comparing/judging means 22 for taking note of one pixel among a plurality of pixels forming the above image, reading out and comparing information of image signals IM of a plurality of images (frame) different in time relating to that noted pixel from the memory means 21, and judging whether they are the same or different. The apparatus judges that there is an abnormality when they are the same. - The above configuration forms the already explained second aspect of the invention. The abnormality is pixel burn occurring in the image sensor of the image input unit (camera)2.
- Note that both the first aspect and second aspect of the
diagnostic apparatus 4 are shown led out to the center of FIG. 1 as independent units for easier understanding, but in actuality they may be formed integrally as part of theimage recognition unit 3. - The actions of the first aspect and second aspect of the present invention will be explained in detail below with reference to
Embodiments 1 to 7 and Embodiments A to H. - Supplementing the explanation of FIG. 1 somewhat, when the diagnosis by the
diagnostic apparatus 4 results in detection of an abnormality, the occurrence of that abnormality is immediately informed to the driver of the vehicle by an alarm AL and an alarm signal is instantaneously supplied to the illustrated controlledobject 5 as well. - The controlled
object 5 is the already explained control device. If this control device is for example a tracking control device for a vehicle in the front, the apparatus immediately actuates a braking operation or other safety operation when receiving the above alarm signal. - FIG. 2 to FIG. 5 are views of density histograms for explaining
Embodiment 1, FIG. 6 is a view of an input image used as an example for obtaining the histograms of FIG. 2 to FIG. 5, and FIG. 7 is a view of a group of pixels on an image sensor for outputting an input image. - An example of the rear end of a vehicle cruising in front captured by the image sensor (FIG. 7) of the image input unit (camera)2 mounted in a vehicle is shown as an input image in FIG. 6.
- The input image shown in FIG. 6 is generated by the outputs of pixels (photo diodes) of the image sensor forming the light receiving portion of the
image input unit 2 shown in FIG. 1. These pixels, as shown by P in FIG. 7, are arranged for example in a matrix of 640×480 pixels. Further, the outputs of these pixels P are expressed as 8 bits of a signal. These eight bits of the signal are transferred in parallel by the eight-bit cords C1, C2, C3 . . . C8 (FIG. 1). - When none of these cords C1 to C8 is broken, that is, when the cable is normal, the density histogram exhibits a clean peaked curve as shown in FIG. 2. Note that a density histogram, as is well known, is expressed by the density values on the abscissa and the number of appearances of pixels P exhibiting those density values in a single image on the ordinate. When one pixel is expressed by 8 bits, there are 256 types of density values from 0 to 255. Normally, these are called “gradations”. In the present invention, the term gradation is mainly used.
- When none of the cords C1, C2, C3 . . . C8 is broken (when the cable is normal), the density histogram exhibits the clean peaked curve (FIG. 2) as explained above, but if any of these cords C1, C2, C3, . . . C8 breaks (when the cable is abnormal), that peaked curve disappears. Several examples are shown in FIG. 3 to FIG. 5.
- FIG. 3 shows a density histogram when the cord C1 among the eight cords breaks. Here, the cord C1 is for the least significant bit (LSB). In this case, every other value in the profile is missing.
- FIG. 4 shows a density histogram when the cord C2 among the eight cords breaks. Here, the cord C2 is for the second bit. In this case, every other two values in the profile are missing.
- FIG. 5 shows a density histogram when the cord C8 among the eight cords breaks. Here, the cord C8 is for the most significant bit (MSB). In this case, half of the peaked curve of FIG. 2 is cut away in the profile.
- Therefore, in
Embodiment 1, the abnormality detecting means 12 of FIG. 1 analyzes the density histogram defined by the number of appearances of the pixels comprising the image for each gradation (0 to 255) and judges that there is an abnormality when not detecting that “a corresponding pixel P appears for all gradations (corresponding to FIG. 2)” (i.e., corresponding to FIG. 3 to FIG. 5). - In
Embodiment 2, the means judges that there is an abnormality when confirming that “a corresponding pixel P appears for all gradations (0 to 255)” is not detected a plurality of consecutive times. - Immediately judging there is an abnormality by the fact that there happens to be a pixel where the value of a gradation does not appear would be inaccurate, so an abnormality is judged only after confirming that the result of FIG. 3 or FIG. 4 is obtained or the result of FIG. 2 is not obtained a plurality of consecutive times. By this, it is possible to improve the accuracy of the judgment of abnormalities.
- FIG. 8 is a flow chart of an example of the operation of
Embodiment 2. The flow chart is comprised of steps S11 to S17. - S11: Start the computation by the histogram computing means 11 (FIG. 1).
- S12: Judge if there is any gradation not output.
- S13: If there isn't any gradation not output (NO at S12), initialize the counter. The counter is not shown, but can be provided as software or hardware in the abnormality judging means 12 of FIG. 1.
- S14: In this case, the situation is judged normal.
- S15: When it is judged at S12 that there is a gradation not output (YES at S12), increment the counter and judge whether or not the judgment was performed for exactly the prescribed consecutive number of times T.
- S16: If continuing for T or more times, judge that there is a breakage abnormality.
- S17: If not more than T, increment the counter and continue with S12.
- In
Embodiment 3, whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected discretely at predetermined sampling intervals. - If performing the above detection substantially consecutively, finer detection is possible, but the processing load of the CPU, forming the main part of the
diagnostic apparatus 4, ends up increasing. Therefore, inEmbodiment 3, this detection is performed discretely at intervals by sampling. - FIG. 9 is a flow chart of an example of the operation of
Embodiment 3. The flow chart is comprised of steps S21 to S26. - S21: Fetch the input image from the image input unit (camera) 2 of FIG. 1 to the
image recognition unit 3. - S22: Judge if the current time t has reached T1. Here, T1 is the time interval of discrete sampling.
- S23: When t does not exceed T1 at S22 (YES), continue to count the current time.
- S24: If t exceeds T1 (NO), execute the breakage judgment processing (for example, FIG. 8).
- S25: Return the above time t to 0 each time the above breakage judgment processing is executed.
- S26: Here, start the inherent image recognition processing.
- Due to this, discrete diagnosis of abnormalities lightening the processing load of the CPU is performed.
- In
Embodiment 4, whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected suitably after input of power and also detected together with an existing initial check on theimage input unit 2 andimage recognition unit 3 at the time of turning on the power before that. - Breakage abnormalities often occur during cruising of a vehicle, but sometimes already are occurring (or occur) when turning on the power in the vehicle. This fact is taken note of by
Embodiment 4. - FIG. 10 is a flow chart of an example of the operation of
Embodiment 4. In particular, steps S31 to S33 are characterizing steps of this example. Steps S34 to S38 are substantially the same as in the above FIG. 9. - S31: Turn on the power of the vehicle.
- S32: Execute the breakage judgment processing (for example, FIG. 8).
- S33: When there is a breakage abnormality (YES), the routine ends. When there isn't one (NO), the routine proceeds to S34. The routine after S34 is the same as in FIG. 9.
- Therefore, in
Embodiment 4, the breakage judgment processing is performed two times (S32 and S35). This would appear at first glance to invite an increase in the processing load of the CPU, but usually when starting a vehicle, there is a routine for automatically executing an initial check of the image recognition system, in particular theimage input unit 2 and theimage recognition unit 3 of FIG. 1, so if executing the breakage judgment processing S32 making use of this routine, the processing load of the CPU will not become that large. - Here, a look at the prior breakage judgment processing S32 and the ordinary breakage judgment processing S35 shows that the two differ completely in regards to the environment in which the vehicle is placed. That is, in the latter (S35), the vehicle is cruising, while in the former (S32), the vehicle is normally in the garage.
- This being the case, in the state with the vehicle in a garage, it would seem that a sufficient density histogram could not be obtained and effective breakage judgment processing (S32) could not be performed. It is true that the effect of the processing (S32) is smaller than the processing (S35). However, normally, an input image constantly finely fluctuates and therefore judgment of the presence of a breakage abnormality is possible.
- In
Embodiment 5, whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected selectively only when it is judged that the fluctuations in the density histogram are large in accordance with the surroundings. - When the vehicle is placed in surroundings where only output of an image of dark gradations can be obtained such as at night, there is a possibility that all gradations will not be output.
- Therefore, it is possible to judge whether it is daytime or nighttime from (i) the signal of the input image, (ii) the signal of the sensor for turning on the headlight of the vehicle automatically in dark surroundings, that is, the so-called light control system, etc. and selectively detect for breakage only in the daytime when the fluctuations in the density histogram are large. Alternatively, it is also possible to selectively detect for breakage only while the vehicle is cruising and the fluctuations in the density histogram are large based on the signal of the vehicle speed sensor.
- In Embodiment 6, when it is judged from the surroundings that the fluctuations in the density histogram are small, whether “a corresponding pixel P appears for all of the gradations (0 to 255)” is detected after determining that any of the shutter speed, automatic gain control (AGC), or iris has been operated.
- As explained above, if the surroundings of the vehicle are dark or the vehicle is at a stop, the fluctuations in the density histogram inevitably become smaller. Therefore, at such times, effective breakage judgment processing is difficult to realize.
- Therefore, in Embodiment 6, it is determined when the density of the input image will become larger and the breakage judgment processing is executed at that time.
- FIG. 11 is a view illustrating a case effective for working Embodiment 6.
- This figure shows the image sensor portion of the image input unit (camera)2 of FIG. 1, in particular the structures of the front end and rear end. In the figure,
reference numeral 31 is a lens. At the rear end, there is aniris 32 for adjusting the amount of light striking theimage sensor 33, a shutter for adjusting the total amount of the incident light, and anAGC circuit 34 for applying AGC to the image signal input from theimage sensor 33. Note that the shutter may be a typical mechanical shutter, but in the figure anelectronic type shutter 35 for controlling the exposure time is illustrated. - No matter which of these
components - Embodiment 6, like
Embodiment 3, also has the advantage of being able to reduce the processing load of the CPU. - Embodiment 7 further provides an abnormality locating means for judging if a density profile pattern shown by the density histogram at the time of detection has a trend close to any of a plurality of abnormality patterns envisioned in advance when detecting whether “a corresponding pixel P appears for all of the gradations (0 to 255)” and, when having that trend, locating the abnormality envisioned unambiguously from the abnormality patterns.
- FIG. 12 is a view of a diagnostic apparatus for abnormalities according to an embodiment 7.
- Embodiment 7 is configured as configured in FIG. 1 plus the above abnormality locating means41. Explaining this abnormality locating means a bit more in detail, the result is the following (i) or (ii).
- (i) The abnormality locating means41 concludes that no abnormality has occurred and the situation is normal when judging that the density profile pattern shown by the density histogram at the time of detection does not have a trend close to any of the above-mentioned plurality of abnormality patterns even if the abnormality judging means 12 detects that a corresponding pixel P does not appear for any of the gradations (0 to 255).
- (ii) The abnormality judging means41 takes note of only for example the
initial gradation 0 and thefinal gradation 255 and judges the appearance of a corresponding pixel P for them even when gradations (0 to 255) are set corresponding to the plurality of stages from theinitial gradation 0 to thefinal gradation 255. At that time, the abnormality locating means 41 locates a unique abnormality when judging that the pixel P does not appear. - In the case of the 8-bit cable (comprising cords C1 to C8) explained above, the output has 256 gradations (0 to 255). When breakage occurs, just the graduations of a certain bit of the signal can no longer be output. If the cord C8 for the MSB among the eight bits is disconnected, the gradations on the input image become 0 to 128. Similarly, if the cord C1 for the LSB is broken, the output gradations skip values and become the 128 gradations of 0, 2, 4 . . . 254. In this way, in the case of breakage, some sort of trend appears, so it is possible to judge breakage based on this trend.
- As opposed to this, even if there are gradations which are not output, it is possible to judge the situation is normal when there is no particular trend in the gradations of the pixel output.
- Further, if looking at the
gradation 0 and thegradation 255, the HIGH/LOW of all cords are reversed. From this, even if reducing the gradations searched through, it is possible to similarly judge the presence of any breakage. According to this, it becomes possible to judge normal/abnormal from the output results of fewer gradations. - The above explanation concerned the first aspect of the invention. Next, the second aspect of the invention will be explained in more detail. As already explained, the abnormality covered by the second aspect is pixel burn occurring in the
image sensor 33 of the image input unit (camera) 2. Further, to detect such pixel burn, as shown at the right side of the above-mentioned FIG. 1, provision is made of a memory means 21 for receiving as input and storing the information of the image signal IM fetched into theimage recognition unit 3 and a comparing/judging means 22 for taking note of one pixel in the plurality of pixels forming an image, reading out and comparing information of image signals IM of a plurality of images (frames) different in time relating to a target pixel from the memory means 21, and judging whether the information are the same or different. The apparatus judges that there is an abnormality when they are the same. - Here, to accelerate understanding, the state of an abnormality due to pixel burn is shown in the figures.
- FIG. 13(a) and FIG. 13(b) are views showing examples of input images when pixel burn occurs.
- This example shows an input image capturing the rear end of a vehicle in the same way as FIG. 6. What should be noted in FIG. 13(a) and FIG. 13(b) are the portions surrounded by the circles R. Three black dots appear. These black dots appear at the portions of the same pixels both at the time t and at a time t+1 after the elapse of time from this. This means pixel burn.
- In Embodiment A, the comparing/judging means22 (FIG. 1) detects any change between information of a current image signal and information of a prior image signal within a certain time (for example, 1 second) set in advance for the same pixel and judges that there is an abnormality when there is no change.
- In Embodiment B, the comparing/judging means22 divides an image in advance into a plurality of sub-image areas, shifts each sub-image area in time, and then executes the comparison and judgment for each of these.
- FIG. 14 is a view of an example of division of an image. Note that the input image covered is the same as that shown in FIG. 6 and FIG. 13.
- In the example of FIG. 14, the case is shown of dividing the image into the four
sub-image areas sub-image areas 51 to 54. Therefore, the required capacity of the memory means 21 (FIG. 1) need only be one-quarter the regular capacity. The operation will be explained below. - FIG. 15 is a flow chart of an example of the operation of Embodiment B. The flow chart is comprised of steps S41 to S47.
- S41: Set one of the four search areas (51 to 54) shown in FIG. 14. First, set the
area 51. - S42: Perform the processing for judging pixel burn for the
area 51. - S43: Increment the time count for the above-mentioned “time shift” (counter formed as software or hardware in comparing/judging means 22).
- S44: Judge if that time is over the above certain time (for example, 1 second) Tc.
- S45: If not over it (YES at S44), continue the pixel burn judgment processing.
- S46: If over it (NO at S44), the result of pixel burn judgment processing is determined.
- S47: Reset the time t to 0 again, switch the search area at S42 from the
area 51 to thearea 52, and repeat the above in the same way for S43 → S44 → S45 → S46 → S47. At S47, next switch from thearea 52 to thearea 53. - In Embodiment C, further provision is made of an abnormal pixel candidate memory means for storing only a plurality of abnormal pixel candidates judged once to be abnormal by the comparing/judging means22. The comparing/judging means 22 continues to repeatedly execute comparison and judgment only for the abnormal pixel candidates in the abnormal pixel candidate memory means. Each time it is judged that the information are different, the abnormal pixel candidate is erased from the abnormal pixel candidate memory means.
- FIG. 16 is a view of a diagnostic apparatus for abnormalities according to Embodiment C.
- Embodiment C is configured as configured in FIG. 1 (second aspect of the invention) plus the above abnormality pixel candidate memory means61.
- The pixel burn judgment processing by the comparing/judging means22 (for example, S45 and S46 of FIG. 15) is repeated at a certain set cycle. It records candidates of pixels with no change in gradation first in the memory means 61. Pixels with changes in gradations among these candidates are successively erased with each repetition of this cycle. The candidate pixels remaining at the end are the true burn pixels sought. This operation is shown in FIG. 17.
- FIG. 17(a) and FIG. 17(b) are flow charts of an example of the operation of Embodiment C. This is comprised of steps S51 to S53.
- S51: Extract the position of the coordinates with changes in gradation by the comparing/judging means 22.
- S52: Refer to the memory means 21 and compare these with the previous data.
- S53: Using the results of the comparison, store the data on candidates for abnormal pixels in the memory means 61 in the data format of FIG. 17(b) for pixels with no changes in gradation.
- In Embodiment D, further provision is made of a no-change flag recording means for recording corresponding to each pixel a no-change flag showing that it has been judged once by the comparing/judging means22 that there is no change between information. The comparing/judging means 22 continues to repeatedly execute the comparison and judgment only for pixels with no-change flags in the no-change flag recording means. Each time it is judged that the information are different, the no-change flag corresponding to that pixel is erased from the no-change flag recording means.
- FIG. 18 is a view of a diagnostic apparatus for abnormalities according to an Embodiment D.
- Embodiment D is configured as configured in FIG. 1 (second aspect of the invention) plus the above no-change flag recording means71.
- While Embodiment C recorded the data shown by FIG. 17(b), Embodiment D records flags showing if there has been a change in the corresponding pixels in the recording means 71. The flags show only whether there has been a change, so 1 bit is sufficient. Therefore, Embodiment D enables a great reduction in the required memory capacity.
- FIGS.19(a), 19(b), 19(c), and 19(d) are views simply showing the operation of Embodiment D.
- FIG. 19(b) is a record of the no-change flags (shown by hatching) obtained by the current pixel burn judgment processing.
- The record (b) of the no-change flags is compared with the record of the no-change flags the previous time (FIG. 19(a)) to obtain a record of no-change flags present at both of (a) and (b) (FIG. 19(c)).
- The record (c) of the no-change flags is used for the next pixel burn judgment processing. The pixels which should be noted at this time are reduced to two in the example of FIG. 19(c). Therefore, the processing load on the CPU becomes lighter.
- FIG. 19(d) shows another example of the results of the “prior pixel burn judgment processing”. In summary, even if the current processing is not performed for all pixels, efficient processing is possible by comparing and processing only the three pixels with no change of the previous time.
- In Embodiment E, the above comparison and judgment are executed suitably after input of power and also executed together with an existing initial check on the
image input unit 2 andimage recognition unit 3 at the time of turning on the power before that. - Note that Embodiment E is exactly the same in content and effect as in
Embodiment 4. - In Embodiment F, the above-mentioned comparison and judgment are selectively performed only when it is judged that the information of the image signal is obtained effectively in accordance with the surroundings.
- Note that Embodiment F is exactly the same in content and effect as in
Embodiment 5. - In Embodiment G, the above comparison and judgment are executed after obtaining a grasp of operation of any of the shutter speed, AGC, or iris at the
image input unit 2. - Note that Embodiment G is exactly the same in content and effect as in Embodiment 6.
- Finally, Embodiment H will be explained. In this Embodiment H, further provision is made of a pixel information reproducing means for reproducing information of a pixel signal which should have been obtained from a pixel P judged to be abnormal by interpolation from information on pixel signals obtained from adjacent pixels surrounding the pixel P.
- FIG. 20 is a view of a diagnostic apparatus for abnormalities according to Embodiment H.
- Embodiment H is configured as configured in FIG. 1 (second aspect of invention) plus the above image
information reproducing means 81. Note that the illustration of the alarm (AL) signal line is omitted. - FIG. 21(a), FIG. 21(b), and FIG. 21(c) are views for explaining the interpolation in Embodiment H.
- In the figures, FIG. 21(a) shows that the pixel at the coordinates (x+1, y) is a burned pixel P′.
- The gradation of the burned pixel P′ is interpolated from the gradations of the surrounding pixels, that is, P(x,y) and P(x+2,y). This state is shown in FIG. 21(b).
- One example of the interpolation formula used at this time is shown in FIG. 21(c). According to this formula, there are 81 (=(136+25)/2) reproduced gradations.
- Note that Embodiment H is preferably applied to the portion of an image not having that much of an effect on image recognition, for example, a portion away from the center of the image such as shown by R in FIG. 13(a) and FIG. 13(b).
- Summarizing the effects of the invention, as explained above, according to the present invention, it is possible to realize a diagnostic apparatus able to quickly and accurately detect abnormalities in an image recognition system such as breakage occurring at a cable from an image sensor or pixel burn in the image sensor.
- While the invention has been described with reference to specific embodiments chosen for purpose of illustration, it should be apparent that numerous modifications could be made thereto by those skilled in the art without departing from the basic concept and scope of the invention.
Claims (19)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JPJP-2002-124615 | 2002-04-25 | ||
JP2002124615A JP3901573B2 (en) | 2002-04-25 | 2002-04-25 | Abnormality diagnosis device in image recognition system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20030202694A1 true US20030202694A1 (en) | 2003-10-30 |
US7174041B2 US7174041B2 (en) | 2007-02-06 |
Family
ID=29243724
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/421,083 Expired - Fee Related US7174041B2 (en) | 2002-04-25 | 2003-04-23 | Diagnostic apparatus for abnormalities in image recognition system |
Country Status (2)
Country | Link |
---|---|
US (1) | US7174041B2 (en) |
JP (1) | JP3901573B2 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050083432A1 (en) * | 2003-10-15 | 2005-04-21 | Fujitsu Ten Limited | Image processing device, operation supporting device, and operation supporting system |
US20100332494A1 (en) * | 2009-06-25 | 2010-12-30 | Goetz Graefe | Histogram processing by trend and pattern removal |
US20150302588A1 (en) * | 2014-04-18 | 2015-10-22 | Fujitsu Limited | Method of determining normality of imaging direction, and evaluation device of evaluating attachment state of imaging device |
US20160301923A1 (en) * | 2014-01-10 | 2016-10-13 | Hitachi Automotive Systems, Ltd. | In-Vehicle-Camera Image Processing Device |
US10027953B2 (en) | 2014-04-18 | 2018-07-17 | Fujitsu Limited | Determination method and determination apparatus |
US10230946B2 (en) | 2014-04-18 | 2019-03-12 | Fujitsu Limited | Method of detecting tilt of image capturing direction and apparatus of detecting tilt of image capturing direction |
CN111722074A (en) * | 2020-06-03 | 2020-09-29 | 四川蓝景光电技术有限责任公司 | LED identification label fault positioning method, device and system |
US10810728B2 (en) * | 2018-11-19 | 2020-10-20 | The Boeing Company | Method for using a vision system to evaluate shield trims on shielded cables |
CN113701890A (en) * | 2021-08-09 | 2021-11-26 | 国网上海市电力公司 | Sensing monitoring method and system for tracking cable hot spot and damage |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4130434B2 (en) * | 2004-11-30 | 2008-08-06 | 本田技研工業株式会社 | Abnormality detection device for imaging device |
JP2007102417A (en) * | 2005-10-03 | 2007-04-19 | Fujitsu Ten Ltd | Image processor |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4933983A (en) * | 1986-02-14 | 1990-06-12 | Canon Kabushiki Kaisha | Image data reading apparatus |
US5596654A (en) * | 1987-04-20 | 1997-01-21 | Fuji Photo Film Co., Ltd. | Method of determining desired image signal range based on histogram data |
US5734740A (en) * | 1994-10-31 | 1998-03-31 | University Of Florida | Method for automated radiographic quality assurance |
US6088473A (en) * | 1998-02-23 | 2000-07-11 | Arch Development Corporation | Method and computer readable medium for automated analysis of chest radiograph images using histograms of edge gradients for false positive reduction in lung nodule detection |
US6215900B1 (en) * | 1997-02-21 | 2001-04-10 | Direct Radiography Corp. | Image data processing for digital x-ray detectors |
US20010008562A1 (en) * | 1997-08-28 | 2001-07-19 | Qualia Computing, Inc | Joint optimization of parameters for the detection of clustered microcalcifications in digital mammograms |
US6353674B1 (en) * | 1997-06-05 | 2002-03-05 | Agfa-Gevaert | Method of segmenting a radiation image into direct exposure area and diagnostically relevant area |
US6449390B1 (en) * | 1997-09-24 | 2002-09-10 | Canon Kabushiki Kaisha | Image processing apparatus and method therefor |
US20030095695A1 (en) * | 2001-11-21 | 2003-05-22 | Arnold Ben A. | Hybrid calibration of tissue densities in computerized tomography |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS62203461A (en) | 1986-03-04 | 1987-09-08 | Canon Inc | Picture read system |
JPH06164846A (en) | 1992-11-19 | 1994-06-10 | Mita Ind Co Ltd | Abnormality detector in picture reader |
JPH06308036A (en) | 1993-04-22 | 1994-11-04 | Takamisawa Denki Seisakusho:Kk | Inspection method for liquid crystal display image |
JPH07131651A (en) | 1993-10-29 | 1995-05-19 | Canon Inc | Image forming device |
JP2001016446A (en) | 1999-06-30 | 2001-01-19 | Oki Electric Ind Co Ltd | Image reader |
JP3312729B2 (en) | 1999-07-14 | 2002-08-12 | 富士重工業株式会社 | Outside monitoring device with fail-safe function |
JP3272696B2 (en) | 1999-07-30 | 2002-04-08 | 富士重工業株式会社 | Outside monitoring device with fail-safe function |
-
2002
- 2002-04-25 JP JP2002124615A patent/JP3901573B2/en not_active Expired - Fee Related
-
2003
- 2003-04-23 US US10/421,083 patent/US7174041B2/en not_active Expired - Fee Related
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4933983A (en) * | 1986-02-14 | 1990-06-12 | Canon Kabushiki Kaisha | Image data reading apparatus |
US5596654A (en) * | 1987-04-20 | 1997-01-21 | Fuji Photo Film Co., Ltd. | Method of determining desired image signal range based on histogram data |
US5734740A (en) * | 1994-10-31 | 1998-03-31 | University Of Florida | Method for automated radiographic quality assurance |
US6215900B1 (en) * | 1997-02-21 | 2001-04-10 | Direct Radiography Corp. | Image data processing for digital x-ray detectors |
US6353674B1 (en) * | 1997-06-05 | 2002-03-05 | Agfa-Gevaert | Method of segmenting a radiation image into direct exposure area and diagnostically relevant area |
US20010008562A1 (en) * | 1997-08-28 | 2001-07-19 | Qualia Computing, Inc | Joint optimization of parameters for the detection of clustered microcalcifications in digital mammograms |
US6449390B1 (en) * | 1997-09-24 | 2002-09-10 | Canon Kabushiki Kaisha | Image processing apparatus and method therefor |
US6088473A (en) * | 1998-02-23 | 2000-07-11 | Arch Development Corporation | Method and computer readable medium for automated analysis of chest radiograph images using histograms of edge gradients for false positive reduction in lung nodule detection |
US20030095695A1 (en) * | 2001-11-21 | 2003-05-22 | Arnold Ben A. | Hybrid calibration of tissue densities in computerized tomography |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050083432A1 (en) * | 2003-10-15 | 2005-04-21 | Fujitsu Ten Limited | Image processing device, operation supporting device, and operation supporting system |
US7557858B2 (en) * | 2003-10-15 | 2009-07-07 | Fujitsu Ten Limited | Imaging processing device and a method of processing an image |
US20100332494A1 (en) * | 2009-06-25 | 2010-12-30 | Goetz Graefe | Histogram processing by trend and pattern removal |
US8768938B2 (en) * | 2009-06-25 | 2014-07-01 | Hewlett-Packard Development Company, L.P. | Histogram processing by trend and pattern removal |
US10154255B2 (en) * | 2014-01-10 | 2018-12-11 | Hitachi Automotive Systems, Ltd. | In-vehicle-camera image processing device |
US20160301923A1 (en) * | 2014-01-10 | 2016-10-13 | Hitachi Automotive Systems, Ltd. | In-Vehicle-Camera Image Processing Device |
US10026178B2 (en) * | 2014-04-18 | 2018-07-17 | Fujitsu Limited | Method of determining normality of imaging direction, and evaluation device of evaluating attachment state of imaging device |
US10027953B2 (en) | 2014-04-18 | 2018-07-17 | Fujitsu Limited | Determination method and determination apparatus |
US20150302588A1 (en) * | 2014-04-18 | 2015-10-22 | Fujitsu Limited | Method of determining normality of imaging direction, and evaluation device of evaluating attachment state of imaging device |
US10230946B2 (en) | 2014-04-18 | 2019-03-12 | Fujitsu Limited | Method of detecting tilt of image capturing direction and apparatus of detecting tilt of image capturing direction |
US10616570B2 (en) | 2014-04-18 | 2020-04-07 | Fujitsu Limited | Determination method and determination apparatus |
US10810728B2 (en) * | 2018-11-19 | 2020-10-20 | The Boeing Company | Method for using a vision system to evaluate shield trims on shielded cables |
CN111722074A (en) * | 2020-06-03 | 2020-09-29 | 四川蓝景光电技术有限责任公司 | LED identification label fault positioning method, device and system |
CN113701890A (en) * | 2021-08-09 | 2021-11-26 | 国网上海市电力公司 | Sensing monitoring method and system for tracking cable hot spot and damage |
Also Published As
Publication number | Publication date |
---|---|
US7174041B2 (en) | 2007-02-06 |
JP3901573B2 (en) | 2007-04-04 |
JP2003319120A (en) | 2003-11-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5801765A (en) | Scene-change detection method that distinguishes between gradual and sudden scene changes | |
US7174041B2 (en) | Diagnostic apparatus for abnormalities in image recognition system | |
US8035725B2 (en) | Multi-focus camera apparatus and image processing method and program used therein | |
US7218757B2 (en) | Method and device for detecting obstruction in image sensor systems | |
US8175334B2 (en) | Vehicle environment recognition apparatus and preceding-vehicle follow-up control system | |
CN100515041C (en) | Method for automatically controlling exposure and device for automatically compensating exposure | |
JP6571424B2 (en) | Fault diagnosis device | |
US9118838B2 (en) | Exposure controller for on-vehicle camera | |
CN107798688B (en) | Moving target identification method, early warning method and automobile rear-end collision prevention early warning device | |
CN111031254B (en) | Camera mode switching method and device, computer device and readable storage medium | |
US20040169745A1 (en) | Method and device for recognizing or displaying image defects in image recording systems | |
CN112312001B (en) | Image detection method, device, equipment and computer storage medium | |
KR101241803B1 (en) | Multi-camera video recording device and method for vehicle using multi-camera system | |
JP3134845B2 (en) | Apparatus and method for extracting object in moving image | |
US8818093B2 (en) | Method and device for analyzing an image of an image recording device for a vehicle | |
CN111832418A (en) | Vehicle control method, device, vehicle and storage medium | |
CN110121055B (en) | Method and apparatus for object recognition | |
JP2985558B2 (en) | Operating condition detection device | |
US20090322879A1 (en) | Method and device for the detection of defective pixels of an image recording sensor, preferably in a driver assistance system | |
CN112770021B (en) | Camera and filter switching method | |
abd el Azeem Marzouk | Modified background subtraction algorithm for motion detection in surveillance systems | |
JP3671460B2 (en) | Lamp lighting detection method and stop lamp lighting detection device | |
CN112419635A (en) | Perimeter alarm method integrating grating and video | |
JP4995555B2 (en) | Image processing device | |
CN117218109A (en) | Vehicle lateral mosaic image integrity detection method, system, equipment and medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FUJITSU TEN LIMITED, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHIMA, NOBUKAZU;REEL/FRAME:014004/0049 Effective date: 20030410 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20190206 |